[Xen-devel] Scheduler portability problem

["Magenheimer, Dan (HP Labs Fort Collins)" <dan.magenheimer@xxxxxx>]

I am working on Xen/ia64 changes (within Xen itself) to support
multiple domains and ran into the following problem:

It appears that __enter_scheduler was derived from an old version
of the Linux scheduler ("schedule()"), with some changes made for
simplification.  Many of the function names are the same but some
of the syntax and semantics have changed.  In particular, four
of note:

1) switch_to now takes two arguments instead of three, and
2) after switch_to is called, "other things" are done which
   utilize the "next" pointer
3) schedule_tail is passed the "next" task, rather than "prev"
4) schedule_tail is assumed to never return

I'm all for simplification if the Linux code is too complicated,
but in this case, some of the complexity is present to support
other architectures.  I can speak for ia64 but I suspect that
similar problems will occur with other non-x86 ports.

On Linux, switch_to is actually a macro and on ia64, another routine
is called which returns a value that is "passed back" in the
third switch_to argument.  Why?  Because switch_to actually does a task
switch and the world may be very different when it returns.
In particular, the values for prev and next are *different* when
it returns.  Why?  Because switch_to (at least on ia64) is the
key point where all of the current task state is put in memory,
stacks are changed, and the new task state is taken back out
of memory.  Actually, that's not quite accurate... at the point of
the call to switch_to, a fair amount of state has *already* been
put in memory in both the memory stack and the register stack.
The only way to restore this state (short of some very complex
stack analysis) is to exit each routine in the call stack the same
way as it was called.

So, on Linux, after the call to switch_to, "next" is no longer
valid and is not used.  "Prev" is used only because of the third
argument macro trick, and "current" has already been changed to
point to the new task.

On Xen/x86, it appears schedule_tail never returns because some cool
assembly tricks are used to jump directly to the right place,
basically as if throwing an exception (I'm guessing because there is no
useful state on the call stack on x86).  As previously noted, this is
problematic on ia64.

Bottom line: The current code in __enter_scheduler() does not easily
accommodate other architectures.  I'll be taking a look at what it
will take to "fix" it, but wanted to open discussion first.  I know
there are some that will say "just change the ia64 code"... because
of architectural constraints, this is far FAR more easily said than
done.  And there are some that will say that mimicking Linux is
a mistake because XINL (Xen is not Linux).  However, I believe this
is a case where leveraging the many many years of experience on many
many architectures (with said experience only documented in the code
itself) of Linux will benefit Xen portability in the long run (and,
in my case, in the short run).

Comments?

Thanks,
Dan


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id396&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/xen-devel