Re: [Xen-devel] [PATCH v1 1/4] xen: add real time scheduler rt

[Dario Faggioli <dario.faggioli@xxxxxxxxxx>]

On mer, 2014-09-03 at 18:18 +0100, George Dunlap wrote:
> On Wed, Sep 3, 2014 at 5:57 PM, Dario Faggioli
> <dario.faggioli@xxxxxxxxxx> wrote:

> > For instance, I can put, in an SMP guest, two real-time applications
> > with different timing requirements, and pin each one to a different
> > (v)cpu (I mean pin *inside* the guest). At this point, I'd like for each
> > vcpu to have a set of RT scheduling parameters, at the Xen level, that
> > matches the timing requirements of what's running inside.
> >
> > This may not look so typical in a server/cloud environment, but can
> > happen (at least in my experience) in a mobile/embedded env.
> 
> But to play devil's advocate for a minute here: 
>
Hehe, please, be my guest! :-D :-D

> couldn't you just put
> them in two different single-vcpu VMs then?
> 
Sure I can! Or not. :-)

What I mean is that, yes, for a whole bunch of people, "let's just use 2
VMs can be an answer". For others, it just can't, e.g., as Meng said, if
there are communication constraints or, perhaps, evenn just if one has
the application implemented like that already, running on a 2-way
baremetal system, and splitting would mean rewriting, instead of just
putting it inside a VM, as it usually happens with virtualization.

All this to say that, even if there exist (as always?) workarounds, this
is a feature I really --at some point, if not now-- would like to
support.

One more reason for that, that I at least wanted to cite, comes from the
so-called "real-time theory". Yeah, yeah, I know, academia people are
just crazy (right Meng! :-P), but from time to time, something that
makes sense comes out of there (ehm... like Xen? :-P).

So, you'd have to bear with me if I don't have all the details about the
involved math fresh in my mind (perhaps Meng can help here, if anyone is
interested), an if I reference a couple of research papers.
The key point is there are theoretically sound and mathematically proved
reasons for why, if you have a 2 vcpus VM and you want to give it 140%
pCPU bandwidth, it may be very bad idea to give 70% to each of its
vcpus.

The papers are these two here:
[1] A Hierarchical Multiprocessor Bandwidth Reservation Scheme
    with Timing Guarantees
    http://www.cs.unc.edu/~anderson/papers/ecrts08c.pdf
[2] Hierarchical Scheduling Framework for Virtual Clustering of
    Multiprocessors
    http://cps.kaist.ac.kr/papers/08ecrts-virtual-clustering.pdf

They (especially the first one) don't mention virtualization explicitly.
Rather, they talk about something called 'hierarchical scheduling'. Just
think about Xen scheduler al tier 0, and DomU scheduler as tier 1, of
the hierarchy, and here you have the analogy!

I find Example 2 (pages 3 and 4) in [1] particularly well suited to
explain the basic idea, while I guess Meng is a lot more familiar with
[2] (it comes right his institution! :-) )

It all comes from the fact that execution (of a real-time task inside a
VM) is sequential. Suppose I have that 2 vcpus VM, to which I need to
assign 140% of pCPU bandwidth, and inside of which there can be one or
more real-time task, together with some other _non_ real-time
activities.
The times that I have more than one rt task active contemporary, inside
the VM, they can exploit the parallelism the 2 vcpus provide them, so
(to certain extent) no big deal about the single vcpus' parameters. The
times I have *only* one rt task active, it either runs on vcpu1 or on
vcpu2. Well, If the two vcpus have 70% pCPU bandwidth each, it is more
likely for them to be scheduled together, with the rt task not being
able to exploit the parallelism, to the point that it may (depending
from its own bandwidth demand) miss its deadlines. If I give 100% to
vcpu1 and 40% to vcpu2, it is less likely for them to be scheduled
together (or, if you want, they will be scheduled together for less
time), and more likely for the rt task alone to fulfill its timing
constraints.

I can try to add more details and numbers to the example, if required/if
someone is interested.

Forgive the long digression, I could not resist. :-D

All this being said. If this scheduler goes in Xen 4.5 (which I think it
should), I'm fine with, especially at the libxl level, where we can't
change things, "take it easy", and, for example, leave the per-vcpu
parameters support as a future development (perhaps for Xen 4.6). I'm
just saying that I think it is an important part, that I'm happy the
scheduling algorithm and the hypervisor interface supports it and that I
want it to be available, _at_some_point_. :-)

> > In any case, even without that in place right now, I think different
> > parameters for different vcpus is certainly something we want from an RT
> > scheduler.
> 
> Yeah, the "expose parameters to guests" was just thinking out loud
> about what would be useful in the future.  
>
Yep.

> I think possibly allowing a
> VM to change its period (while keeping the budget / period ratio the
> same) might make sense as well; that way you could have an RT
> "appliance" VM that you could just pop onto a system and let it
> configure itself, without the user having to do more than give it
> basic parameters.
> 
Indeed. Definitely interesting as a future work!

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel