Re: [Xen-devel] Prioritising dom0 vcpus

[Dario Faggioli <dario.faggioli@xxxxxxxxxx>]

On ven, 2013-05-31 at 18:18 +0100, Marcus Granado wrote:
> As an experiment trying to reduce the latency when scheduling dom0 
> vcpus, I applied the following patch to __runq_insert() to xen 4.2:
> 
> diff -r 8643ca19d356 -r 91b13479c1a2 xen/common/sched_credit.c
> --- a/xen/common/sched_credit.c
> +++ b/xen/common/sched_credit.c
> @@ -205,6 +205,15 @@
>       BUG_ON( __vcpu_on_runq(svc) );
>       BUG_ON( cpu != svc->vcpu->processor );
> 
> +    /* if svc is a dom0 vcpu, put it always before all the other vcpus 
> in the runq,
> +     * so that dom0 vcpus always have priority
> +     */
> +    if (svc->vcpu->domain->domain_id == 0) {
> +        svc->pri = CSCHED_PRI_TS_BOOST; /* make sure no vcpu goes in 
> front of this one  until this vcpu is scheduled */
> +        list_add(&svc->runq_elem, (struct list_head *)runq);
> +        return;
> +    }
> +
>       list_for_each( iter, runq )
>       {
>           const struct csched_vcpu * const iter_svc = __runq_elem(iter);
> 
Mmm... Are we talking about wakeup latency --which, BTW, is what
TS_BOOST is all about, AFAIUI ?

In that case, isn't a waking vcpu, whether or not it belongs to Dom0,
being boosted already in csched_vcpu_wake()? __runq_insert() is called
right after that, so I think it sees the boosting already, without the
need of the above.

If it's not only wakeup latency issues that you're trying to address,
then I'm not sure, but still, __runq_insert() does not look the right
place where to place such logic, at least per my personal taste. :-)

> However, this patch seems to have had the opposite effect, and I would 
> like to understand why. A win7 guest now takes hours to start up, and I 
> believe this is due to dom0 taking an order of 10ms to serve each vm i/o 
> request, even though the dom0 vcpus and the guest vcpu are in different 
> pcpus.
> 
Well, just shooting in the dark, but __runq_insert() is also called in
csched_schedule(). Perhaps your modification above interacts badly with
the current scheduling logic?

Another way of trying to achieve what you seem to be up to, could be to
put an "is_it_dom0?" check in csched_vcpu_acct() and, if true, do not
clear the boosting. Beware, I'm not saying that it makes sense, or that
I like it, it just seems more clean (at least to me) than hijacking
__runq_insert().

What do you think?

> xenalyze-a.out: http://pastelink.me/getfile.php?key=390a25
> xentrace-D-T5.out: http://pastelink.me/getfile.php?key=b3d584
> 
Sorry, can't look at the traces right now... If I find 5 mins for them
and spot something weird, I'll let you know.

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel