Re: [Xen-devel] [RFC PATCH v1 00/16] xen: sched: implement core-scheduling

[Tamas K Lengyel <tamas.k.lengyel@xxxxxxxxx>]

On Thu, Oct 18, 2018 at 2:16 AM Dario Faggioli <dfaggioli@xxxxxxxx> wrote:
>
> On Wed, 2018-10-17 at 15:36 -0600, Tamas K Lengyel wrote:
> > On Fri, Aug 24, 2018 at 5:36 PM Dario Faggioli <dfaggioli@xxxxxxxx>
> > wrote:
> > >
> > > They give me a system that boots, where I can do basic stuff (like
> > > playing with dom0, creating guests, etc), and where the constraint
> > > of
> > > only scheduling vcpus from one domain at a time on pcpus that are
> > > part
> > > of the same core is, as far as I've seen, respected.
> > >
> > > There are still cases where the behavior is unideal, e.g., we could
> > > make a better use of some of the cores which are, some of the
> > > times,
> > > left idle.
> > >
> > > There are git branches here:
> > >  https://gitlab.com/dfaggioli/xen.git rel/sched/core-scheduling-
> > > RFCv1
> > >  https://github.com/fdario/xen.git rel/sched/core-scheduling-RFCv1
> > >
> > > Any comment is more than welcome.
> >
> > Hi Dario,
> >
> Hi,
>
> > thanks for the series, we are in the process of evaluating it in
> > terms
> > of performance. Our test is to setup 2 VMs each being assigned enough
> > vCPUs to completely saturate all hyperthreads and then we fire up CPU
> > benchmarking inside the VMs to spin each vCPU 100% (using swet). The
> > idea is to force the scheduler to move vCPUs in-and-out constantly to
> > see how much performance hit there would be with core-scheduling vs
> > plain credit1 vs disabling hyperthreading. After running the test on
> > a
> > handful of machines it looks like we get the best performance with
> > hyperthreading completely disabled, which is a bit unexpected. Have
> > you or anyone else encountered this?
> >
> Do you mean that no-hyperthreading is better than core-scheduling, as
> per this series goes?
>
> Or do you mean that no-hyperthreading is better than plain Credit1,
> with SMT enabled and *without* this series?
>
> If the former, well, this series is not at a stage where it makes sense
> to run performance benchmarks. Not even close, I would say.

Understood, just wanted to get a rough idea of how much it effects it
in the worst case. We haven't got to actually run the tests on
core-scheduling yet. On my laptop Xen crashed when I tried to create a
VM after booting with sched_smt_cosched=1. On my desktop which has
serial access creating VMs works but when I fired up swet in both VMs
the whole system froze - no crash or anything reported on the serial.
I suspect a deadlock because everything froze
display/keyboard/serial/ping. But no crash and reboot.

> If the latter, it's a bit weird, as I've often seen hyperthreading
> causing seemingly weird performance figures, but it does not happen
> very often that it actually slows down things (although, I wouldn't
> rule it out! :-P).

Well, we ran it on at least 4 machines thus far (laptops, NUC,
desktops) and it is consistently better, and quite significantly. I
can post our detailed results if interested.

>
> Of course, this could also be a problem in the scheduler. Something I'd
> do is (leaving this series aside):
> - try to run the benchmark with half the number of vCPUs wrt the total
> number of CPUs (i.e., with only 1 vCPU per core);
> - try to run the benchmark as above, but explicitly pinning 1 vCPU on
> each core;
> - go back to nr vCPUs == nr pCPUs, but explicitly pin each vCPU to one
> pCPU (i.e., to one thread, in this case).
>
> You can do all the above with Credit, and results should already tell
> us whether we may be dealing with some scheduling anomaly. If you want,
> since you don't seem to have oversubscription, you can also try the
> null scheduler, to have even more data points (in fact, one of the
> purposes of having it was exactly this, i.e., making it possible to
> have a reference for other schedulers to compare against).

We do oversubscribe. Say we have 4 pCPUs, 8 cores showing with
hyperthread enabled. We run 2 VMs each with 8 vCPUs (16 total) and in
each VM we create 8 swet processes each spinning at 100% at the same
time. The idea is to create a scenario that's the "worst case", so
that we can pinpoint the effect of the scheduler changes. With pinning
or fewer vCPUs then hyperthreads I believe it would be harder to spot
the effect of the scheduler changes.

>
> BTW, when you say "2 VMs with enough vCPUs to saturate all
> hyperthreads", does that include dom0 vCPUs?

No, dom0 vCPUs are on top of those but dom0 is largely idle for the
duration of the test.

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel