Re: Sketch of an idea for handling the "mixed workload" problem

[George Dunlap <george.dunlap@xxxxxxxxx>]

On Mon, Jan 22, 2024 at 12:17 PM Marek Marczykowski-Górecki
<marmarek@xxxxxxxxxxxxxxxxxxxxxx> wrote:
>
> On Mon, Jan 22, 2024 at 11:54:14AM +0000, George Dunlap wrote:
> > The other issue I have with this (and essentially where I got stuck
> > developing credit2 in the first place) is testing: how do you ensure
> > that it has the properties that you expect?
>
> Audio is actually quite nice use case at this, since it's quite
> sensitive for scheduling jitter. I think even a simple "PCI passthrough a
> sound card and play/record something" should show results. Especially
> you can measure how hard you can push the system (for example artificial
> load in other domains) until it breaks.

Are we going have a gitlab runner which says, "Marek sits in front of
his test machine and listens to audio for pops"? :-)

>
> > How do you develop a
> > "regression test" to make sure that server-based workloads don't have
> > issues in this sort of case?
>
> For this I believe there are several benchmarking methods already,
> starting with old trusty "Linux kernel build time".

First of all, AFAICT "Linux kernel bulid time" is not representative
of almost any actual server workload; and the end-to-end throughput
completely misses what most server workloads will actually care about,
like latency.

Secondly, what you're testing isn't the performance of a single
workload on an empty system; you're testing how workloads *interact*.
If you want ideal throughput for a single workload on an empty system,
use the null scheduler; more complex schedulers are only necessary
when multiple different workloads interact.

FWIW this was my first stab at trying to be systematic about testing
the scheduler:

https://github.com/gwd/schedbench

The rump kernel project has basically died AFAIK, so anyone trying to
resurrect this would probably have to try to rebase that bit of it
against something like XTF or unikernels.

 -George