Re: "The Kernel is the Problem, Not the Solution"

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

On 14 May 2013, at 09:48, Stanislav Sedov <stas@xxxxxxxxxxx> wrote:

> On Tue, 14 May 2013 09:07:23 +0100
> Anil Madhavapeddy <anil@xxxxxxxxxx> mentioned:
> 
>> Yeah, it's all the other stuff -- managing the iperf instances, making sure 
>> they're not all hogging CPU, splitting it across aliases, interfaces, 
>> bridges and VMs, etc.
>> 
>> All a bit tedious. Isn't there some magic load generator that already helps 
>> out with this sort of thing?
>> 
> 
> I can try to get access to the SwiftTest appliance (the company I worked
> for and run the test.  Do you want to run it agains Mirage in Xen?
> What kind of protocols do you want to test against?

While it would be very nice to have access to some 'big iron' tests,
what we really need right now is a lightweight way for developers to
interactively run these tests and fix the scalability bugs they find
along the way.

Balraj did this in the early days for particular aspects of TCP (by
manually looking at the window plots of traces and fixing scheduling
issues), but that's obviously not too sustainable.

I wouldn't be surprised at all to find many low-hanging scalability
problems from testing the Mirage-net stack against thousands (and then
millions) of connections.  So we do need a few shell scripts to automate
the VM launch and load generation of the clients and server.  Once the
issues from these are fixed, I would be very interested in using
SwiftTest as a regression test (i.e. once 10 million connections work,
they should stay working).

-anil