Xen project Mailing List

On Thu, Jul 18, 2013 at 11:02 AM, George Dunlap <george.dunlap@xxxxxxxxxxxxx> wrote:

On 18/07/13 15:51, Shriram Rajagopalan wrote:

On Thu, Jul 18, 2013 at 9:33 AM, Ian Campbell <Ian.Campbell@xxxxxxxxxx> wrote:

On Thu, 2013-07-18 at 09:26 -0400, Shriram Rajagopalan wrote:

> Right, and that's why I would prefer to avoid a dependency on
> Python.
> Since I think at least some of these projects will see it as
> an
> additional barrier.
>
> On the other hand if its just an implementation detail of a
> remus
> specific script which libxl happens to call out to when asked
> then I
> suppose it is up to the Remus folks whether they find this
> acceptable.
>
>
>
> correction. This script will be called by xl not libxl. As George put
> it, other toolstacks
> may choose to do this setup in their own way.

Actually I think this is exactly the sort of complexity which libxl
serves to remove from all toolstacks. If they all need to do it then it
belongs in libxl.

> system("modprobe ifb numifbs=10")

Only this can go into the host config toolstack.

I'm not sure what you mean by "host config toolstack"; but the idea is to treat this configuration just like the bridging or OVS -- one thing set up at the beginning which the toolstack doesn't bother about.

Yep. thats what I meant.

> system("ip link set ifbX up")

the ifbX is an example.. the ifb module names the interfaces ifb0 to ifbN where N is

determined by modprobe ifb numifbs=N.

Now, lets say we have 10 ifbs in the system. which ones do we pick for the guest ?

if the VM has 3 interfaces, we need 3 ifbs, ifb0-2. Thats easy.

What if there are two remus streams in the same system ? then we need to maintain

a list of ifbs that are being used and which ones are free. The current remus python code

(tools/python/xen/remus/device.py) has some code to do this ("class Netbufpool").

Something similar needs to go into libxl.

This is exactly like assigning vif numbers to guests, isn't it?

Unfortunately no. the IFB names are fixed. If I load the IFB module during "physical" host boot, with

modprobe ifb numifbs=20,

the module creates a set of 20 ifb devices apriori. No dynamic creation/deletion like vifs.

and no ability to control the naming (they are static: ifb0, ifb1, ... ifb19). May be we could alias it but that

doesnt solve the ifb-pool requirement.

These two should be part of the required host configuration I think.
Along the same lines as how we pushed general host networking setup out
of the toolstack and into the administrators capable hands, e.g
http://wiki.xen.org/wiki/Network_Configuration_Examples_%28Xen_4.1%2B%29

IOW just document it, same as we document "create xenbr0"

> system("tc filter add dev vif1.0 ingress")
> system("tc filter add dev vif1.0 parent ffff: proto ip pref 10 u32
> match u32 0 0 action mirred egress redirect dev ifbX")

These should be part of the existing vif hotplug scripts (called from
libxl), shouldn't they?

They don't belong in the vif-hotplug script.

Adding these lines means that all egress traffic from the VM will be routed via the IFB device

whether or not Remus is running. I don't think people would want that.

No, the idea is that if an ifbX argument is present, you use it; otherwise not. Since this needs to be done exactly once per interface before it can be considered fully set up, the vif script is the natural place to put it.

I see your point. I was just leaning towards doing this setup (i.e. ingress filtering and traffic

redirection to IFB device) only when staring Remus and not when starting the domain.

But as you said, if the user specifies that he wishes that interface output to be buffered,

we might as well do this setup in the vif script itself. It incurs very little overhead without Remus anyway.

Perhaps based on a new vif parameter to specify
the ifbX

See previous explanation on finding a free IFB device.

If we throw the responsibility of specifying IFB devices onto the admin, libxl can basically

do two system() calls, as stated above and install the plug_qdisc and move on.

If we throw that to the admin it's a royal pain to use. This can easily be done by a computer; an admin has better things to spend his mental energy on.

I agree. Which is why that netbufpool code was added to Remus long time back.

Since the ifbs have to be allocated dynamically from the pool, here is what I suggest:

in the domain config:

vifs = ['mac=xxxxx,bridge=yyyy,buffer=yes']

in the hotplug script, we can allocate an ifb from the ifb pool and assign an IFB to this vif

and put this mapping into xenstore.

xl can later read this mapping, and do the rest when remus is enabled.

This would allow the user to specify selectively the set of interfaces on which

output buffering should be installed.

-George

Re: [Xen-devel] xl remus - Invoking scripts from xl