Re: [Xen-devel] [PATCH 2 of 4 RFC] xl/remus: Network buffering cmdline switch, setup/teardown

[Shriram Rajagopalan <rshriram@xxxxxxxxx>]

>
>> > Also, I'm not sure how this is supposed to work when driver domains are
>> > in use either,
>>
>> Remus wont work with driver domains, unless there are agents in each of the
>> driver domains, that are co-ordinated by the memory checkpoint code in dom0.
>
> That doesn't seem to be to hard to arrange. We are already thinking of a
> "xenbackendd" daemon which runs hotplug scripts in both dom0 and domU in
> order to reduce our dependency on udev (in dom0 the daemon may actually
> be libxl directly).
>
>> Irrespective of the hotplug scripts, Remus needs to control the IFB
>> network interface
>> attached to the Guest's vifs/tap devices. Given that all these
>> interfaces will be inside
>> a driver domain, which does not have a fast (not xenstore)
>> communication channel to dom0,
>> there is no way the memory checkpointing code can co-ordinate with the
>> driver domain in order
>> to buffer/release its network packets after each checkpoint.
>
> It doesn't seem implausible that the toolstack and that daemon might not
> also negotiate a memory-checkpoint evtchn or something like that for a
> given domain.
>
>> One alternative would be to have a network agent running inside each
>> of these driver domains,
>> assuming that the driver domains would have network access (practical ? ).
>> The memory checkpoint code would have to control the IFB devices via
>> the network agents.
>>
>> All this is in the long run.. The immediate goal is to get
>> network/disk buffering to work with xl.
>
> We still need to consider the future when designing the libxl API since
> that is stable and we avoid changing it once committed (there are
> mechanisms to evolve the interface, see libxl.h, but they are best
> avoided).
>

Assuming that we push all of the remus setup into libxl, to ease the pain off
other toolstacks, then the external API to activate/deactivate remus
will not change.
(Also assuming that libxl.h is the libxl API you are talking about).

The API is already there. libxl_domain_remus_start with the remus_info
structure. So, even if we add driver domain support later, external
toolstacks would
not be impacted. I will certainly add stub code to handle driver
domains. However,
adding any piece of mechanism in there would be premature unless we know how
xenbackendd is going to work and the nature of communication channels.

>> If you are suggesting that we invoke a hotplug script when "xl remus"
>> command is issued,
>> I dont mind doing so either. The code in libxl (to control the plug
>> qdisc) is not going to go away
>> in either case.
>
> Would doing the setup in a shell script be easier/cleaner etc?
>

Neither. The current C-code (xl_netbuf) can do everything that a shell
script does.
However, if we were to invoke an external "hotplug" style script, it
would give us
more flexibility.
For e.g., people could tweak the script to add other traffic shaping
stuff onto the VM's traffic.  Or, with openvswitch/SDN, other fun
stuff could be done.

> I'm still not sure I understand why finding an ifb and binding it to a
> vif cannot be done by the vif hotplug script and the ifb communicated
> back to the toolstack via xenstore.
>

To make sure we are on the same page:
 I am under the assumption that the vif hotplug script is invoked
during the domain boot.
 I am also assuming that you are talking about existing vif hotplug
scripts (vif-setup, vif-bridge, etc).

 So, if we lock onto an IFB and bind it to the vif (i.e. redirect all
egress traffic via the IFB),
 then what we are basically doing is to acquire an IFB device for
every vif in the VM, for every VM,
 that may be run under Remus sometime in future.
 This basically means we would be setting up an IFB device for every
domain right from boot
 and not when Remus is started for the domain.

In case you mean something like a "vif-remus" hotplug script that is
invoked when one invokes
libxl_domain_remus_start API, then, you are right. all the setup can
be done in this hotplug script
and the chosen ifbs can be communicated via xenstore back to libxl.

Hope that clears things up.


shriram

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel