Re: [Xen-API] Kronos: bonded interfaces

[Jonathan Ludlam <Jonathan.Ludlam@xxxxxxxxxxxxx>]

On 20 Dec 2011, at 22:38, Dave Whitla wrote:

> Thanks Jon.
> 
> I was thinking about doing it myself. I'll take a look at what Rob has done 
> so far.
> My only concern is that it will likely make assumptions about SYSV init being 
> the principal run control system and that it's OK to take interfaces down.

If you enable the daemon as it's currently being written, it will own the 
configuration of networking on the box, and will indeed not play nicely with 
other things. The primary motivation for this is to make things work for XCP 
the distro - the embedded platform. It coincidentally will fix the problem that 
none of the current networking scripts will work with debian, but it will still 
be of the mindset that it is the sole thing that fiddles with networks on the 
system. The concession for other use cases is currently the ability to turn it 
off :-)

> I prefer my Pacemaker cluster solution (which I have invested a lot of time 
> in) to XCP's reinvention of all the wheels. In this scenario Pacemaker will 
> detect networks going down as a system failure and will fence the host unless 
> I'm provided a way to integrate the two.

The daemon is using json rpc to talk to xapi. It shouldn't be too hard to write 
a different one, though the obvious problem there is lack of documentation. I 
recommend talking to Rob about it for now, or asking on the IRC channel. We've 
got a nice IDL defining these interfaces (nice if you like ocaml, that is), but 
producing documentation from it isn't yet sorted. It's on the to-do list.

> So a robust solution really needs to be modular and pluggable if you want to 
> support simply adding xapi to existing platforms.
> 

Yes.

> Incidentally I've noticed that when the service names were changed to be 
> prefixed with xcp- none of the supporting libraries of scripts were updated.
> Is there some way we can get this stuff out in he open where I can check out 
> and iteratively fix-build-deploy-test.

As I said before, xapi-autobuilder is how we're building this stuff, but it's 
all normal debian packages. apt-get source xcp-xapi, and fire off the patches 
:-)

> I'm really under the pump to get this stuff working in our own deployment and 
> I don't have time to sit around and wait for other people to fix stuff which 
> I can do myself.
> If you can't check out the source and fix bugs then it's not really open 
> source.
> 
> Dave
> 
> On 21/12/2011, at 1:50 AM, Jonathan Ludlam wrote:
> 
>> The problem is that the network scripts that are used in XCP are designed to 
>> work with Centos style ifcfg scripts, not /etc/network/interfaces. The 
>> script /usr/lib/xcp/lib/interface-reconfigure was no-opified in order to 
>> prevent it screwing up anything.
>> 
>> Rob Hoes is working on a new platform independent network control daemon 
>> that will replace this mechanism for XCP/XenServer - we'll provide an option 
>> to use it with Debian/Ubuntu too.  Unfortunately it's not quite ready to go 
>> yet, but he's feverishly working on it, ETA before the end of the year I 
>> hope!
>> 
>> You can check his progress here: https://github.com/robhoes/xen-api/tree/net
>> 
>> Jon
>> 
>> On 20 Dec 2011, at 06:28, Dave Whitla wrote:
>> 
>>> Update:
>>> 
>>> /usr/lib/xcp/lib/interface-reconfigure is a no-op.
>>> And the reason it fails silently is (from 
>>> /usr/lib/xcp/bin/xe-reset-networking):
>>> 
>>> os.system(interface_reconfigure + if_args + ' >/dev/null 2>/dev/null')
>>> 
>>> 
>>> On 20/12/2011, at 4:05 PM, Dave Whitla wrote:
>>> 
>>>> After much pain I managed to get a bonded management interface on my pool 
>>>> master.
>>>> I then, following the advice of the XenServer 6.0 Administrator's Guide, 
>>>> configured the bonded interface for VM traffic before adding any pool 
>>>> slaves.
>>>> I then cleanly installed two machines (Debian Wheezy + Open vSwitch + xcp 
>>>> packages from the xcp unstable repo).
>>>> When I joined the new slaves to the pool the bonds were reversed bond0 was 
>>>> attached to the wrong network as xapi1 instead of xapi0 so I was unable to 
>>>> make bond0 the management interface as it has no IP.
>>>> I've now tried every possible combination over the last 24 hours to get 
>>>> this working.
>>>> 
>>>> None of the network related xe tools work - they all barf with exceptions.
>>>> The Xencenter UI won't let me delete networks, bonds or anything else.
>>>> I can't remove the broken machines from the pool.
>>>> The whole things a bit of a mess.
>>>> 
>>>> Has anyone tested bonded networking?
>>>> Incidentally, in arriving at my present situation I discovered that the 
>>>> bridge names xenbr# and xapi# are hard-coded somewhere so it is necessary 
>>>> to create bridges of the same same in /etc/network/interfaces.
>>>> Why a bridge is required for the management interface at all beats me.
>>>> xe host-management-reconfigure never works because the interface you are 
>>>> switching to can't possibly have an IP yet - because what you are trying 
>>>> to do is migrate the IP from eth0 to bond0 (xenbr0 to xapi0 is required by 
>>>> xapi).
>>>> It would seem as though some library behind xe is silently failing to 
>>>> reconfigure host networking correctly.
>>>> Being fluent in about a dozen programming languages, but none of them 
>>>> OCaml, I have no idea where to start looking.
>>>> 
>>>> I'd be very grateful for any assistance.
>>>> Sorry if this sounds more like a whine than a plea for help - I'm frazzled 
>>>> after 24 hours straight beating my head against the desktop.
>>>> 
>>>> Dave
>>>> _______________________________________________
>>>> xen-api mailing list
>>>> xen-api@xxxxxxxxxxxxxxxxxxx
>>>> http://lists.xensource.com/mailman/listinfo/xen-api
>>> 
>>> 
>>> _______________________________________________
>>> xen-api mailing list
>>> xen-api@xxxxxxxxxxxxxxxxxxx
>>> http://lists.xensource.com/mailman/listinfo/xen-api
>> 
> 


_______________________________________________
xen-api mailing list
xen-api@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/mailman/listinfo/xen-api