Re: [Xen-users] 3.2 changes network layout?

[Freddie Cash <fjwcash@xxxxxxxxx>]

On June 5, 2008 04:22 pm jim burns wrote:
> On Thursday June 05 2008 07:09:07 pm Freddie Cash wrote:
> > So, how does one configure eth0 to have an IP, and eth1 to be the
> > bridge, without any IP configuration occuring on eth1 or the bridge?
>
> On standard xen, if you have two physical nics eth0 & eth1, bridges
> will be created for them, and the renamed devices peth0 & peth1 will be
> enslaved to the corresponding bridge, and *something* on those bridges,
> or the bridge itself, will have an ip.

Ah, but see, that's the crux:  I don't want a bridge created on eth0, and 
I don't want an IP on eth1.

With Xen 3.0.x, this was simple, and easy to do.  Just write a wrapper for 
network-bridge,  and in there set vifnum=X netdev=ethX and bridge=xenbrX 
(where X is 1 and above), then in the VM config file, set bridge=xenbrX 
to assign each VM to the specified bridge.

With Xen 3.1.x and 3.2.x, this is no longer possible (at least I could 
never get it to work).  The default network-bridge checks for an active 
interface with an IP and uses that as the default bridge.  Well, on our 
servers, eth0 is a 10/100 NIC, eth1 through eth6 are 10/100/1000 NICs.  
We don't want to use eth0 for anything but management traffic.

network-bridge doesn't use netdev=, bridge=, or vifnum= in any way, shape, 
or form.  You can't write a wrapper for it like you could with Xen 3.0.x.  
And trying to do it manually doesn't work either as the way the bridge is 
setup in the dom0 is hokey to say the least and doesn't work for 
interfaces without IPs.

The only way I could get things to even slightly work in Xen 3.2 was to 
for udev to rename my NICs to number them in reverse, making the 10/100 
port eth6.  But, even then, I couldn't get more than a single bridge to 
come up in the dom0, and only if I assigned a ficticious IP to that 
interface first.

> If I read you correctly, you want a non-standard setup. Assuming you
> don't have a physical nic eth1, you can create any bridge you want, and
> say call it eth1, and enslave eth0 (w/ an ip) and peth0 (to do the
> actual hardware xfers) to it. I believe you can do this with just a
> simple 'bridge=name netdev=eth0' clause in your xend-config.sxp.

Don't know how non-standard it is to want a management NIC with an IP, and 
a bridge without an IP that the domUs will use.  Seems perfectly 
reasonable to me, especially since it worked so nicely in Xen 3.0.x (and 
is very simple to do with KVM).

What would be ideal (and is also something that never worked in Xen) would 
be to have eth0 be the management NIC, and then bond together eth3 
through eth6 as bond0 and then us that as the bridge in the dom0 that all 
the domUs would use.  But, I gave up on that after a few days as I could 
either have traffic to the dom0 or to the domUs, but not both.

> If I have misinterpreted what you need, you will probably need more of
> a brctl or /etc/sysconfig/network-scripts guru than me :-)

That's fine.  I've resigned myself to using Xen 3.0.x with kernel 2.6.18 
on our older systems (Tyan Thunder K8SD-Pro boards with dual-Opteron 
200s) and KVM on newer systems (Tyan h2000M with 2x dual-core Opteron 
2200s).  At least on those setups, networking works correctly.

-- 
Freddie Cash
fjwcash@xxxxxxxxx

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users