Re: [Xen-users] Finally moving to xen 3

["Molle Bestefich" <molle.bestefich@xxxxxxxxx>]

Frank DiRocco wrote:
> I set up a debian sarge machine with one network interface configured via 
> dhcp.
> If I understand corectly, first eth0 is started by system boot process.
> It gets its address via dhcp. Then xend creates xenbr0, I may be mistaken but 
> I
> have compared xenbr0 to a switch in my mind.

No mistake, they're conceptually the same, a layer 2 (MAC address
based) switching device.

In Linux, when you add a network interface to a bridge, that network
interface stops delivering packets to the kernel and the kernel's
TCP/IP stack and instead starts giving incoming packets to the bridge.
 You can think of the interface as suddenly becoming a physical port
in a hardware switch, instead of being a physical port in the Linux
box.  The interface is also set to listen to all networking traffic
and not just that destined for it's own MAC address (promiscuous
mode).  The bridge then forwards the packets it receive from this
interface to the other "ports" on the bridge, again acting as a
hardware switch would.

Since the kernel no longer receives packets in "the ordinary fashion"
from the network interfaces that you've added to a bridge, you will
seem to loose network connectivity from the Linux box.  Luckily, as is
the case with some hardware switches too, you can assign a "management
IP address" to the bridge.  There is no special bridge management
software listening on this IP address, instead packets destined to it
are passed through the kernel's regular incoming packet logic.  Thus
you will be able to reach the Linux box itself by assigning a
management IP address to the bridge.  Doing this is rather simple,
because the bridging device shows up as an ethernet interface (eg.
xenbr0) in ifconfig.  I believe this is a normal way to set up a Linux
bridge.

With Xen, the story is a bit different.  Xen has the concept of (let's
call it) virtual wire which connects two virtual network interfaces. 
With this, you can add one of the virtual interfaces to the bridge and
have the other one act as a vanilla network interface.  You can
therefore skip the whole management IP address dance, and as a bonus
your setup will reminisce a physical setup with a hardware switch and
a hardware dom0 box.

That's all fine and dandy.  Except my personal experience is that it
doesn't work though.  It breaks the network init scripts of my distro
(Gentoo), so /etc/rc.inetd/net.ethX up/down no longer works.  Anybody
with other distros, please shout out your experiences.  Also it seems
that the network-bridge script (which does the bridge setup for Xen)
has to hack the kernel IP routing tables to make things work, which
seems like a messy thing to do.  Anybody know why this is needed,
please speak up.

> Now eth0 is brought down. The IP and MAC from the "real"
> eth0 are now copied to veth0. The now down "real" eth0 is renamed peth0.

Which will later be added to the bridge..

> The veth0 which is a clone of the "real" eth0 is renamed to eth0.

Agreed..
I seem to remember having one box where something was amiss - I ran
"ifconfig -a" on it, and *both* veth0 and eth0 were present.  Not sure
what went wrong, but maybe it's something to look out for.

> So, my dom0 should think the "new" eth0 is the original eth0
> and use it for network traffic.

> I think I understand the theory behind the way the virtual networking
> works, but when I log in and ifconfig I see peth0 vif0.0 xenbr0 but I don't
> see eth0 and networking doesn't werk.  I have tried ifconfig eth0 up and I do
> get an address via dhcp,

Ok, so if 'ifconfig eth0' works, at least the interface exists.

> but still have no network access. It was my understanding
> that the "new" eth0 would be brought up automagicly, but

It should have been.

> this does not seem to be the case with my mess.

I'd start by disabling any firewall(s), fx. iptables.

Next, are your IP routing tables correctly defined?  You can "netstat
-ran" and post the output to the list if you want more eyes on it.

Assuming they are, try tcpdump.  Tcpdump can listen on an interface
and tell you what traffic passes by, so if you fx. "tcpdump -i eth0",
and then ping some IP external to your box from dom0, you should be
able to see whether the ICMP packets passes the "virtual wire" between
vif0.0 and over to eth0.  Similarly, "tcpdump -i xenbr0" should tell
you whether the packets reach the bridge and "tcpdump -i peth0" will
tell you if they reach the physical interface.

Then you'll have to trace the return packets.  Make sure they hit the
external interface [check promiscuous mode], enter the bridge, leave
the bridge and end up in vif0.0.

I realize that this was a quite verbose posting and that I didn't
exactly provide the solution that you're looking for.  Sorry - I hope
the input is helpful anyway!

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users