Re: [Xen-users] Finally moving to xen 3

["Frank DiRocco" <ofanged1@xxxxxxxxx>]

OK, so to be sure First I set up a new Virtual machine in vmware on a
private network with the host. Setup Xen3.0.1 as described here:
http://www.debian-administration.org/articles/304
additionally I installed tcpdump on both the VMWareVM and the Host running VMWare.
the VMWareVM running xen3.0.1 received a dhcp address 172.16.40.201 from the gateway 172.16.40.1
at this time dom0 has eth0 172.16.40.201 vif0.0 xenbr0 and peth0 up as per ifconfig.

so as if following a flowchart for automotive electrical repair i begin with.

                 router
            172.16.40.1
                    
|   
                   
+--------gamerTESTmachine 172.16.40.220
                     |
                     |
   eth0-------(cloud)---------peth0--------vif0.0---------eth0
     | 
                         
|_____________________|
    
|                                                       
|
172.16.40.3                                  
172.16.40.201
 
Suse10                            
Debian 3.1 sarge w/ Xen3.0.1
                                                    
VMWareVM

1) 
>tcp -i xenbr0
which results in
"who-has 172.16.40.201 tell 172.16.40.3"
"reply 172.16.40.201 is-at 00:0c:29:f6:fb:f9"
(note: i pinged 172.16.40.201 from another vm which has network access
and successfully pings and ssh to and from the router host.)

2)
>tcp -i vif0.0

which results in

"who-has 172.16.40.201 tell 172.16.40.3"

"reply 172.16.40.201 is-at 00:0c:29:f6:fb:f9"

(note: i pinged 172.16.40.201 from another vm which has network access
and successfully pings and ssh to and from the router host.)

3)
>tcp -i eth0

which results in nothing

(note: i pinged 172.16.40.201 from another vm which has network access
and successfully pings and ssh to and from the router host.)

At this point this is what I have mapped
         o---------------------------------------->|stops here
    (cloud)---------peth0--------vif0.0---- x ----eth0

The above test listens on the xen machine while another machine which can make a successful network connection send packets

The following will test for network activity in the reverse direction.

1)
>tcpdump -i vif0.0
which results in
"who-has 172.16.40.3 tell 172.16.40.201"

(note: i pinged 172.16.40.3 from the VMWareVM running Xen3.0.1)

2)
>tcpdump -i xenbr0

which results in

"who-has 172.16.40.3 tell 172.16.40.201"


(note: i pinged 172.16.40.3 from the VMWareVM running Xen3.0.1)
(note: additionally i received "who-has 172.16.40.1 tell 172.16.40.220"
this is another machine on my network so I think xenbr0 is properly in
promiscuous mode)

Now this is what my map looks like

         o---------------------------------------->|stops here

    (cloud)---------peth0--------vif0.0---- x ----eth0

         |<--------------------------------------------------o

    xenbr0---------peth0--------vif0.0------------eth0

Finally, I need to test outbound from the VMWareVM to the LAN
>tcpdump -i eth0    (this was run from 172.16.40.3 a Suse10 VM)


which results in


"who-has suse-test01.leaptech.local tell 172.16.40.201"



(note: i pinged 172.16.40.3 from the VMWareVM running Xen3.0.1)

Adding this to my map
       |<-----------------------------------------------------------------------------o


    eth0------(LAN)-----xenbr0---------peth0--------vif0.0-----------eth0

unfortunatly, no pings make it back. at this point i'm stumped and can
only deduce my problem may exist, at least partially, between vif0.0
and eth0 on the Xen machine.

I am sorry for the verbose post. If your post was not as detailed as it is, I probably would not be here.    : - D

On 3/26/06, Molle Bestefich <molle.bestefich@xxxxxxxxx> wrote:

Frank DiRocco wrote:
> I set up a debian sarge machine with one network interface configured via dhcp.
> If I understand corectly, first eth0 is started by system boot process.
> It gets its address via dhcp. Then xend creates xenbr0, I may be mistaken but I
> have compared xenbr0 to a switch in my mind.

No mistake, they're conceptually the same, a layer 2 (MAC address
based) switching device.

In Linux, when you add a network interface to a bridge, that network
interface stops delivering packets to the kernel and the kernel's
TCP/IP stack and instead starts giving incoming packets to the bridge.
You can think of the interface as suddenly becoming a physical port
in a hardware switch, instead of being a physical port in the Linux
box.  The interface is also set to listen to all networking traffic
and not just that destined for it's own MAC address (promiscuous
mode).  The bridge then forwards the packets it receive from this
interface to the other "ports" on the bridge, again acting as a
hardware switch would.

Since the kernel no longer receives packets in "the ordinary fashion"
from the network interfaces that you've added to a bridge, you will
seem to loose network connectivity from the Linux box.  Luckily, as is
the case with some hardware switches too, you can assign a "management
IP address" to the bridge.  There is no special bridge management
software listening on this IP address, instead packets destined to it
are passed through the kernel's regular incoming packet logic.  Thus
you will be able to reach the Linux box itself by assigning a
management IP address to the bridge.  Doing this is rather simple,
because the bridging device shows up as an ethernet interface (eg.
xenbr0) in ifconfig.  I believe this is a normal way to set up a Linux
bridge.

With Xen, the story is a bit different.  Xen has the concept of (let's
call it) virtual wire which connects two virtual network interfaces.
With this, you can add one of the virtual interfaces to the bridge and
have the other one act as a vanilla network interface.  You can
therefore skip the whole management IP address dance, and as a bonus
your setup will reminisce a physical setup with a hardware switch and
a hardware dom0 box.

That's all fine and dandy.  Except my personal experience is that it
doesn't work though.  It breaks the network init scripts of my distro
(Gentoo), so /etc/rc.inetd/net.ethX up/down no longer works.  Anybody
with other distros, please shout out your experiences.  Also it seems
that the network-bridge script (which does the bridge setup for Xen)
has to hack the kernel IP routing tables to make things work, which
seems like a messy thing to do.  Anybody know why this is needed,
please speak up.

> Now eth0 is brought down. The IP and MAC from the "real"
> eth0 are now copied to veth0. The now down "real" eth0 is renamed peth0.

Which will later be added to the bridge..

> The veth0 which is a clone of the "real" eth0 is renamed to eth0.

Agreed..
I seem to remember having one box where something was amiss - I ran
"ifconfig -a" on it, and *both* veth0 and eth0 were present.  Not sure
what went wrong, but maybe it's something to look out for.

> So, my dom0 should think the "new" eth0 is the original eth0
> and use it for network traffic.

> I think I understand the theory behind the way the virtual networking
> works, but when I log in and ifconfig I see peth0 vif0.0 xenbr0 but I don't
> see eth0 and networking doesn't werk.  I have tried ifconfig eth0 up and I do
> get an address via dhcp,

Ok, so if 'ifconfig eth0' works, at least the interface exists.

> but still have no network access. It was my understanding
> that the "new" eth0 would be brought up automagicly, but

It should have been.

> this does not seem to be the case with my mess.

I'd start by disabling any firewall(s), fx. iptables.

Next, are your IP routing tables correctly defined?  You can "netstat
-ran" and post the output to the list if you want more eyes on it.

Assuming they are, try tcpdump.  Tcpdump can listen on an interface
and tell you what traffic passes by, so if you fx. "tcpdump -i eth0",
and then ping some IP external to your box from dom0, you should be
able to see whether the ICMP packets passes the "virtual wire" between
vif0.0 and over to eth0.  Similarly, "tcpdump -i xenbr0" should tell
you whether the packets reach the bridge and "tcpdump -i peth0" will
tell you if they reach the physical interface.

Then you'll have to trace the return packets.  Make sure they hit the
external interface [check promiscuous mode], enter the bridge, leave
the bridge and end up in vif0.0.

I realize that this was a quite verbose posting and that I didn't
exactly provide the solution that you're looking for.  Sorry - I hope
the input is helpful anyway!

-- 
Thank you,
Frank  Di Rocco

"Does an optimistic person look at a hard drive as half-full or half-empty?" -ofanged1-at-gmail.com
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users