Re: [Xen-users] network problems

[Mike Lovell <mike@xxxxxxxxxxxx>]

Mike Lovell wrote:
> So, I have been running into network problems for a while on 4 boxes 
> that I installed xen on so that some engineers have places to test 
> code. This particular problem is happening on all 4 of these boxes. 
> (although, it isn't happening on an older box running xen from debian 
> etch).
> What appears to be the problem is that traffic is getting dropped 
> between the vif#.0 interface in dom0 and the eth0 interface in the 
> guest. To find this out, I started a ping flood from one domU that was 
> pinging another domU. About every 10 minutes, there will be a lot of 
> ping requests going out but no replies coming back. I think it really 
> weird that it happens like ever 10 minutes plus about 2 seconds. While 
> the ping was going, I did tcpdumps from the domU starting the ping, 
> from the vif#.0 of the pinging machine, from the virtual bridge, from 
> the vif#.0 for the receiving guest, and then from the receiving domU. 
> The packets are making it all the way to the dom0 vif for the 
> receiving guest but not making it to the eth0 in the guest. I have no 
> clue why this is happening and it happens in rather regular intervals. 
> The same thing happens in pinging a different guest and it happens in 
> about the same interval but at different times. Also, during the ping 
> flood, there is never a pause in the sending of packets out of the 
> guest. Only a pause on the packets going from the host to the guest.
> I am running this on 64 bit Debian Lenny using the distribution's 
> packages. xen-hypervisor-3.2-1-amd64 version 3.2.1-2 and  
> linux-image-2.6.26-2-xen-amd64 version 2.6.26-17. Here are the 
> networking configs.
> ---------
> dom0# cat /etc/network/interfaces
> # The loopback network interface
> auto lo
> iface lo inet loopback
>
> # The primary network interface
> auto eth0
> iface eth0 inet static
>    address 10.135.7.34
>    netmask 255.255.255.224
>    network 10.135.7.32
>    broadcast 10.135.7.63
>    gateway 10.135.7.33
>    # dns-* options are implemented by the resolvconf package, if 
> installed
>    dns-nameservers 10.135.7.34
>    dns-search qa1.mozyops.com
>
> auto vmnet
> iface vmnet inet static
>        address 10.135.2.71
>        netmask 255.255.255.224
>        bridge_ports eth1
> #        bridge_stp off
> #        bridge_fd 9
> #        bridge_hello 2
> #        bridge_maxage 12
>
> ---------
> DomU# cat /etc/network/interfaces
> # The loopback network interface
> auto lo
> iface lo inet loopback
>
> # The primary network interface
> auto eth0
> iface eth0 inet dhcp
>   post-up ethtool -K eth0 tx off
>
> ---------
> Dom0# brctl show vmnet
> bridge name    bridge id        STP enabled    interfaces
> vmnet        8000.003048c8166d    no        eth1
>                            vif1.0
>                            vif10.0
>                            <other interfaces>
> ---------
>
> Does anyone have any ideas as to what is going on here? Or more 
> importantly, any ideas on how to solve this? I have tried building a 
> newer domU kernel from scratch but I haven't been able to make any 
> progress there. The guest fails to boot without showing anything on 
> the console. It then goes into this loop of trying to reboot the guest 
> but failing. I would really like to stay with the debian kernels.
> I have been banging my head against a wall for a week or so on this 
> and desperately need some help to get this working. I have engineers 
> that are getting held up by this bug.
This problem still exists.

I tried setting an independent wallclock on all of the virtual machines. 
I also managed to miss that I had the wrong netmask configured for the 
vmnet bridge. It should have been 255.255.255.128. The vms were able to 
talk to each other before changing the netmask and I saw traffic flowing 
past the switch.
Does anyone have any clue as to what might be going on? I am great need 
of some help here.
Thanks

mike

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users