[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] Bonding ethernet on xen 3.3 (ARP issues)

  • To: xen-users@xxxxxxxxxxxxxxxxxxx
  • From: Muhammad Atif <m_atif_s@xxxxxxxxx>
  • Date: Wed, 3 Dec 2008 20:14:59 -0800 (PST)
  • Cc: wferi@xxxxxxx, sassyn@xxxxxxxxx
  • Delivery-date: Wed, 03 Dec 2008 20:15:41 -0800
  • Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:MIME-Version:Content-Type:Message-ID; b=W6R8/eHHlHMv3TuIdlENrfNDYPxWbvq6K984C/8M2zt20zdndoabFRaJUrTX2jUC4ebivw/MZcfnpjl8iC+IbZTz8JGcQ2f16n051DH6cAMSoD+CA34WNKnMI15LozHS8ZcME3Zo7P4lxxBGhf6pLh7xnHl+nIbqU9E53KxrvjA=;
  • List-id: Xen user discussion <xen-users.lists.xensource.com>

Did not feel like stealing the email thread so posting this one with new subject

I am struggling with ethernet bonding on ubuntu 8.04. My configuration is a 
touch different as I have 3 ethernet interfaces. I want eth0 to be independent 
but want to try ethernet aggregation for eth1 and eth2. i.e. Load balancing 
So what I am doing is that I let xen bootup with normal network-bridge script 
and it creates eth0 bridge (et al) and all works. Eth1 and Eth2 are enslaved to 
Bond0. I am able to use Bond0 to ping/ssh to other machines. So I am sure that 
ethernet are bonded correctly.
Routing table for this configuration at dom0 (without Xenbr added) is 

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface     *        U     0      0        0 bond0     *        U     0      0        0 eth0
default         xxxx           UG    100    0        0 eth0

#brctl show
bridge name     bridge id               STP enabled     interfaces
eth0            8000.00d06809191a       no              peth0

Now I struggle with Xenbr creation (call it brbond). After bring up the brbond 
my dom0 is as follows:-

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface     *        U     0      0        0 brbond     *        U     0      0        0 eth0
default         xxxx           UG    100    0        0 eth0

#brctl show
bridge name     bridge id               STP enabled     interfaces
brbond             8000.00d06809191b       no              bond0
eth0            8000.00d06809191a       no              peth0

I am able to ping anywhere from dom0 (sanity check).

Now comes the interesting part.... I fire up by domu. The 
/etc/network/interface is set to get IP address from dhcp server... it comes 
up. It gets the IP addresses from dhcp server. so far so good..

The problem: I cannot ping 192.168.4.XX (i.e. .4.x subnet) from this domU and 
no other machine (be it virtual or not) can ping this domU. The dom0 (hosting 
this domU) can ping and get replys from domU using eth1. At random my domU is 
able to communicate with other machine, but its random.

The routing table for domU is. 

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface     *        U     0      0        0 eth1     *        U     0      0        0 eth0
default         xxxx           UG    0      0        0 eth0

On dom0 the brctl show looks fine, 

bridge name     bridge id               STP enabled     interfaces
brbond          8000.00d06809191b       no              bond0
eth0            8000.00d06809191a       no              peth0

Further inspection shows that domU ARP table has incomplete entries. The domU 
cannot get any thing related to 192.168.4.x network (except for its host 
domain).  From other machines, when I ping this domain there are no replies, 
but ARP does contain the mac address for this IP / machine. Manually adding ARP 
makes this problem go away.

Can someone please guide/point me to decipher what is going on.

Muhammad Atif

----- Original Message -----
From: "Ferenc Wagner" <wferi@xxxxxxx>
To: "Sassy Natan" <sassyn@xxxxxxxxx>
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Sent: Thursday, December 4, 2008 3:55:30 AM (GMT+1000) Auto-Detected
Subject: [Xen-users] Re: Fwd: bonding ethernet and xen 3.2.1 or later.

"Sassy Natan" <sassyn@xxxxxxxxx> writes:

> I hope it is ok to ask this by email, since I have spent like 4 days trying
> to configure XEN with bonding.

It would have been better to ask this on the list.

> I saw your post on
> http://thread.gmane.org/gmane.comp.emulators.xen.user/41406/focus=41511
> ans some other links like:
> http://www.performancemagic.com/Dell1950_MD3000i_Xen_Debian_iSCSI_RDAC/Networking.html
> http://www.debian-administration.org/users/lters/weblog/14
> but i think you are right that it is better to configure network devices
> using OS and not XEN Scripts

Most of the above is obsolete in Lenny, where you can easily configure
bonding and bridging from /etc/network/interfaces *alone*.

> I did the following: (using debian lenny RC1 with xen 3.2.1 on amd64)
> # The loopback network interface
> auto lo
> iface lo inet loopback
> # The primary network interface
> allow-hotplug eth0
> allow-hotplug eth1
> auto bond0
> iface bond0 inet manual
>         # Bonding Options
>         pre-up /sbin/modprobe -v bonding -o bond0 mode=0 miimon=100
> downdelay=200 updelay=200
>         pre-up /sbin/ifconfig eth0 up
>         pre-up /sbin/ifconfig eth1 up
>         pre-up /sbin/ifconfig bond0 up
>         pre-up /sbin/ifenslave bond0 eth0
>         pre-up /sbin/ifenslave bond0 eth1
>         pre-up /usr/sbin/brctl addbr br0
>         pre-up /sbin/ifconfig br0 up
>         pre-up /usr/sbin/brctl addif br0 bond0
>         pre-up /usr/sbin/brctl stp br0 off
>         pre-up /usr/sbin/brctl setfd br0 0
>         post-down /usr/sbin/brctl delif br0 bond0
>         post-down /sbin/ifconfig br0 down
>         post-down /usr/sbin/brctl delbr br0
>         post-down /sbin/ifenslave -d bond0 eth1
>         post-down /sbin/ifenslave -d bond0 eth0
>         post-down /sbin/ifconfig bond0 down
>         post-down /sbin/ifconfig eth1 down
>         post-down /sbin/ifconfig eth0 down
>         # This might cause Kernel Panic !!!
>         # post-down rmmod bond0
> auto br0
> iface br0 inet static
>         address
>         netmask
>         network
>         broadcast
>         gateway
>         # dns-* options are implemented by the resolvconf package, if
> installed
>         dns-nameservers
>         dns-search primesense.com
> everything looks fine

No, that looks horrible.  You mix up manual bond and bridge setup
instead of simply using the OS facilities.  Just adapt Massimiliano's
solution, which you quoted first.  Once you have something that simple
which doesn't work, ask again (on the list).
Good luck,

Xen-users mailing list


Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.