[Xen-users] Re: Yet another question about multiple NICs [FIXED]

[Philippe Combes <Philippe.Combes@xxxxxxxxxxx>]

Hi all,

This is an old thread already, but I have had higher priorities and I 
have not been able to check for your last propositions for a few weeks.
I confess that I first considered irrelevant for my case the 
suggestions of JP P., Thomas J. and Simon H. about the order of the 
NICs in my domU debian system. I was too confident in my tests... I 
realized I was wrong when I tried to connect the first vif of a 
*fresh* domU to eth1 of dom0 (and the 2nd vif to eth0 of dom0). Indeed 
I had eventually the same symptoms on the problematic subnet.
I do not know now what kind of test made me tell you it worked, and I 
am sorry for this wrong statement.
After that, the questions from Fajar A. N. about the configuration of 
the switches became still more sensible. I had promised I would 
perform the test with my laptop and a crossover-cable. And indeed, it 
worked perfectly with the default way (wrapping the embedded 
network-bridge script in xen-config.sxp)
Then I had hard time trying to reconfigure the switches, because it 
was the first time I had to do such things, and I could only use 
documentation found on the web. No root passwd, etc.
Anyway I succeeded at last to deactivate the port security policy and 
everything is fine now.
Thank you very much to all for your very precious help !
Regards,
Philippe


Philippe Combes wrote:
> Dear Xen users,
>
> I have tried for weeks to have a domU connected to both NICs of the
> dom0, each in a different LAN. Google gave me plenty of tutos
> and HowTos about the subject, including the Xen and the Debian Xen
> wiki's, of course. It seems so simple !
> Some advise to use a simple wrapper to /etc/xen/network-bridge, others
> to let it aside and to set bridges on my own.
> But there must be something obvious that I miss, something so obvious
> that no manual need to explain it, because I tried every solution and
> variant I found on the Internet with no success.
>
> My dom0 first ran CentOS 5.5, Xen 3.0.3. I tried to have eth1 up and 
> configured both in dom0 and in a domU. I never succeeded (details 
> below), so I followed the advice of some colleagues who told me my 
> issues might have come from running a Debian lenny domU on a CentOS dom0 
> (because the domU used the CentOS kernel instead of the one of Debian 
> lenny, which is more recent).
> So now my dom0 runs an up-to-date Debian lenny, with Xen 3.2.1, but I 
> have the same behaviour when trying to get two interfaces in a domU. As 
> I said it before, I tried several configurations, but let's stick for 
> now to one based on the network-bridge script.
> In /etc/network/interfaces:
>  auto eth0
>  iface eth0 inet dhcp
>  auto eth1
>  iface eth1 inet dhcp
> In /etc/xen/xend-config.sxp:
>  (network-script network-bridge-wrapper)
> /etc/xen/scripts/network-bridge-wrapper:
>  #!/bin/bash
>  dir=$(dirname "$0")
>  "$dir/network-bridge" "$@" vifnum=0 netdev=eth0 bridge=eth0
>  "$dir/network-bridge" "$@" vifnum=1 netdev=eth1 bridge=eth1
> In domU configuration file:
>  vif = [ 'mac=00:16:3E:55:AF:C2,bridge=eth0', 
> 'mac=00:16:3E:55:AF:C3,bridge=eth1' ]
> With this configuration, I get both bridges eth<i> configured and 
> usable: I mean I can ping one machine of every LAN through the 
> corresponding interface.
> When I start a domU however, the dom0 and the domU are alternatively 
> connected to the LAN of eth1, but mutually exclusively. In other words, 
> the dom0 is connected to the LAN on eth1 for a couple of minutes, but 
> not the domU, and then, with no other reason than inactivity on the 
> interface, it switches to the reverse situation: domU connected, not the 
> dom0. After another couple of minutes of inactivity, back to the first 
> situation, and so on...
> I noticed that the 'switch' does not occur if the one that is currently 
> connected performs a continuous ping on another machine of the LAN.
> This happened with the CentOS too. But I did not try anything else under 
> that distro. Under Debian, I tried to have dom0's eth1 down (no IP), but 
> then the domU's eth1 does not work at all, not even periodically.
> I was pretty sure the issue came from the way my bridges were 
> configured, that there was something different with the dom0 primary 
> interface, etc. Hence  I tried all solutions I could find on the 
> Internet with no success.
> I then made a simple test. Instead of binding domU's eth<i> to dom0's
> eth<i>, I bound it to dom0's eth<1-i>: I changed
>  vif = [ 'mac=00:16:3E:55:AF:C2,bridge=eth0',
> 'mac=00:16:3E:55:AF:C3,bridge=eth1' ]
> to
>  vif = [ 'mac=00:16:3E:55:AF:C3,bridge=eth1',
> 'mac=00:16:3E:55:AF:C2,bridge=eth0' ]
> I was very surprised to see that dom0's eth0, domU's eth0 and dom0's 
> eth1 were all working normally, not domU's eth1. There was no alternance 
> between dom0's eth0 and domU's eth1 there, probably because there is 
> always some kind of activity on dom0's eth0 (NFS, monitoring).
> So it seems that my issue is NOT related to the dom0 bridges, but to the 
> order of the vifs in the domU description. However, in the xend.log 
> file, there is no difference in the way both vifs are processed.
>  [2010-12-16 14:51:27 3241] INFO (XendDomainInfo:1514) createDevice: vif 
> : {'bridge': 'eth1', 'mac': '00:16:3E:55:AF:C2
> ', 'uuid': '9dbf60c7-d785-96e2-b036-dc21b669735c'}
>  [2010-12-16 14:51:27 3241] DEBUG (DevController:118) DevController: 
> writing {'mac': '00:16:3E:55:AF:C2', 'handle': '0'
> , 'protocol': 'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend': 
> '/local/domain/0/backend/vif/2/0'} to /local/d
> omain/2/device/vif/0.
>  [2010-12-16 14:51:27 3241] DEBUG (DevController:120) DevController: 
> writing {'bridge': 'eth1', 'domain': 'inpiftest',
> 'handle': '0', 'uuid': '9dbf60c7-d785-96e2-b036-dc21b669735c', 'script': 
> '/etc/xen/scripts/vif-bridge', 'mac': '00:16:
> 3E:55:AF:C2', 'frontend-id': '2', 'state': '1', 'online': '1', 
> 'frontend': '/local/domain/2/device/vif/0'} to /local/d
> omain/0/backend/vif/2/0.
>  [2010-12-16 14:51:27 3241] INFO (XendDomainInfo:1514) createDevice: vif 
> : {'bridge': 'eth0', 'mac': '00:16:3E:55:AF:C3
> ', 'uuid': '1619a9f8-8113-2e3c-e566-9ca9552a3a93'}
>  [2010-12-16 14:51:27 3241] DEBUG (DevController:118) DevController: 
> writing {'mac': '00:16:3E:55:AF:C3', 'handle': '1'
> , 'protocol': 'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend': 
> '/local/domain/0/backend/vif/2/1'} to /local/d
> omain/2/device/vif/1.
>  [2010-12-16 14:51:27 3241] DEBUG (DevController:120) DevController: 
> writing {'bridge': 'eth0', 'domain': 'inpiftest',
> 'handle': '1', 'uuid': '1619a9f8-8113-2e3c-e566-9ca9552a3a93', 'script': 
> '/etc/xen/scripts/vif-bridge', 'mac': '00:16:
> 3E:55:AF:C3', 'frontend-id': '2', 'state': '1', 'online': '1', 
> 'frontend': '/local/domain/2/device/vif/1'} to /local/d
> omain/0/backend/vif/2/1.
>
> There I am stuck, and it is very frustrating. It looks so simple when 
> reading at tutos, that I clearly missed something obvious, but what ?
> Any clue, any track to follow down will be welcome, truly. Please do not 
> hesitate to ask me for relevant logs, or for any experiment you would 
> think useful.
> Thanks for your help,
> Philippe.
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users