[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] Network Problem after DomU migration

shreyas pandya wrote:

why are you using two switches?should not one switch suffice for two nodes?

There can be many reasons - I have (now) four nodes spread across three switches which are themselves connected via a fourth. Ie, we have a 'root' switch, and then a switch in the top of each rack. If traffic is between nodes in different racks, that means there are 3 switches involves - top-of-rack -> root switch -> top-of-other-rack.

And don't forget that in many common Xen setups, there is a "switch" in the Dom0 - usually called a bridge. This acts exactly as described below. You can see the MAC tables for this switch with the showmacs command to brctl.

i guess if you use one switch instead of two then the information about migrated vm will propagate to switches quickly

It shouldn't make any difference. When you move a device and then send a packet from it, one of two things will happen :

1) If the MAC of the remote node is already in the tables, then the switch will forward the packet out through the correct port AND update it's own table with the new MAC-port pair. If the packet has to go through other switch(es) then the same will happen on those until the packet reaches it's destination.

2) If the MAC-port pairing is not known, then the switch will forward the packet via all ports other than the one it was received on. The switch will also learn the MAC-port pairing for the packet source as above. Again, the same will be repeated on all switches the packet hits until it reaches it's destination.

Thus, assuming there's been traffic between A and B, involving switch S1. You move A to switch S2 and ping B. Assuming S2 already knew where B is, then it will forward the packet to S1 and update it's tables. S1 will forward the packet to B and update it's tables. S1 now knows how to forward packets to A via S2.

Suppose there is node C, connected to S3 which is connected to S1. It sends a packet to A, S1 now knows where A is and sends the packet to S2, which sends it to A.

Now consider if A is moved but remains silent.
B tries to send a packet to A, but S1 simply sends it out the wrong port. Ditto for C, S3 sends the packet to S1, but S1 misdirects it.

Two things will deal with this :

1) All switches age their MAC tables. After a period (which may be fixed, or admin configurable) they will remove entries for devices from which they have not had any traffic lately. After this, traffic addressed to those devices will be broadcast to all ports until traffic FROM the node is seen again at which point the MAC tables will again be populated.

2) Even before this, If B or C don't get a reply from A, they may well do an ARP request again. Since this is broadcast, the switches will forward it to all ports and all devices on the network will see it. A can respond, and the return packet will trigger the switches to update their MAC tables. As I mentioned earlier, I have seen some switches which will not update the table until the old entry has expired - thus when you move a device, it will not be visible on the network for (in this case) 5 minutes.

The problem people are seeing is during the period between the guest getting moved, and traffic FROM it triggering updates to the MAC tables. This can generally be fixed by simply getting the guest to send some traffic, preferably broadcast OR by getting another device to send an ARP request for it (which may not update every switch in the network).

If nothing is done about it, then the device disappears from the network until one of these happens - which may be when something expires the device from it's ARP cache and sends an ARP request for it.

Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.