Ok, I've managed to pin it down and you are quite right - it's ARP.
Now the question is, how do I fix it.
here's what I have
Dom0 :: 10.0.0.1
DomU :: 10.0.0.12
Both machines work fine for 40 mins .. then;
DomU reports Dom0 unreachable.
Sure enough ping 10.0.0.1 gives no response.
However, ping 10.0.0.12 from Dom0 responds fine.
A one-way ping!
arp -na on Dom0 reports as expected.
arp -na on the broken DomU shows;
? (10.0.0.1) at FE:FF:FF:FF:FF:FF [ether] on eth0
It's picking up FE:EE ... instead of the desired MAC address ?!
How can it do this ?!
On Dom0:
eth0 Link encap:Ethernet HWaddr 00:15:C5:5D:C0:BE
inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0
inet6 addr: fe80::215:c5ff:fe5d:c0be/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:205397 errors:0 dropped:0 overruns:0 frame:0
TX packets:413848 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:41267633 (39.3 MB) TX bytes:95050228 (90.6 MB)
On DomU:
eth0 Link encap:Ethernet HWaddr 00:00:10:00:00:0C
inet addr:10.0.0.12 Bcast:10.0.0.255 Mask:255.255.255.0
inet6 addr: fe80::200:10ff:fe00:c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6297 errors:0 dropped:0 overruns:0 frame:0
TX packets:11662 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1157351 (1.1 MB) TX bytes:907972 (886.6 KB)
Help!
--
Managing Director, Encryptec Limited
Tel: 0845 25 77033, Mob: 07853 305393, Int: 00 44 1443205756
Email: gareth@xxxxxxxxxxxxx
Statements made are at all times subject to Encryptec's Terms and
Conditions of Business, which are available upon request.
----- Original Message -----
From: "Matthew Crocker" <mcrocker@xxxxxxxxxxx>
To: "Gareth Bult" <gareth@xxxxxxxxxxxxx>
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Sent: Monday, January 7, 2008 10:00:40 PM (GMT) Europe/London
Subject: Re: Re; [Xen-users] Ethernet has Alzheimers
Is the machine answering ARP replies?
Honestly, I don't know .. the machine tends to lock up for other
reasons when it dies hence it's not easy to track ..
Does the upstream router have the IP & MAC in its ARP table?
Does the upstream switch have the MAC in its mac-address-table?
Assigned to the correct port?
Sounds to me like an ARP timeout problem.
This occurs between DomU's and Dom0 in addition to external
addresses ... so I don't think it's linked to anything outside of
Xen .. I've experienced the same problem on 4 different machines,
all different HW config .. so again I think faulty HW is out.
For what it's worth;
I'm using Ubuntu Gutsy (7.10) with the stock Xen 3.1 kernel all on
AMD64 and Intel/Xeon machines all running 64 bit kernels and distros.
All machines are using bridging with two physical ethernet ports.
All DomU's are running two matching virtual ports.
I'm using IPTABLES (firehol) fairly heavily for port filtering.
Bridging is Layer2, IP is Layer 3, you are having a problem at layer
3 so you need to look to make sure your layer 2 stuff is working
properly.
If Xen is bridging only then you won't really have visibility into the
Layer 3 problem from Dom0. You could look at the bridging config and
see if it knows about the MAC address properly in the switch. At
some point upstream from the Xen hardware you have another Layer 3
device, most likely a router. You need to get into that router and
see if it has the IP -> MAC entry in its ARP table. If it doesn't
have it then there is your problem. Something is stopping the DomU
from answering the ARP queries from the router. The route loses track
of the MAC address and can no longer send Ethernet frames to your
DomU. If your router does have the ARP entry then I would look into
your switches and see if they are dropping the MAC address from their
table.
