Re: [Xen-devel] pv_ops routing problem?

[Michael D Labriola <mlabriol@xxxxxxxx>]

xen-devel-bounces@xxxxxxxxxxxxxxxxxxx wrote on 04/28/2010 06:40:32 AM:

> Michael D Labriola <mlabriol@xxxxxxxx> 
> Sent by: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> 
> 04/28/2010 06:40 AM
> 
> To
> 
> Jeremy Fitzhardinge <jeremy@xxxxxxxx>
> 
> cc
> 
> Jayson A Dyke <jdyke@xxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx, xen-
> devel-bounces@xxxxxxxxxxxxxxxxxxx
> 
> Subject
> 
> Re: [Xen-devel] pv_ops routing problem?
> 
> xen-devel-bounces@xxxxxxxxxxxxxxxxxxx wrote on 04/27/2010 06:02:12 PM:
> 
> > Jeremy Fitzhardinge <jeremy@xxxxxxxx> 
> > Sent by: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> > 
> > 04/27/2010 06:02 PM
> > 
> > To
> > 
> > Michael D Labriola <mlabriol@xxxxxxxx>
> > 
> > cc
> > 
> > Jayson A Dyke <jdyke@xxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx
> > 
> > Subject
> > 
> > Re: [Xen-devel] pv_ops routing problem?
> > 
> > On 04/27/2010 02:22 PM, Michael D Labriola wrote:
> > > Ok, I've got a problem.  My current setup is as follows:
> > >
> > > I have a pv_ops dom0 (xen/master from a few weeks ago) running 3 
> domUs. 
> > > Each domU has an eth0 interface and one or more VLAN interfaces.
> > >
> > > vm1, vm2, and vm3 are all domUs running with the same pv_ops kernel 
as 
> 
> > > dom0.  vm1's eth0.107 interface routes through vm3 to get to vm2's 
> > > eth0.1020 interface.
> > >
> > > dom0:
> > >   eth0:     192.168.1.20/24
> > >
> > > vm1:
> > >   eth0:     192.168.1.10/24
> > >   eth0.107: 10.7.1.1/16
> > >   gateway:  10.7.0.1
> > >
> > > vm2:
> > >   eth0:      192.168.1.11/24
> > >   eth0.1020: 10.20.1.1/16
> > >   gateway:   10.20.0.1
> > >
> > > vm3:
> > >   eth0:      192.168.1.12/24
> > >   eth0.107:  10.7.0.1/16
> > >   eth0.1020: 10.20.0.1/16
> > >
> > >
> > > >From vm1, I can ping the gateway (10.7.0.1) and vm3's 1020 
interface 
> > > (10.20.0.1), but get no response if I try to ping vm2's 10.20.1.1.
> > >
> > > vm3 can ping both 10.7.1.1 and 10.20.1.1.
> > >
> > > Should be a pretty simple packet forwarding example, and it used to 
> work 
> > > using the old xen-linux 2.6.18 kernel we were using on our deployed 
> > > systems.  But now it's broken...
> > >
> > > I have verified that the exact same setup running on 3 physical PCs 
> > > bare-metal works w/out any problems.  I had the problem initially on 

> Xen 
> > > 3.4.2, but just upgraded to 4.0.0 and have the exact same symptoms.
> > > 
> > 
> > Have you tried turning off tx checksum offload?  (ethtool -K <dev> tx 
> off)
> > 
> > Though for me that just fixes network access between dom0 and domU;
> > external traffic normally works fine.
> 
> Jeremy, I haven't tried that yet.  I'll try it this morning, although I 
> don't really expect that to fix the problem.  I'm also going to pull 
from 
> the xen/master branch again to see if that makes a difference.  My last 
> 2.6.31.12 kernel was pulled from your tree a few weeks ago.
> 
> By the way, the packet forwarding is definitely happening on my router 
> domU.  If I look at vm3's eth0 w/ wireshark, I can see the VLAN 
> encapsulated ICMP echo request coming in on the 107 VLAN and I can see 
> that vm3 is forwarding it out over its 1020 VLAN interface.  However, if 
I 
> look at my dom0's vif3.0 interface with wireshark I only see the VLAN 
107 
> traffic...  no sign of the 1020 forwarded packet anywhere.  It's as if 
> forwarded packets aren't finding their way onto dom0's vif3.0.  I've 
never 
> seen this before, and was under the impression that dom0's vif 
interfaces 
> _WERE_ the domU's interfaces just with a different name... 

Should the tx checksum offload only be disabled on dom0?  And does it have 
to get disabled before xend starts?  I've tried disabling it on just dom0 
and also on domUs with no difference.

Also, I updated to the current xen/master and it behaves the same.

I just tried adding 107 and 1020 VLAN interfaces to my dom0 (10.7.0.2 and 
10.20.0.2) and it turns out that dom0 also cannot ping the VLAN interfaces 
on any of the domUs...  not sure if that's a separate issue or if it's all 
related...

-Mike

---
Michael D Labriola
Electric Boat
mlabriol@xxxxxxxx
401-848-8871 (desk)
401-848-8513 (lab)
401-316-9844 (cell)



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel