Re: [Xen-users] DomU PV network problems

[Sauro Saltini <saltini@xxxxxx>]

Some news about the problem...

after receiving some indirect Matej suggestions from Mahdavian, I've 
looked into the checksum problem mentioned here:
http://wiki.xensource.com/xenwiki/XenFaq#head-4ce9767df34fe1c9cf4f85f7e07cb10110eae9b7

This time I've tried to connect to an http service on a phisical lan 
host (telnet 192.168.99.202 80) from the testing DomU (10.0.0.102)
here are the results of tcpdump -nvv on the various elements of the lan 
connected bridge in Dom0 during the connection tests (192.168.99.88 is 
the fw 'external' IP correctly set by SNAT rule) :
ON vif1.1 - DomU side of bridge :

tcpdump -nvvi vif1.1
tcpdump: WARNING: vif1.1: no IPv4 address assigned
tcpdump: listening on vif1.1, link-type EN10MB (Ethernet), capture size 
96 bytes
17:31:07.423789 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 
192.168.99.202 tell 192.168.99.88, length 28
17:31:07.423889 ARP, Ethernet (len 6), IPv4 (len 4), Reply 
192.168.99.202 is-at 00:09:6b:89:d0:8a, length 46
17:31:07.423911 IP (tos 0x10, ttl 63, id 3345, offset 0, flags [DF], 
proto TCP (6), length 60)
    192.168.99.88.56168 > 192.168.99.202.80: Flags [S], cksum 0x8d22 
(correct), seq 244003632, win 5840, options [mss 1460,sackOK,TS val 
474657 ecr 0,nop,wscale 5], length 0
....

here the cksum field is correct


ON br1 - the bridge itself:

tcpdump -nvvi br1
tcpdump: listening on br1, link-type EN10MB (Ethernet), capture size 96 
bytes
17:38:13.454259 IP (tos 0x10, ttl 63, id 40899, offset 0, flags [DF], 
proto TCP (6), length 60)
    192.168.99.88.54824 > 192.168.99.202.80: Flags [S], cksum 0x5545 
(correct), seq 2632204777, win 5840, options [mss 1460,sackOK,TS val 
581162 ecr 0,nop,wscale 5], length 0
again the cksum field is correct


ON peth0 - the phisical NIC :
tcpdump -nvvi peth0 port not 22
tcpdump: WARNING: peth0: no IPv4 address assigned
tcpdump: listening on peth0, link-type EN10MB (Ethernet), capture size 
96 bytes
17:40:48.397392 IP (tos 0x10, ttl 63, id 59382, offset 0, flags [DF], 
proto TCP (6), length 60)
    192.168.99.88.44172 > 192.168.99.202.80: Flags [S], cksum 0x48a2 
(incorrect -> 0x05a8), seq 774410895, win 5840, options [mss 
1460,sackOK,TS val 619897 ecr 0,nop,wscale 5], length 0
Here the cksum field is INCORRECT thus the packet is probably refused !

I've tried to switch off all forms of offloading from peth0 and also 
from eth0 and eth1 in fw DomU (ethtool -K peth0 tx off rx off sg off tso 
off ufo off gso off) but the result is the same !
I'm also still confused about the reason why icmp packets (pings) don't 
even show up on vif1.1 while tcp connection tries do... any ideas ?
What next ? Do I need to try a completely new xenified kernel ?

Many thanks in advance to all...

Sauro.



On 08/09/2010 14:25, Sauro Saltini wrote:
> On 08/09/2010 10:42, Fajar A. Nugraha wrote:
> > On Wed, Sep 8, 2010 at 4:18 AM, Sauro Saltini<saltini@xxxxxx>  wrote:
> > > My current configuration is :
> > > - Xen 4.0.0
> > > - DOM0 kernel 2.6.31.13
> > > - DOMU kernels either 2.6.31.13 or 2.6.36-rc3 (vanilla)
> > > As soon as I've converted the fw DomU's to a PV domain (using either
> > > 2.6.31.13 or 2.6.36-rc3 kernels with PV drivers) something changed in a
> > > weird way...
> > Do you mean convert it to PV domain, or keep it as HVM but use PV 
> > drivers?
> I've converted it to a PV domain (builder=linux)
> > > on Dom0 "tcpdump -nvvi br0" (br0 = external bridge) never shows up any
> > > traffic !
> > What does tcpdump on domU's vif (by default it's named vifx.y, where x
> > and are numbers) within dom0 shows?
> > Does both dom0 and your PV domU has ip forwarding
> > (/proc/sys/net/ipv4/ip_forward) enabled?
> > Do you still use ioemu line? What does your domU config look like?
> tcpdump on domU's vif shows no traffic, except some arp requests 
> coming from the firewall domU asking for the external pinged address.
> Both dom0 and domU has ip_forward activated
> I have type=paravirtual for both firewall domU's nics in domU's config
>
> my config is quite simple:
>
> kernel='/boot/vmlinuz-r410-2.6.31.13'
> builder='linux'
> memory=512
> name='fw'
> vcpus=1
> vif=[ 'type=paravirtual, bridge=br0, mac=00:16:3e:00:00:02',
>       'type=paravirtual, bridge=br1, mac=00:16:3e:00:00:20' ]
> disk=[ 'drbd:fw,hda,w' ]
> on_poweroff='destroy'
> on_reboot='restart'
> on_crash='restart'
> boot='c'
> root='/dev/xvda3 ro'
> sdl=0
> vnc=0
>
> The chosen kernel is the same xenified kernel built from sources and 
> used for dom0
>
> > > Please help...
> > > Many thanks in advance.
> It seems that the packet flow interrupts in some way inside xen's PV 
> netfront/netback drivers (i.e. between vethx.y and vifx.y) but ONLY 
> for packets coming from another domU and forwarded.
> As I said i can regularly ping both domU's network and external lan 
> from the fw domU, only the packets coming from the other domU's 
> disappear !
> Just for clarity, having :
> testing (pinger) domU : 10.0.0.102 /24 (gw 10.0.0.101)
> fw domU "internal" nic -eth0- (domU's gateway): 10.0.0.101 /24
> fw domU "external" nic -eth1- (bridged to LAN): 192.168.99.88 /24
> pinged lan host : 192.168.99.202 /24
>
> On fw domU:
> echo "1" > /proc/sys/net/ipv4/ip_forward
> iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
>
> Pinging 192.168.99.202 from FW works, and tcpdump -nvvi on dom0 (both 
> listening on vifx.y or bridge) gives:
> ...
> 14:09:40.553104 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto 
> ICMP (1), length 84)
>     192.168.99.88 > 192.168.99.202: ICMP echo request, id 24837, seq 
> 1, length 64
> 14:09:40.553223 IP (tos 0x0, ttl 64, id 21099, offset 0, flags [none], 
> proto ICMP (1), length 84)
>     192.168.99.202 > 192.168.99.88: ICMP echo reply, id 24837, seq 1, 
> length 64
> ...
>
> Pinging 192.168.99.202 from testing domU (10.0.0.102) doesn't work, 
> and tcpdump -nvvi on dom0 (both listening on vifx.y or bridge) gives 
> (every 5 to 10  seconds):
> ....
> 14:17:29.922210 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 
> 192.168.99.202 tell 192.168.99.88, length 28
> 14:17:29.922304 ARP, Ethernet (len 6), IPv4 (len 4), Reply 
> 192.168.99.202 is-at 00:09:6b:89:d0:8a, length 46
> ....
>
> but no icmp traffic at all !
>
>
> > You can try using xenified kernel on domU and see if it helps: patch
> > from http://code.google.com/p/gentoo-xen-kernel/downloads/list +
> > kernel 2.6.34.6 (or whatever 2.6.34.x that the patch can cleanly apply
> > to).
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users