[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] tcp wrong checksum



My dom0 is sending packets to the domU with incorrect TCP checksums
(not in all cases, but in some).  I've looked at the various FAQ and
documentation pages but I'm afraid I'm still stumped.  (ethtool -k
didn't work.)

Below are tcpdumps of the sessions, and the output of route -vn and
ifconfig and brctl show on both dom0 and domU.  I'm not using bridges;
I'm using a slightly modified vif-route, so I also include the output
of iptables -L -v -n and a copy of the vif-route script.

The topology is fairly simple:

 real ethernet       dom0 as router        domU
 172.18.45/25 ------ 172.18.45.97 -------- 172.18.45.65
 gw 172.18.45.11     eth0     vif*.0       eth0
 to real Internet

I have tried a variety of different networking configs on the domU to
try to get it not to check the tcp checksums (since the dom0
apparently insists on not generating them correctly), without any
success.  I _am_ able to ssh from another machine on my network to
domU via the routing in dom0, showing that tcp checksums are at least
being generated correctly in one direction.

Ideally I would like to COMPLETELY DISABLE this fragile optimisation.
Is there a way to do that ?

Failing that I need to either (a) persuade dom0 to generate proper
checksums on packets leaving for domU via vif*, or (b) persuade domU
to accept broken checksums but only on some packets (the ones from
dom0 itself rather than routed via dom0).


Versions:

I'm using the Debian Xen packages from Ralph Passgang (3.0.1-0tha3)
locally compiled on sarge but without patches.  Both host and guest
are running the same 2.6.12, which is vanilla except for the Xen
patches.

The host is Debian sarge; the guest is Ubuntu dapper (constructed with
pbuilder/debootstrap and some home-grown scripts I'm working on).


Ian.


dom0:

lalonde:~# tcpdump -vvs500 -lnivif31.0
tcpdump: listening on vif31.0, link-type EN10MB (Ethernet), capture size 500 
bytes
17:56:39.806453 IP (tos 0x0, ttl  64, id 19458, offset 0, flags [DF], length: 
60) 172.18.45.97.37227 > 172.18.45.65.22: S [tcp sum ok] 
1738938563:1738938563(0) win 5840 <mss 1460,sackOK,timestamp 31872193 
0,nop,wscale 2>
17:56:39.807082 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], length: 60) 
172.18.45.65.22 > 172.18.45.97.37227: S [tcp sum ok] 1245122588:1245122588(0) 
ack 1738938564 win 5792 <mss 1460,sackOK,timestamp 273577 31872193,nop,wscale 2>
17:56:39.807114 IP (tos 0x0, ttl  64, id 19460, offset 0, flags [DF], length: 
52) 172.18.45.97.37227 > 172.18.45.65.22: . [tcp sum ok] 1:1(0) ack 1 win 1460 
<nop,nop,timestamp 31872193 273577>
17:56:39.823210 IP (tos 0x0, ttl  64, id 15543, offset 0, flags [DF], length: 
90) 172.18.45.65.22 > 172.18.45.97.37227: P [tcp sum ok] 1:39(38) ack 1 win 
1448 <nop,nop,timestamp 273578 31872193>
17:56:39.823483 IP (tos 0x0, ttl  64, id 19462, offset 0, flags [DF], length: 
52) 172.18.45.97.37227 > 172.18.45.65.22: . [tcp sum ok] 1:1(0) ack 39 win 1460 
<nop,nop,timestamp 31872195 273578>
17:56:39.824197 IP (tos 0x0, ttl  64, id 19464, offset 0, flags [DF], length: 
93) 172.18.45.97.37227 > 172.18.45.65.22: P [bad tcp cksum b316 (->4d05)!] 
1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872195 273578>
17:56:40.028407 IP (tos 0x0, ttl  64, id 19466, offset 0, flags [DF], length: 
93) 172.18.45.97.37227 > 172.18.45.65.22: P [bad tcp cksum b316 (->4cf0)!] 
1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872216 273578>
17:56:40.448393 IP (tos 0x0, ttl  64, id 19468, offset 0, flags [DF], length: 
93) 172.18.45.97.37227 > 172.18.45.65.22: P [bad tcp cksum b316 (->4cc6)!] 
1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872258 273578>
17:56:41.288386 IP (tos 0x0, ttl  64, id 19470, offset 0, flags [DF], length: 
93) 172.18.45.97.37227 > 172.18.45.65.22: P [bad tcp cksum b316 (->4c72)!] 
1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872342 273578>
17:56:42.968366 IP (tos 0x0, ttl  64, id 19472, offset 0, flags [DF], length: 
93) 172.18.45.97.37227 > 172.18.45.65.22: P [bad tcp cksum b316 (->4bca)!] 
1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872510 273578>

10 packets captured
10 packets received by filter
0 packets dropped by kernel
lalonde:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:13:20:21:DF:C1  
          inet addr:172.18.45.97  Bcast:172.18.45.255  Mask:255.255.255.0
          inet6 addr: fe80::213:20ff:fe21:dfc1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:694410 errors:148 dropped:0 overruns:0 frame:148
          TX packets:478168 errors:1 dropped:0 overruns:0 carrier:1
          collisions:80299 txqueuelen:1000 
          RX bytes:574281356 (547.6 MiB)  TX bytes:45053982 (42.9 MiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:7 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1248 (1.2 KiB)  TX bytes:1248 (1.2 KiB)

vif31.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet addr:172.18.45.97  Bcast:172.18.45.97  Mask:255.255.255.255
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:68 errors:0 dropped:0 overruns:0 frame:0
          TX packets:96 errors:0 dropped:5 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:4487 (4.3 KiB)  TX bytes:14813 (14.4 KiB)

lalonde:~# route -vn
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
172.18.45.65    0.0.0.0         255.255.255.255 UH    0      0        0 vif31.0
172.18.45.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
0.0.0.0         172.18.45.11    0.0.0.0         UG    0      0        0 eth0
lalonde:~# brctl show
bridge name     bridge id               STP enabled     interfaces
lalonde:~# iptables -L -v -n
Chain INPUT (policy ACCEPT 689K packets, 562M bytes)
 pkts bytes target     prot opt in     out     source               destination 
        
   14   976 AdtXenIn   all  --  vif31.0 *       0.0.0.0/0            0.0.0.0/0  
         

Chain FORWARD (policy ACCEPT 1434 packets, 2046K bytes)
 pkts bytes target     prot opt in     out     source               destination 
        
   16  1159 AdtXenIn   all  --  vif31.0 *       0.0.0.0/0            0.0.0.0/0  
         

Chain OUTPUT (policy ACCEPT 477K packets, 38M bytes)
 pkts bytes target     prot opt in     out     source               destination 
        

Chain AdtXenIn (2 references)
 pkts bytes target     prot opt in     out     source               destination 
        
  740 40461 ACCEPT     tcp  --  *      *       0.0.0.0/0            172.18.45.6 
        tcp dpt:80 
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            172.18.45.6 
        tcp dpt:53 
   10   730 ACCEPT     udp  --  *      *       0.0.0.0/0            172.18.45.6 
        udp dpt:53 
   16  1236 ACCEPT     icmp --  *      *       0.0.0.0/0            172.18.45.6 
        
   49  3284 ACCEPT     tcp  --  *      *       0.0.0.0/0            
172.18.45.97        tcp flags:!0x16/0x02 
    5   420 ACCEPT     icmp --  *      *       0.0.0.0/0            
172.18.45.97        
    9  2090 ACCEPT     tcp  --  *      *       0.0.0.0/0            172.18.45.6 
        tcp flags:!0x16/0x02 
    0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            172.18.45.6 
        
    0     0 REJECT     all  --  *      *       0.0.0.0/0            
192.168.0.0/24      reject-with icmp-net-prohibited 
   14  1176 REJECT     all  --  *      *       0.0.0.0/0            
172.16.0.0/12       reject-with icmp-net-prohibited 
    0     0 REJECT     all  --  *      *       0.0.0.0/0            10.0.0.0/8  
        reject-with icmp-net-prohibited 
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0   
        tcp dpt:80 
    0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0   
        
    8   608 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0   
        reject-with icmp-admin-prohibited 
lalonde:~# egrep . /proc/sys/net/ipv4/conf/*/proxy_arp
/proc/sys/net/ipv4/conf/all/proxy_arp:0
/proc/sys/net/ipv4/conf/default/proxy_arp:0
/proc/sys/net/ipv4/conf/eth0/proxy_arp:1
/proc/sys/net/ipv4/conf/lo/proxy_arp:0
/proc/sys/net/ipv4/conf/vif31.0/proxy_arp:1
lalonde:~# ethtool -k eth0
Offload parameters for eth0:
Cannot get device rx csum settings: Operation not supported
Cannot get device tx csum settings: Operation not supported
Cannot get device scatter-gather settings: Operation not supported
Cannot get device tcp segmentation offload settings: Operation not supported
no offload info available
lalonde:~# 


And the command in dom0 I'm using to test:

lalonde:~# ssh -v root@xxxxxxxxxxxx
OpenSSH_3.8.1p1 Debian-8.sarge.4, OpenSSL 0.9.7e 25 Oct 2004
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to 172.18.45.65 [172.18.45.65] port 22.
debug1: Connection established.
debug1: identity file /root/.ssh/identity type 0
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_dsa type 2
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.2p1 
Debian-7ubuntu1
debug1: match: OpenSSH_4.2p1 Debian-7ubuntu1 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_3.8.1p1 Debian-8.sarge.4
debug1: SSH2_MSG_KEXINIT sent

lalonde:~# 


domU:

root@lalonde:~# tcpdump -vvs500 -lnieth0
device eth0 entered promiscuous mode
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 500 bytes
17:56:44.217660 IP (tos 0x0, ttl  64, id 19458, offset 0, flags [DF], proto: 
TCP (6), length: 60) 172.18.45.97.37227 > 172.18.45.65.22: S, cksum 0x15dc 
(correct), 1738938563:1738938563(0) win 5840 <mss 1460,sackOK,timestamp 
31872193 0,nop,wscale 2>
17:56:44.218441 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: TCP 
(6), length: 60) 172.18.45.65.22 > 172.18.45.97.37227: S, cksum 0x8efa 
(correct), 1245122588:1245122588(0) ack 1738938564 win 5792 <mss 
1460,sackOK,timestamp 273577 31872193,nop,wscale 2>
17:56:44.219100 IP (tos 0x0, ttl  64, id 19460, offset 0, flags [DF], proto: 
TCP (6), length: 52) 172.18.45.97.37227 > 172.18.45.65.22: ., cksum 0xcead 
(correct), 1:1(0) ack 1 win 1460 <nop,nop,timestamp 31872193 273577>
17:56:44.231565 IP (tos 0x0, ttl  64, id 15543, offset 0, flags [DF], proto: 
TCP (6), length: 90) 172.18.45.65.22 > 172.18.45.97.37227: P, cksum 0x229a 
(correct), 1:39(38) ack 1 win 1448 <nop,nop,timestamp 273578 31872193>
17:56:44.233412 IP (tos 0x0, ttl  64, id 19462, offset 0, flags [DF], proto: 
TCP (6), length: 52) 172.18.45.97.37227 > 172.18.45.65.22: ., cksum 0xce84 
(correct), 1:1(0) ack 39 win 1460 <nop,nop,timestamp 31872195 273578>
17:56:44.233426 IP (tos 0x0, ttl  64, id 19464, offset 0, flags [DF], proto: 
TCP (6), length: 93) 172.18.45.97.37227 > 172.18.45.65.22: P, cksum 0xb316 
(incorrect (-> 0x4d05), 1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872195 
273578>
17:56:44.437078 IP (tos 0x0, ttl  64, id 19466, offset 0, flags [DF], proto: 
TCP (6), length: 93) 172.18.45.97.37227 > 172.18.45.65.22: P, cksum 0xb316 
(incorrect (-> 0x4cf0), 1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872216 
273578>
17:56:44.857091 IP (tos 0x0, ttl  64, id 19468, offset 0, flags [DF], proto: 
TCP (6), length: 93) 172.18.45.97.37227 > 172.18.45.65.22: P, cksum 0xb316 
(incorrect (-> 0x4cc6), 1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872258 
273578>
17:56:45.697115 IP (tos 0x0, ttl  64, id 19470, offset 0, flags [DF], proto: 
TCP (6), length: 93) 172.18.45.97.37227 > 172.18.45.65.22: P, cksum 0xb316 
(incorrect (-> 0x4c72), 1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872342 
273578>
17:56:47.377132 IP (tos 0x0, ttl  64, id 19472, offset 0, flags [DF], proto: 
TCP (6), length: 93) 172.18.45.97.37227 > 172.18.45.65.22: P, cksum 0xb316 
(incorrect (-> 0x4bca), 1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872510 
273578>

10 packets captured
20 packets received by filter
0 packets dropped by kernel
device eth0 left promiscuous mode
root@lalonde:~# route -vn
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
172.18.45.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
0.0.0.0         172.18.45.97    0.0.0.0         UG    0      0        0 eth0
root@lalonde:~# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:16:3E:7C:AA:7F  
          inet addr:172.18.45.65  Bcast:172.18.45.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fe7c:aa7f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:90 errors:0 dropped:0 overruns:0 frame:0
          TX packets:52 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:15213 (14.8 KiB)  TX bytes:3332 (3.2 KiB)

root@lalonde:~# ethtool -k eth0
Offload parameters for eth0:
Cannot get device rx csum settings: Operation not supported
Cannot get device scatter-gather settings: Operation not supported
Cannot get device tcp segmentation offload settings: Operation not supported
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp segmentation offload: off
root@lalonde:~# 


And just to prove it works:

-davenant:~> traceroute -n 172.18.45.65
traceroute to 172.18.45.65 (172.18.45.65), 30 hops max, 38 byte packets
 1  172.18.45.97  0.334 ms  0.378 ms  0.214 ms
 2  172.18.45.65  0.430 ms  0.258 ms  0.240 ms
-davenant:~> ssh -v root@xxxxxxxxxxxx
OpenSSH_3.8.1p1 Debian-8.sarge.4, OpenSSL 0.9.7e 25 Oct 2004
debug1: Reading configuration data /u/ian/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 23: Deprecated option "RhostsAuthentication"
debug1: Connecting to 172.18.45.65 [172.18.45.65] port 22.
debug1: Connection established.
debug1: identity file /u/ian/.ssh/identity type 0
debug1: identity file /u/ian/.ssh/id_rsa type -1
debug1: identity file /u/ian/.ssh/id_dsa type 2
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.2p1 
Debian-7ubuntu1
debug1: match: OpenSSH_4.2p1 Debian-7ubuntu1 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_3.8.1p1 Debian-8.sarge.4
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-cbc hmac-md5 none
debug1: kex: client->server aes128-cbc hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
The authenticity of host '172.18.45.65 (172.18.45.65)' can't be established.
RSA key fingerprint is 78:9f:f9:40:72:4a:3b:66:33:0f:e1:4a:3b:1f:e3:7d.
Are you sure you want to continue connecting (yes/no)? 

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.