[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] lots of dropped packets on vif interfaces

To: xen-users@xxxxxxxxxxxxxxxxxxx
From: Joel Heenan <joelh@xxxxxxxxxxxxxx>
Date: Thu, 6 Aug 2009 10:42:31 +1000
Delivery-date: Wed, 05 Aug 2009 17:43:34 -0700
List-id: Xen user discussion <xen-users.lists.xensource.com>

Xen Users,

We recently attempted an upgrade only to have to roll back because of very unusual network problems. Our software is supposed to setup 10 TCP connections and maintain them yet when I looked with a "watch netstat" showed the connections constantly being dropped and re-established. Looking at the dom0 I see lots and lots of dropped packets. Does anyone know under what conditions a vif interface would drop packets?

We started the upgrade with jumbo frames but fearing that this was the issue we rolled them back. They were implemented by having a modified vif-bridge script

73a78,79
>       do_without_error ifconfig "$vif" mtu 9000
>       (sleep 10; ifconfig "$vif" mtu 9000) &
74a81
>

and having a script which went through and periodically set the vif interfaces to mtu 9000. We changed the MTU's back to 1500 and disabled the periodic but still we saw lots of dropped TCP connections. Unfortunately it was only post-upgrade that we noticed the dropped packets on the dom0.

CentOS 5.3, Xen 3.03, Kernel 2.6.18-128.2.1.el5xen

ifconfig output:
"""
bond0     Link encap:Ethernet HWaddr 00:15:17:90:55:A0
          inet addr:192.168.12.11 Bcast:192.168.255.255 Mask:255.255.0.0
          inet6 addr: fe80::215:17ff:fe90:55a0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:4127440 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2011946 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:3137102607 (2.9 GiB) TX bytes:4043618659 (3.7 GiB)

eth0      Link encap:Ethernet HWaddr 00:30:48:C5:5A:CE
          inet addr:208.85.150.68 Bcast:208.85.150.127 Mask:255.255.255.192
          inet6 addr: fe80::230:48ff:fec5:5ace/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:40971777 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4868631 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:8325395107 (7.7 GiB) TX bytes:1774099435 (1.6 GiB)

eth1      Link encap:Ethernet HWaddr 00:30:48:C5:5A:CF
          inet addr:10.12.1.11 Bcast:10.12.255.255 Mask:255.255.0.0
          inet6 addr: fe80::230:48ff:fec5:5acf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:4811203 errors:0 dropped:0 overruns:0 frame:0
          TX packets:105483 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1893191654 (1.7 GiB) TX bytes:19080833 (18.1 MiB)

eth2      Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
          RX packets:109015823 errors:0 dropped:0 overruns:0 frame:0
          TX packets:61527076 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:124155590545 (115.6 GiB) TX bytes:7949776919 (7.4 GiB)
          Memory:da320000-da340000

eth3      Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
          RX packets:87295647 errors:0 dropped:0 overruns:0 frame:0
          TX packets:208743472 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:79482318567 (74.0 GiB) TX bytes:131665787763 (122.6 GiB)
          Memory:da360000-da380000

eth4      Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
          RX packets:368798651 errors:0 dropped:0 overruns:0 frame:0
          TX packets:77141330 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:358699531637 (334.0 GiB) TX bytes:11545699506 (10.7 GiB)
          Memory:da420000-da440000

eth5      Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
          RX packets:79557756 errors:0 dropped:0 overruns:0 frame:0
          TX packets:219450792 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:94309068673 (87.8 GiB) TX bytes:251020520979 (233.7 GiB)
          Memory:da460000-da480000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING MTU:16436 Metric:1
          RX packets:10950 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10950 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1322596 (1.2 MiB) TX bytes:1322596 (1.2 MiB)

pbond0    Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MASTER MTU:1500 Metric:1
          RX packets:644667877 errors:0 dropped:0 overruns:0 frame:0
          TX packets:566862670 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:656646509422 (611.5 GiB) TX bytes:402181785167 (374.5 GiB)

peth0     Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:41399478 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5177320 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:8392686228 (7.8 GiB) TX bytes:1794386410 (1.6 GiB)
          Memory:da020000-da040000

peth1     Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:2471814 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3417446 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:362232436 (345.4 MiB) TX bytes:3259230903 (3.0 GiB)
          Memory:da060000-da080000

vif0.0    Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:4868631 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40971795 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1774099435 (1.6 GiB) TX bytes:8325397390 (7.7 GiB)

vif0.1    Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:2011948 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4127418 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:4043620328 (3.7 GiB) TX bytes:3137070612 (2.9 GiB)

vif0.2    Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:105483 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4811203 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:19080833 (18.1 MiB) TX bytes:1893191654 (1.7 GiB)

vif15.0   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16167343 errors:0 dropped:2605 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:496 (496.0 b) TX bytes:3048611500 (2.8 GiB)

vif15.1   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:25545589 errors:0 dropped:0 overruns:0 frame:0
          TX packets:86572687 errors:0 dropped:14971 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:218968845538 (203.9 GiB) TX bytes:12499004215 (11.6 GiB)

vif15.2   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:90628 errors:0 dropped:0 overruns:0 frame:0
          TX packets:148849 errors:0 dropped:582 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:43151873 (41.1 MiB) TX bytes:19783310 (18.8 MiB)

vif41.0   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:67 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15133388 errors:0 dropped:2917 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:2092 (2.0 KiB) TX bytes:2784566746 (2.5 GiB)

vif41.1   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:18750910 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5144760 errors:0 dropped:299090 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:2825180056 (2.6 GiB) TX bytes:2525900266 (2.3 GiB)

vif41.2   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:152588 errors:0 dropped:0 overruns:0 frame:0
          TX packets:72101 errors:0 dropped:1194 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:88969564 (84.8 MiB) TX bytes:9209610 (8.7 MiB)

vif43.0   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:30 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14733600 errors:0 dropped:1573 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:1056 (1.0 KiB) TX bytes:2709705832 (2.5 GiB)

vif43.1   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:24540834 errors:0 dropped:0 overruns:0 frame:0
          TX packets:36136102 errors:0 dropped:11070 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:68309891084 (63.6 GiB) TX bytes:4202123253 (3.9 GiB)

vif43.2   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:566939 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1134448 errors:0 dropped:579 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:2593043740 (2.4 GiB) TX bytes:84875829 (80.9 MiB)

vif45.0   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12104256 errors:0 dropped:2097 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:496 (496.0 b) TX bytes:2217362387 (2.0 GiB)

vif45.1   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:11496076 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21208103 errors:0 dropped:76369 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:826925754 (788.6 MiB) TX bytes:28052880670 (26.1 GiB)

vif45.2   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:36670 errors:0 dropped:0 overruns:0 frame:0
          TX packets:53919 errors:0 dropped:1042 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:3744887 (3.5 MiB) TX bytes:5659141 (5.3 MiB)

vif46.0   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12046566 errors:0 dropped:1895 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:496 (496.0 b) TX bytes:2206365310 (2.0 GiB)

vif46.1   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:15506324 errors:0 dropped:0 overruns:0 frame:0
          TX packets:24790384 errors:0 dropped:40576 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:1061414707 (1012.2 MiB) TX bytes:27871247434 (25.9 GiB)

vif46.2   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:36491 errors:0 dropped:0 overruns:0 frame:0
          TX packets:52028 errors:0 dropped:1016 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:3696538 (3.5 MiB) TX bytes:5193571 (4.9 MiB)

vif47.0   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11998949 errors:0 dropped:1581 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:496 (496.0 b) TX bytes:2197379798 (2.0 GiB)

vif47.1   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:3120044 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4318924 errors:0 dropped:64900 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:220041974 (209.8 MiB) TX bytes:4477173983 (4.1 GiB)

vif47.2   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:53782 errors:0 dropped:0 overruns:0 frame:0
          TX packets:67304 errors:0 dropped:564 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:5578127 (5.3 MiB) TX bytes:6662880 (6.3 MiB)

vif48.0   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11977782 errors:0 dropped:1645 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:496 (496.0 b) TX bytes:2193205238 (2.0 GiB)

vif48.1   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:58647 errors:0 dropped:0 overruns:0 frame:0
          TX packets:161069 errors:0 dropped:47969 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:11373930 (10.8 MiB) TX bytes:78154482 (74.5 MiB)

vif48.2   Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:38945 errors:0 dropped:0 overruns:0 frame:0
          TX packets:53491 errors:0 dropped:1092 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:4114153 (3.9 MiB) TX bytes:5008162 (4.7 MiB)

virbr0    Link encap:Ethernet HWaddr 00:00:00:00:00:00
          inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
          inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:32 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b) TX bytes:7273 (7.1 KiB)

xenbr0    Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:146301 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:7282456 (6.9 MiB) TX bytes:0 (0.0 b)

xenbr1    Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:248368 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:147614308 (140.7 MiB) TX bytes:0 (0.0 b)

xenbr2    Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
          RX packets:37916 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2005907 (1.9 MiB) TX bytes:0 (0.0 b)
"""

Joel

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

Prev by Date: [Xen-devel] HOWTO use Virtual Hard Disk - VHD with Xen domUs?
Next by Date: Re: [Xen-users] What can collapse all the domU????????'
Previous by thread: [Xen-devel] HOWTO use Virtual Hard Disk - VHD with Xen domUs?
Next by thread: [Xen-users] Load Balancing
Index(es):
- Date
- Thread

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.