|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-users] Xen dom0/domU networking performance issue
Hi,I have a group of physical xen servers (dom0) which have been working well for some time. However, recently I converted an existing Linux machine to a domU and have been having networking issues with this domU. Specifically, if from a physical machine on the network, I ping the domU, then I get normal ping response times, but for some periods (assuming when under a heavier load) the ping time is slow, though no packets are ever dropped. This impacts all domU's and the dom0 at the same time, moving the domU (live migrate) to a different dom0 causes the other dom0 to have the same problem (ie, the problem follows the domU). ie, 20140529-14:53:18 x.x.10.9 50 pings in the range 0.18 to 0.27ms 0.27 157.61 117.40 472.00 623.85 956.83 1360.31 1472.61 2045.73 2528.93 2304.32 2651.66 2853.93 - - -The last three pings were lost (ie, more than three seconds, and fping didn't wait for them). When testing manually, if I wait long enough, no packets are ever lost, but the delay can get very high). This is the exact same period pinging to the dom0 on the xen bridge interface x.x.10.34 0.16 0.16 372.27 453.27 820.93 985.35 1358.14 1747.34 2000.70 2370.45 2660.26 2356.98 2859.98 - - -Pinging to other physical network interfaces on the same dom0 is not affected. eg in the same period of time: 2997 packets transmitted, 2997 received, 0% packet loss, time 2996002ms rtt min/avg/max/mdev = 0.044/0.091/0.410/0.018 msNo slow/delayed pings at all on the iSCSI network interface (not bridged for xen, not exposed to the domU's 2998 packets transmitted, 2998 received, 0% packet loss, time 2997024ms rtt min/avg/max/mdev = 0.084/6.858/1232.196/69.792 ms, pipe 2 Some delayed pings, up to 1.2 seconds as seen here: 64 bytes from x.x.10.34: icmp_seq=2211 ttl=64 time=0.294 ms 64 bytes from x.x.10.34: icmp_seq=2212 ttl=64 time=0.287 ms 64 bytes from x.x.10.34: icmp_seq=2213 ttl=64 time=0.288 ms 64 bytes from x.x.10.34: icmp_seq=2214 ttl=64 time=317 ms 64 bytes from x.x.10.34: icmp_seq=2215 ttl=64 time=724 ms 64 bytes from x.x.10.34: icmp_seq=2216 ttl=64 time=1155 ms 64 bytes from x.x.10.34: icmp_seq=2217 ttl=64 time=1232 ms 64 bytes from x.x.10.34: icmp_seq=2218 ttl=64 time=1202 ms 64 bytes from x.x.10.34: icmp_seq=2219 ttl=64 time=838 ms 64 bytes from x.x.10.34: icmp_seq=2220 ttl=64 time=556 ms 64 bytes from x.x.10.34: icmp_seq=2221 ttl=64 time=229 ms 64 bytes from x.x.10.34: icmp_seq=2222 ttl=64 time=0.266 ms 64 bytes from x.x.10.34: icmp_seq=2223 ttl=64 time=0.239 ms 64 bytes from x.x.10.34: icmp_seq=2224 ttl=64 time=0.238 ms 64 bytes from x.x.10.34: icmp_seq=2225 ttl=64 time=0.265 ms System details of dom0 (Debian Stable): root@pm04:~# uname -an Linux pm04 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3+deb7u1 x86_64 GNU/Linux root@pm04:~# dpkg -l | grep xen ii libxen-4.1 4.1.4-3+deb7u1 amd64 Public libs for Xenii libxenstore3.0 4.1.4-3+deb7u1 amd64 Xenstore communications library for Xen ii xen-hypervisor-4.1-amd64 4.1.4-3+deb7u1 amd64 Xen Hypervisor on AMD64 ii xen-linux-system-3.2.0-4-amd64 3.2.57-3+deb7u1 amd64 Xen system with Linux 3.2 on 64-bit PCs (meta-package) ii xen-linux-system-amd64 3.2+46 amd64 Xen system with Linux for 64-bit PCs (meta-package) ii xen-system-amd64 4.1.4-3+deb7u1 amd64 Xen System on AMD64 (meta-package) ii xen-utils-4.1 4.1.4-3+deb7u1 amd64 XEN administrative tools ii xen-utils-common 4.1.4-3+deb7u1 all Xen administrative tools - common files ii xenstore-utils 4.1.4-3+deb7u1 amd64 Xenstore utilities for Xen domU config file: name = "mail" hostname = 'mail' kernel = "/mnt/images/server_images/mail/vmlinuz-3.2.0-4-686-pae" ramdisk = "/mnt/images/server_images/mail/initrd.img-3.2.0-4-686-pae" root = "/dev/xvda1" disk = ['phy:/dev/mapper/mpathmail,xvda,w' ] memory = 2048 cpus = "2,3" vcpus = 2vif = ['bridge=xenbr5, mac=00:16:3e:43:a8:09', 'bridge=xenbr0, mac=00:16:3e:43:d8:09']
The dom0 network config:
cat /etc/network/interfaces
# The primary network interface
allow-hotplug eth0
auto xenbr0
iface xenbr0 inet static
address x.x.10.34
netmask 255.255.240.0
gateway x.x.10.254
bridge_maxwait 5
bridge_ports regex eth0
# This is the vlan which has the DSL modem plus the problem domU/VM
auto xenbr5
iface xenbr5 inet manual
bridge_ports eth0.5
# Used by dom0 for iscsi SAN only
auto eth1
iface eth1 inet static
address x.x.16.34
netmask 255.255.255.0
mtu 9000
post-up /etc/network/ifup.eth1
# Used by dom0 for iscsi SAN only
auto eth2
iface eth2 inet static
address x.x.16.44
netmask 255.255.255.0
mtu 9000
post-up /etc/network/ifup.eth2
xm vcpu-list
Name ID VCPU CPU State Time(s) CPU
Affinity
Domain-0 0 0 0 -b- 965.7 0 Domain-0 0 1 1 r-- 662.4 1 mail 4 0 2 -b- 1861.0 2 mail 4 1 3 -b- 1134.2 3 oldcal 5 0 4 -b- 80.9 4-5 oldcal 5 1 5 -b- 32.7 4-5 xm network-list mail Idx BE MAC Addr. handle state evt-ch tx-/rx-ring-ref BE-path0 0 00:16:3e:43:a8:09 0 4 14 768 /769 /local/domain/0/backend/vif/4/0 1 0 00:16:3e:43:d8:09 1 4 15 770 /1280 /local/domain/0/backend/vif/4/1 The domU is running: # uname -an Linux mail 3.2.0-4-686-pae #1 SMP Debian 3.2.57-3+deb7u1 i686 GNU/LinuxThe domU is the gateway/firewall for the network which routes between the internal network and Internet, also runs email/proxy servers etc. Can anyone suggest what I could do to resolve this issue? Additional information I should provide? Thanks, Adam -- Adam Goryachev Website Managers www.websitemanagers.com.au _______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxx http://lists.xen.org/xen-users
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |