All,
similar to some mail threads found in this forum and some
other xen-related threads, I had problems with the network performance of my
test system:
- software base of dom0/domU:
RHEL5 (Xen 3.0.3, Redhat 2.6.18-8el5xen SMP kernel)
- ibm x306 servers with
3Ghz P4 /MT support; coupled via Gigabit Ethernet switch
- standard xen bridging
network configuration
- test tool: iperf
- Xen domUs working in
PV mode (the P4 does not support VT)
the data transfer rates with ‘iperf’ were
as follows:
·
dom0/machine 1 =>
dom0/machine 2 ~800MBit/s
·
domU/machine 1 =>
dom0/machine 2 ~700MBit/s
·
dom0/machine 1 =>
domU/machine 2 ~ 40MBit/s
this flaw of the last test case and the difference between
test case 2 and 3 remained more or less constant with various configs of the test
systems:
- credit or sedf
scheduler
- various configs of
the schedulers
- copy mode and
flipping mode of netfront driver
A detailed analysis with tcpdump/wireshark showed that
there must be some losses of data within the TCP stream, resulting in TCP
retransmissions and therefore breaks within data transfer (in one test case I saw
a transmission gap of 200 ms caused by TCP retransmissions every 230 ms - explaining
the breakdown of the data rate).
Now, looking for the reason for the data losses (this
was the reason why I checked the copy mode of the netfront driver) I noticed
that the txqueuelen parameter of the vif devices connecting the bridge to the
domUs were set to ‘32’ (no idea where and for what reason this
value is configured initially - note that the txqueuelen value for Ethernet
devices is set to 1000 ).
After changing this parameter to higher values (128-512)
I got a much higher performance in test case 3: tcp throughput now reaches
values of 700MBit/s and higher; using iperf –d option (tcp data streams
in both directions) now gave sum values of more than 900 MBit/s.
I’ll evaluate also the other test cases
parameter settings to find out the best setting of the parameters, but I think
a suited configuration of the txqueuelen parameter of the vif interfaces will
be most important for getting a good network performance for a configuration as
described above (comparable to other virtualization solutions)
Regards
Werner
