[Xen-users] HVM Linux DomU with PV Drivers - Performance

["Emre ERENOGLU" <erenoglu@xxxxxxxxx>]

Hi Guys,

In order to compare James GPL PV drivers for Windows apple-to-apple to a counterpart, I decided to compile and install the "unmodified_drivers" from xen-unstable tree to my HVM DomU Linux guest. 

My Dom0 is Ubuntu 7.10 , xen-3.2 with 2.6.18 Dom0 kernel.  DomU is Pardus 2007.3 with it's own 2.6.18-86 kernel (stock kernel)

The compiled and loaded drivers are xen-balloon.ko ,  xen-platform-pci.ko , xen-vbd.ko  and xen-vnif.ko from the Xen 3.2 unmodified_drivers folder.

After compiling, I've included them in the initramfs and using a special init script, I booted the system off a PV root disk. Boot partition is still ioemu.
> voyager tmp # mount
> /dev/xvda1 on / type ext3 (rw,errors=remount-ro)
> /dev/hda1 on /boot type ext3 (rw)
> /dev/xvda2 on /home type ext3 (rw,errors=remount-ro)
Here's the output of lsmod
> voyager tmp # lsmod
> Module                  Size  Used by
> nls_utf8                2240  1
> cifs                  212064  1
> serio_raw               7012  0
> joydev                  9472  0
> usbhid                 45824  0
> lp                     11364  0
> tsdev                   7680  0
> i2c_piix4               8332  0
> i2c_core               19968  1 i2c_piix4
> parport_pc             36804  1
> parport                33896  2 lp,parport_pc
> xen_vnif               30768  0
> xen_balloon            15496  1 xen_vnif
> xen_vbd                16496  2
> uhci_hcd               21900  0
> usbcore               116964  3 usbhid,uhci_hcd
> xen_platform_pci       68632  3 xen_vnif,xen_balloon,xen_vbd,[permanent]
As seen, no physical NIC drivers are loaded.  Then it comes to the test. I use iperf for this purpose. Note that I was remotely connected to the DomU through XDMCP, so this may have some little impact even though there were no important screen updates.

 Packet flow from Dom0 to HVM-DomU-w/PV-Drivers : 2.05 Gbits/sec
> root@xenroot:~# iperf -c 192.168.1.7 -p 5002 -t 20
> ------------------------------------------------------------
> Client connecting to 192.168.1.7, TCP port 5002
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.1.2 port 40777 connected with 192.168.1.7 port 5002
> [  3]  0.0-20.0 sec  4.76 GBytes  2.05 Gbits/sec


Packet flow from HVM-DomU-w/PV-Drivers to Dom0:  3,3 Gbits/sec
> voyager tmp # ./iperf -c 192.168.1.2
> ------------------------------------------------------------
> Client connecting to 192.168.1.2, TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.1.7 port 34685 connected with 192.168.1.2 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec  3.84 GBytes  3.30 Gbits/sec


Packet flow from HVM-DomU-w/PV-Drivers to Dom0, when there's also a packet flow from Dom0 to DomU: 3,12 Gbits/sec
> voyager tmp # ./iperf -c 192.168.1.2 -t 20
> ------------------------------------------------------------
> Client connecting to 192.168.1.2, TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.1.7 port 60230 connected with 192.168.1.2 port 5001
> [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-20.0 sec  7.26 GBytes  3.12 Gbits/sec


Packet flow from Dom0 to HVM-DomU-w/PV-Drivers, when there's also a packet flow from DomU to Dom0: 62,9 Mbits/sec

root@xenroot:~# iperf -c 192.168.1.7 -p 5002 -t 30
------------------------------------------------------------

> Client connecting to 192.168.1.7, TCP port 5002
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.1.2 port 54553 connected with 192.168.1.7 port 5002
> [  3]  0.0-30.0 sec    225 MBytes  62.9 Mbits/sec


So in summary:
> Dom0  --> DomU = 2.05 Gbits/sec
> DomU  --> Dom0:  3,3 Gbits/sec
> DomU  --> Dom0, simulatenous with Dom0  --> DomU: 3,12 Gbits/sec
> Dom0  --> DomU, simultaneous with DomU --> Dom0: 62,9 Gbits/sec
Conclusion:
- PV Drivers in HVM Linux setup are really fast
- Dom0 to DomU traffic is severely impacted when there's heavy traffic from DomU to Dom0. (don't know why).

Hope this helps for tuning James' GPL PV Drivers for Windows.

Best regards,

Emre Erenoglu

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users