Re: [win-pv-devel] Windows on Xen bad IO performance

[Jakub Kulesza <jakkul@xxxxxxxxx>]

2018-07-31 9:51 GMT+02:00 Paul Durrant <Paul.Durrant@xxxxxxxxxx>:
>
> De-htmling... Responses below...
>
> -----
> From: win-pv-devel [mailto:win-pv-devel-bounces@xxxxxxxxxxxxxxxxxxxx] On 
> Behalf Of Jakub Kulesza
> Sent: 30 July 2018 16:08
> To: win-pv-devel@xxxxxxxxxxxxxxxxxxxx
> Subject: [win-pv-devel] Windows on Xen bad IO performance
>
> I have a number of different hosts with different xen and windows versions, 
> but they all share the same thing. Each time I install xen windows pv drivers 
> 8.2.0 from here: https://www.xenproject.org/developer...v-drivers.html I'm 
> getting worse IO performance than before, on standard Windows drivers.
>
[cut]
>
> I found out that I need to modify the gnttab_max_frames parameter to the xen 
> hypervisor at boottime. A lot of links and reading starts here: 
> https://wiki.gentoo.org/wiki/Xen#Xen..._kernel_4.3.2B
>
> I did some testing and I am very confused right now. The gnttab_max_frames is 
> by default 32 (increased to 64 in some xen version), and to solve the issues 
> i would need to set it higher to 256. The results I get seem to show 
> something totally different.
>
> New test rig:
> • ubuntu 18.04 LTS with everything from normal repositories, updated, xen 4.9
> • i5-8500, 16GB ram, Samsung 850 evo SSD,
> • windows 2016 installed on a LVM volume,
> • xen pv drivers 8.2.0 installed on Windows,
> • logged to the VM using VNC from a laptop in the same local network.
>
> I've tested this at a number of values of gnttab_max_frames from 4 to 4096.
>
> Passmark provides consistent results at around 510 MB/s READ, 305 MB/s WRITE, 
> 330 MB/s Random ReadWrite, regardless of the setting of gnttab_max_frames. I 
> guess that it does not saturate the grant tables mechanism of XEN that much. 
> But with ATTO, the situation is sooo different.
> • gnttab_max_frames = 4
> o Windows is very snappy, responsive, even under heavy load from ATTO.
> o Atto shows good results, with some signs of saturation with packets bigger 
> than 512KB.
> • gnttab_max_frames = 10
> o Windows is very snappy but stops being responsive, even under heavy load 
> from ATTO.
> o Atto shows mediocre results, saturation is very high with packets bigger 
> than 512KB.
> • gnttab_max_frames = 64
> o You can feel that the windows windows open a little bit slower, system 
> feels dead with high load from ATTO.
> o Atto shows bad results, saturation kills the system with packets bigger 
> than 512KB. System is getting back OK after ATTO finishes.
> • gnttab_max_frames = 256
> o Even worse than 64, the results show similarity to 64, but the system just 
> did not react. I fed up with waiting.
> • gnttab_max_frames = 4096
> o Windows did not boot. I just got fed up with waiting.
[cut]

>
> As discussed on IRC, it would be useful if you tried the 8.2.2 drivers and 
> also highly useful if you could capture logging from QEMU.
>
> One other thing that occurs to me is that XENVBD implements indirect granting 
> but this is relatively under tested because the only backend that implements 
> it is blkback, and we don't use that in XenServer. Whilst is may be slower 
> overall, you might get more stability using QEMU qdisk. (We have a couple of 
> performance fixes for this in the pipeline in Citrix as we are now starting 
> to use it as our default backend, but it should be reasonable as-is).
>
>   Paul

I did test 8.2.2 PV drivers. Did not managed to get QEMU logging thou.
Will read more and retry.

Results on the i5-8500 rig - everything set the same as in the tests
mentioned above:

https://imgur.com/gallery/PTm5f4G

gnttab_max_frames = 4:
no signs or very little signs of saturation, everything is flying,
scores are better than with 8.2.0

gnttab_max_frames = default for ubuntu 18.04 (so 32 or 64)
saturation, system goes unresponsive, as bad as before

gnttab_max_frames = 256
saturation, system goes unresponsive, as bad as before

Passmark shows better results on all gnttab_max_frames settings:
Read: 514-515 (same as 8.2.0)
Write: 477 (better!)
Random ReadWrite: 300-360 (same as 8.2.0)

Is this behaviour (lowering max frames to get better results) working
as expected?

How low should I NOT go with max_frames?

Does XenServer recommend any windows guest drivers if used with qemu backend?


-- 
Pozdrawiam
Jakub Kulesza

_______________________________________________
win-pv-devel mailing list
win-pv-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/win-pv-devel