PCI passthrough: possible bug in memory relocation

[Mateusz <mati7337@xxxxxxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hello,
I'm working on resolving a bug in Qubes OS. The problem is that
when someone gives a VM too much RAM the GPU passthrough doesn't
work as intended. I think it's because in these cases RAM overlaps
with PCI addresses and memory relocation doesn't work in Xen.

Here are the memory BARs of the GPU I've tried to passthrough:
Memory at f3000000 (32-bit, non-prefetchable) [size=16M]
Memory at e0000000 (64-bit, prefetchable) [size=256M]
Memory at f0000000 (64-bit, prefetchable) [size=32M]

The interesting thing is that in xl dmesg hvmloader prints these
lines:
(d1) Relocating 0xffff pages from 0e0001000 to 187000000 for lowmem MMIO hole
(d1) Relocating 0x1 pages from 0e0000000 to 196fff000 for lowmem MMIO hole
so it looks like it tries to move these pages to a different address
to make space for PCI memory, but for some reason it doesn't work.
Changing TOLUD to 3.5G solves the problem.
I tested it on xen 4.14.3
Here's the issue on github regarding this problem:
https://github.com/QubesOS/qubes-issues/issues/4321

Is it a bug in Xen and fixing it would fix the problem or is there
something I'm missing?
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEE0RoD3+S7b5zXZ6lW6IGQZRJEAKsFAmJKKjwACgkQ6IGQZRJE
AKtoDQ/+MQmH72WfmkCzbvpfmzsMW4xWpsx9Q6yZg58eEVv/pT06CC1KD8KUnyN3
ChqBNUQ3W2CZtJG0pJ1eON2wlByFuwnnl67MX0D0oKQQDfEf/oVEUspcXvfdgzR9
vOSqkZTKmZRjKOujrbiyO0Ooyrx7ED8IJE9UcfGzi8OYsYCbRg6vZtMghYoep6c4
w2ho7XLFO0pRjzzO+ZMIWzWJq2yp4H7/8F7QXv57tcszoWCrm/THkM8AGE51ilus
N8S3mlVJM+h2hWpZus9BTxnSULNby80nK3MfWAo6XSTPc/uRo3+NegBhFOFBNkJq
YDC+sbLj9Nr4/4uOKl+lLvg0HBSr/b0qebwE1EAwlj3k9byfFjBHCrf63as7AtO0
AuCE9hFqVE+vM8HPghhvcwzuUywwAljD8kxtvKT73JP/+F7v7PfKr6zMatfGcIMP
LYX7X64dMDsi9ySbjRyWWQFPEYlDKYGFA+88qJQZISQXSykO7QqKxeGpMKL/VUhX
BTLNLa568UReIhOzR5jvzDSl6c2kSxZluKpIuQIRE1iqShnUQPbbykA5zI6NzZOk
L5k/RJuRBYoKh/BWbqZsnLOrkYOltaWmFnHb3PPWNo3bVsQVl0ifkcaEoko6KJ0r
nhbpoieuL7Yn/o3aVLPOI7PP4Qy9LqC47L7h1BsojA75JfeFak8=
=eUPS
-----END PGP SIGNATURE-----