[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Struggling with PCI-Passthrough

On 03/25/2014 08:33 PM, Alexander Brychcy wrote:
Am 25.03.2014 11:46, schrieb Gordan Bobic:
On 2014-03-24 19:18, Alexander Brychcy wrote:

As ever there are only problems with domU #2. The first one, that i
could not passthrough an USB controller resolved. No matter at which
port you plug an USB device on the mainboard, it is always assigned to
the XHCI controller.

I found that on Intel ICH USB controllers the PCI USB device that the
USB device is attached to changes dependant on what is connected. There
is some odd routing going on. This is especially the case when a USB
hub is connected to a port. If you just want to pass through, say, a
keyboard, it is stable, but IIRC the device has to be connected to the
port before you re-assign the PCI device that is the port to xen-pcistub.

My motherboard also comes with a NEC USB 3.0 adapter on-board which
doesn't suffer from this problem, so for VMs that require multiple
USB devices I have a USB hub on that controller and I pass the
whole controller to the VM.

My workaround is to add another PCIe USB controller
for the vm, so i'm still able to have usb connection in dom0 for backup
or other purposes.

Sounds similar to what I do. Note that you can also pass specific USB
devices to the domU, rather than using PCI passthrough for the USB
controllers. Unfortunately this comes with ~5% CPU idling overhead
in domU due to USB polling.

Tried to passthrough a NIC to verify if it is a bios related bug. This
works without any problems and nothing freezes at all.

I suspect you are just seeing the Intel ICH USB weirdness I described

I think that should be the same weirdness. I declare the problem solved
:) vm will get their own dedicated controller when everythin else is

If you have a spare slot for a USB card, that is probably the easiest option. The NEC USB 3.0 controller works very well on my system. Having said that, I did manage to make passing through of my ICH USB devices to work as well, it's just fiddly until you get it right.

So last but not least the VGA Passthrough. Issue is still present. HVM
boots and GPU is assigned and working. But after some time respectively
an application tries to access some kind of function the whole system

How much RAM are you passing to the VM? It is possible your motherboard
suffers from the same problem as mine and IOMEM DMA is bypassing the
root PCIe hub. Do the following:

lspci -v | grep "Memory at" | sed -e 's/.*Memory at //' -e 's/ .*//' | sort

Convert the first address that comes back from hex, and set the amount
of memory you are passing to your VM to a few MB below that.

Then see if the crash still happens after a while. If it doesn't, then
there is a good chance that what is happening is that your VM's
memory is trampling over the IOMEM region of the host because it
isn't being translated by the IOMMU, probably because DMA is
bypassing it. In my case this is caused by a bug/feature of NF200
PCIe switches, but it could also be caused by a buggy motherboard
chipset or firmware.

Edited the config to 768MB - way below the address 2051MB. First i
though it works, but only because the 5 Minutes felt like hours. Win7 is
a pain in the ass with such scanty amount of memory.

Your motherboard maps IOMEM as low as 768MB? Seriously?

Anyway, you are saying it crashed anyway, so it may not be the same problem. Then again, it could be - can you check in device manager where the IOMEM for the GPU was mapped in domU? Then cross-check that against the IOMEM ranges given by lspci -vvv and see if there is an overlap. If there isn't, then it's definitely not the same problem. If there is, it could plausibly still be the same problem (only much harder to solve in your case).

Nearly forgot the RAM problem. The maximum amount of RAM i can assign to
the HVM when i passthrough the GPU is 3GB. Haven't tried it yet but may
this patch still be usable with 4.4.0?

Tried to apply the patch, but there were to many changes since 4.2.0 and
currently i don't have to time to find all the changes in the current files.

I think this is for a different bug, but probably worth a try
depending on what results you get with domU memory size reduced to
ensure it cannot trample IOMEM regions.

I think it has definetly something to do with the memory mapping. I want
to assign 8GB of RAM to the hvm. And the dedicated memory of the gpu to
passthrough itself is 4GB.

That can't be right. No standard GPU maps a 4GB aperture. Newer Nvidia cards map 128MB+32MB of IOMEM. ATI, IIRC, map 128MB or 256MB. Certainly not 4GB. You can modify the Nvidia cards' BIOS to enlarge the direct mapped IOMEM area to cover the whole of the GPU RAM (e.g. if you want to use it as a fast RAM disk for swapping when you are not otherwise using the GPU). There may be a similar hack for ATI cards. But either way, no way is all of the VRAM mapped into IOMEM on an unmodified card.

The thread "[Xen-devel] [PATCH v3 0/4] Add
max-ram-below-4g (was Add pci_hole_min_size machine option)" aroused my

Yes, I've been keeping an eye on it, too, but it may not solve your problem if you find that the domU IOMEM overlaps dom0 IOMEM. I lucked out and it doesn't on my machine, but the proper fix would be a generic, adaptive vBAR=pBAR patch. I started working on a patch in that direction (what my half finished working bodge set out to achieve initially, but I've not had much time recently to work on it. The domU memory map is defined in multiple places and I hadn't managed to get them all to agree.


Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.