[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] [PCI passthrough] My experience with PCI memory stomp/corruption issues when passing through multiple PCI devices in Xen

  • To: xen-users@xxxxxxxxxxxxx
  • From: Gordan Bobic <gordan@xxxxxxxxxx>
  • Date: Fri, 19 Sep 2014 14:49:21 +0100
  • Delivery-date: Fri, 19 Sep 2014 13:50:37 +0000
  • List-id: Xen user discussion <xen-users.lists.xen.org>

On 2014-09-16 20:55, Andrej Komelj wrote:

After a lot of Googling, I found a potential culprit - PCI hole and
all the magic behind remapping the devices and/or memory when the
default hole (roughly 200 MB) is too small for all the pass-though PCI
devices in the system (I need roughly 300 MB for both VGA and USB
controller to work).

I never realized the default hole is only 200MB. What is a little more
surprising is that your USB controller requires such a large chunk of
IOMEM. My USB controllers take between 1KB (Intel ICH10 USB 2) and 8KB
(NEC USB 3).

By remapping VGA adapter to 64-bit space, I was able to boot domU and
use both VGA and USB controller at the same time. The only thing that
bothered me was the inability to see any boot messages in domU until X
environment was loaded and the graphics card reset.

You should get output via the emulated adapter over VNC until the
GPU driver loads in domU. You only get that output on the passed
through GPU with primary passthrough, which required side-loading
the VBIOS. IIRC that doesn't work for any recent VBIOS since they
are bigger than 64KB. Or has this been fixed recently? Does primary
passthrough work now with a VBIOS > 64KB?

The key point is - if I set low memory bar in HVM guest's E820 map to
0xe0000000 (right up to the beginning of PCI hole size), something
messes up with my USB PCI controller and the thing simply doesn't
work. Setting the low memory bar lower apparently leaves some buffer
space for whomever or whatever is poking at that addresses and my USB
controller now works as charm. :-D It might be a hardware bug (saw a
lot of reports in xen-devel) or a software bug - I don't know.

It might be worth checking what your physical e820 map looks like.
The problem I was having with my motherboard was that if domU
memory map overlapped the IOMEM, the domU memory I/O would happily
trample the dom0's IOMEM regions mapped to various hardware devices
which would very quickly crash the host.

Can you check if you are having such overlaps, e.g. by only
assigning your domU the memory amount up to your first IOMEM


Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.