[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH] libxl, hvmloader: Don't relocate memory for MMIO hole

On 17/06/13 17:43, George Dunlap wrote:
At the moment, qemu-xen can't handle memory being relocated by
hvmloader.  This may happen if a device with a large enough memory
region is passed through to the guest.  At the moment, if this
happens, then at some point in the future qemu will crash and the
domain will hang.  (qemu-traditional is fine.)

It's too late in the release to do a proper fix, so we try to do
damage control.

hvmloader already has mechanisms to relocate memory to 64-bit space
if it can't make a big enough MMIO hole.  By default this is 2GiB; if
we just refuse to make the hole bigger if it will overlap with guest
memory, then this codepath will be taken by default.

Signed-off-by: George Dunlap <george.dunlap@xxxxxxxxxxxxx>
CC: Ian Campbell <ian.campbell@xxxxxxxxxx>
CC: Ian Jackson <ian.jackson@xxxxxxxxxx>
CC: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
CC: Hanweidong <hanweidong@xxxxxxxxxx>

This isn't exactly what we had discussed, but looking at the code, I think this is the easiest thing to do.

A brief description of what hvmloader does:
1. enumerate all BARs from all devices, ordering them from large to small
2. Double* the size of the <4G MMIO hole until it is big enough to fit all the MMIO, or until it reaches 2GiB, whichever comes first
3. Move guest memory which overlaps the <4G MMIO hole above 4G.
4. Place devices in memory, placing devices above 4G if the hole isn't big enough for everything.

Unfortunately, the option of "just don't map the passed-through device if there's not enough MMIO space" doesn't really work -- there's no way to really tell which device is the one you shouldn't map.

Of course we have the same issue with relocating devices to 64-bit: we can't tell which devices will be located above 4G or not. So for devices not capable of accessing physical address space above 4G, random devices will simply not appear; this may cause strange problems if the emulated network card or disk, for example, is not visible. There may also be examples of assumptions baked into drivers or emulation code about the physical address space. On the whole, given that this is KVM's behavior, I'm inclined to be optimistic about the state of guest OSes and drivers.

Ian Jackson had proposed just crashing in hvmloader if we were using qemu-xen and needed to relocate memory. This changes a "crash randomly at some point after boot" into a "crash immediately before the BIOS even runs". While that is certainly an improvement, I think on the whole that it's still bad enough that "mostly works but occasionally has strange failures" is still better; hence this patch.


Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.