Re: [Xen-devel] [BUG 1747]Guest could't find bootable device with memory more than 3600M

[George Dunlap <george.dunlap@xxxxxxxxxxxxx>]

On 12/06/13 08:25, Jan Beulich wrote:
> > > > On 11.06.13 at 19:26, Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx> 
> > > > wrote:
> > I went through the code that maps the PCI MMIO regions in hvmloader
> > (tools/firmware/hvmloader/pci.c:pci_setup) and it looks like it already
> > maps the PCI region to high memory if the PCI bar is 64-bit and the MMIO
> > region is larger than 512MB.
> >
> > Maybe we could just relax this condition and map the device memory to
> > high memory no matter the size of the MMIO region if the PCI bar is
> > 64-bit?
> I can only recommend not to: For one, guests not using PAE or
> PSE-36 can't map such space at all (and older OSes may not
> properly deal with 64-bit BARs at all). And then one would generally
> expect this allocation to be done top down (to minimize risk of
> running into RAM), and doing so is going to present further risks of
> incompatibilities with guest OSes (Linux for example learned only in
> 2.6.36 that PFNs in ioremap() can exceed 32 bits, but even in
> 3.10-rc5 ioremap_pte_range(), while using "u64 pfn", passes the
> PFN to pfn_pte(), the respective parameter of which is
> "unsigned long").
>
> I think this ought to be done in an iterative process - if all MMIO
> regions together don't fit below 4G, the biggest one should be
> moved up beyond 4G first, followed by the next to biggest one
> etc.
First of all, the proposal to move the PCI BAR up to the 64-bit range is 
a temporary work-around.  It should only be done if a device doesn't fit 
in the current MMIO range.
We have three options here:
1. Don't do anything
2. Have hvmloader move PCI devices up to the 64-bit MMIO hole if they 
don't fit
3. Convince qemu to allow MMIO regions to mask memory (or what it thinks 
is memory).
4. Add a mechanism to tell qemu that memory is being relocated.

Number 4 is definitely the right answer long-term, but we just don't 
have time to do that before the 4.3 release.  We're not sure yet if #3 
is possible; even if it is, it may have unpredictable knock-on effects.
Doing #2, it is true that many guests will be unable to access the 
device because of 32-bit limitations.  However, in #1, *no* guests will 
be able to access the device.  At least in #2, *many* guests will be 
able to do so.  In any case, apparently #2 is what KVM does, so having 
the limitation on guests is not without precedent.  It's also likely to 
be a somewhat tested configuration (unlike #3, for example).
 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel