Re: [Xen-devel] [BUG 1747]Guest could't find bootable device with memory more than 3600M

[George Dunlap <george.dunlap@xxxxxxxxxxxxx>]

On 12/06/13 11:11, Jan Beulich wrote:
> > > > On 12.06.13 at 12:05, George Dunlap <george.dunlap@xxxxxxxxxxxxx> wrote:
> > On 12/06/13 08:25, Jan Beulich wrote:
> > > > > > On 11.06.13 at 19:26, Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>
> > wrote:
> > > > I went through the code that maps the PCI MMIO regions in hvmloader
> > > > (tools/firmware/hvmloader/pci.c:pci_setup) and it looks like it already
> > > > maps the PCI region to high memory if the PCI bar is 64-bit and the MMIO
> > > > region is larger than 512MB.
> > > >
> > > > Maybe we could just relax this condition and map the device memory to
> > > > high memory no matter the size of the MMIO region if the PCI bar is
> > > > 64-bit?
> > > I can only recommend not to: For one, guests not using PAE or
> > > PSE-36 can't map such space at all (and older OSes may not
> > > properly deal with 64-bit BARs at all). And then one would generally
> > > expect this allocation to be done top down (to minimize risk of
> > > running into RAM), and doing so is going to present further risks of
> > > incompatibilities with guest OSes (Linux for example learned only in
> > > 2.6.36 that PFNs in ioremap() can exceed 32 bits, but even in
> > > 3.10-rc5 ioremap_pte_range(), while using "u64 pfn", passes the
> > > PFN to pfn_pte(), the respective parameter of which is
> > > "unsigned long").
> > >
> > > I think this ought to be done in an iterative process - if all MMIO
> > > regions together don't fit below 4G, the biggest one should be
> > > moved up beyond 4G first, followed by the next to biggest one
> > > etc.
> > First of all, the proposal to move the PCI BAR up to the 64-bit range is
> > a temporary work-around.  It should only be done if a device doesn't fit
> > in the current MMIO range.
> >
> > We have three options here:
> > 1. Don't do anything
> > 2. Have hvmloader move PCI devices up to the 64-bit MMIO hole if they
> > don't fit
> > 3. Convince qemu to allow MMIO regions to mask memory (or what it thinks
> > is memory).
> > 4. Add a mechanism to tell qemu that memory is being relocated.
> >
> > Number 4 is definitely the right answer long-term, but we just don't
> > have time to do that before the 4.3 release.  We're not sure yet if #3
> > is possible; even if it is, it may have unpredictable knock-on effects.
> >
> > Doing #2, it is true that many guests will be unable to access the
> > device because of 32-bit limitations.  However, in #1, *no* guests will
> > be able to access the device.  At least in #2, *many* guests will be
> > able to do so.  In any case, apparently #2 is what KVM does, so having
> > the limitation on guests is not without precedent.  It's also likely to
> > be a somewhat tested configuration (unlike #3, for example).
> That's all fine with me. My objection was to Stefano's consideration
> to assign high addresses to _all_ 64-bit capable BARs up, not just
> the biggest one(s).
Oh right -- I understood him to mean, "*allow* hvmloader to map the 
device memory to high memory *if necessary* if the BAR is 64-bit". I 
agree, mapping them all at 64-bit even if there's room in the 32-bit 
hole isn't a good idea.
 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel