Re: [Xen-devel] Bug: Limitation of <=2GB RAM in domU persists with 4.3.0

[George Dunlap <george.dunlap@xxxxxxxxxxxxx>]

On 31/07/13 20:35, Gordan Bobic wrote:
> On 07/31/2013 06:53 PM, George Dunlap wrote:
> > On Fri, Jul 26, 2013 at 2:11 PM, Gordan Bobic <gordan@xxxxxxxxxx> wrote:
> > > > Now that is intereting - if this makes the memory holes the same 
> > > > between
> > > > the guest and the host, does it also implicitly vBAR=pBAR?
> > >
> > > Another thing that occurred to me might be useful to check - it is
> > > pretty easy to modify the BAR size on Nvidia cards. The defaults are
> > > 64MB and 128MB for the two BARs. They can be made much, much larger,
> > > and there is often advantage to enlarging them to at least be equal to
> > > VRAM size. Soooooo... If I boost the BAR from 128MB to 2GB, being a
> > > 64-bit BAR, it might make the BIOS do the sane thing and map it above
> > > 4GB. With the other BAR also suitably enlarged and it being done on
> > > the second GPU as well, there is no obvious option but to map them
> > > above 4GB (unless the BIOS is broken, which it may well be, in
> > > which case all bets are off).
> > >
> > > Which may just alleviate the memory issue if not completely fix
> > > the problem.
> > >
> > > Will try this and see what happens.
> > I believe XenServer has a patch that allows the toolstack (in this
> > case xapi) to set the default size of the MMIO hole.  Andrew, did that
> > ever make it upstream?
> >
> > Unfortunately, it is unlikely to work with upstream qemu until we fix
> > the memory relocation issue...
> Interesting you should mention something like this. I've been 
> pondering whether it might be easier (even if it is a bodge) to simply 
> always set the domU E820 map to have 0x80000000 - 0xFFFFFFFF 
> (2GB->4GB) reserved. I have not yet seen a motherboard that maps 
> 32-bit BARs below 2GB.
I'm pretty sure we've seen a memory hole larger than 2GiB, in a box 
loaded up with a boatload of GPUs.
The main problem with doing this unconditionally is that the relocated 
memory isn't available to non-PAE 32-bit guests.  I think we should have 
a work-around in place for 4.4 that will avoid a collision between the 
host MMIO and guest memory addresses; but it will need to be off by 
default, at least for guests that don't have a passed-through device.
 -George


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel