Re: [Xen-devel] [BUG 1747]Guest could't find bootable device with memory more than 3600M

["Hao, Xudong" <xudong.hao@xxxxxxxxx>]

> -----Original Message-----
> From: xen-devel-bounces@xxxxxxxxxxxxx
> [mailto:xen-devel-bounces@xxxxxxxxxxxxx] On Behalf Of Stefano Stabellini
> Sent: Friday, June 07, 2013 11:57 PM
> To: George Dunlap
> Cc: Ren, Yongjie; xen-devel@xxxxxxxxxxxxxxxxxxx; Keir Fraser; Stefano 
> Stabellini;
> Tim Deegan; Paolo Bonzini; Xu, YongweiX; Liu, SongtaoX
> Subject: Re: [Xen-devel] [BUG 1747]Guest could't find bootable device with
> memory more than 3600M
> 
> On Fri, 7 Jun 2013, George Dunlap wrote:
> > On 06/07/2013 01:15 PM, Stefano Stabellini wrote:
> > > On Fri, 7 Jun 2013, Xu, YongweiX wrote:
> > > > Hi Stefano Stabellini,
> > > >
> > > > I found a new bug "Guest could't find bootable device with memory more
> > > > than 3600M".
> > > > Attach the link:
> > > > http://bugzilla-archived.xenproject.org//bugzilla/show_bug.cgi?id=1857
> > > >
> > > > When booting up guest(include Linux&Windows guest) with memory more
> than
> > > > 3600M,the guest will show "No bootable device" and could not boot up.
> Then
> > > > the guest will continuous reboot automatically and never found bootable
> > > > device. But with guest memory less than or equal to 3600M, boot up
> > > > successfully.
> > > >
> > > > I found this is the qemu(qemu-upstream-unstable) issue, the latest
> version
> > > > (commit:4597594c61add43725bd207bb498268a058f9cfb) caused this
> issue:
> > > > xen: start PCI hole at 0xe0000000 (same as pc_init1 and
> > > > qemu-xen-traditional)
> > > > author      Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>
> > > >             Wed, 5 Jun 2013 19:36:10 +0800 (11:36 +0000)
> > > > committer   Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>
> > > >             Wed, 5 Jun 2013 19:36:10 +0800 (11:36 +0000)
> > > > commit     4597594c61add43725bd207bb498268a058f9cfb
> > > > tree        d6831f75f4a7d4ad7a94bd4e33584ac358808ee6
> > > > parent      25adf763933faddcc6a62bf55e1c52909a9bafbb
> > > >
> > > > Can you debug this issue soon? Thanks.
> > >
> > > Thank you very much for testing and bisecting the issue!
> > >
> > > The problem is that by default hvmloader sets PCI_MEM_START to
> > > 0xf0000000, then dynamically expands it backwards if needed.
> > > It works with qemu-xen-traditional, because it doesn't do any checking
> > > when registering memory regions.
> > > It doesn't work with qemu-xen because it needs to know where the ram and
> > > where the pci hole are when building the emulated machine.
> > > We can't have hvmloader increasing the pci hole in a way that overlaps
> > > with the guest ram (or where qemu thinks that the guest ram is).
> > > It is also worth noting thet seabios has its own view of where the pci
> > > hole starts and at the moment is set to 0xe0000000 at build time.
> > >
> > > I can see two ways of fixing this now:
> > >
> > > 1) have everybody agree about the pci hole starting at 0xe0000000
> > >
> > > - set PCI_MEM_START in hvmloader to 0xe0000000 to match qemu's view
> of
> > > the pci hole and have a bigger pci hole by default
> > >
> > > - remove the loop in hvmloader to increase the pci hole
> > >
> > > - set HVM_BELOW_4G_RAM_END to 0xe0000000, so that
> low_mem_pgend is set
> > > accordingly in tools/libxc/xc_hvm_build_x86.c:build_hvm_info
> > >
> > >
> > > 2) have everybody agree about the pci hole starting at 0xf0000000
> > >
> > > - revert 4597594c61add43725bd207bb498268a058f9cfb in qemu-xen
> > >
> > > - set BUILD_PCIMEM_START to 0xf0000000 in seabios
> > >
> > > - remove the loop in hvmloader to increase the pci hole
> > >
> > >
> > > Given that in both cases we need to remove the loop to increase the pci
> > > hole in hvmloader I would rather go with 1) and have a bigger pci hole
> > > by default to avoid problems with the pci hole being too small.
> > >
> >
> > Did we ever figure out what actually happens on real hardware?  Or on KVM?
> I
> > have a hard time believing that real hardware is hard-coded -- who would be
> > the person to ask about that, do you reckon?
> 
> For KVM I am CC'ing Paolo, for real hardware I'll let the Intel guys
> speak.
> To be clear the question is:
> 
> What happens on real hardware when the BIOS boots and finds out that the
> PCI hole is too small to contain all the MMIO regions of the PCI devices?
> 

I do not know what does BIOS do details, but I think it's in 4G above if pci 
hole is too small. I saw native OS program the pci hole to a very high addr 
0x3c0c00000000 if the pci device has large bar size.

For Xen hvmloader, it already policy that is: if PCI hole is too small, another 
MMIO region will be created starting from the high_mem_pgend.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel