Re: [Xen-devel] qemu-xen-dir + PCI passthrough = BOOM

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

Friday, January 10, 2014, 4:19:15 PM, you wrote:

> On Thu, Jan 09, 2014 at 10:28:47PM -0500, Konrad Rzeszutek Wilk wrote:
>> On Thu, Jan 09, 2014 at 02:56:24PM +0000, Anthony PERARD wrote:
>> > On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
>> > > On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
>> > > > On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
>> > > > > On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
>> > > > > > On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
>> > > > > > [...]
>> > > > > > > > Those Xen report something like:
>> > > > > > > > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 
>> > > > > > > > 131329 >
>> > > > > > > > 131328
>> > > > > > > > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46
>> > > > > > > > memflags=0 (62 of 64)
>> > > > > > > > 
>> > > > > > > > ?
>> > > > > > > > 
>> > > > > > > > (I tryied to reproduce the issue by simply add many emulated 
>> > > > > > > > e1000 in
>> > > > > > > > QEMU :) )
>> > > > > > > > 
>> > > > 
>> > > > > -bash-4.1# lspci -s 01:00.0 -v 
>> > > > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network 
>> > > > > Connection (rev 01)
>> > > > >         Subsystem: Intel Corporation Gigabit ET Dual Port Server 
>> > > > > Adapter
>> > > > >         Flags: fast devsel, IRQ 16
>> > > > >         Memory at fbc20000 (32-bit, non-prefetchable) [disabled] 
>> > > > > [size=128K]
>> > > > >         Memory at fb800000 (32-bit, non-prefetchable) [disabled] 
>> > > > > [size=4M]
>> > > > >         I/O ports at e020 [disabled] [size=32]
>> > > > >         Memory at fbc44000 (32-bit, non-prefetchable) [disabled] 
>> > > > > [size=16K]
>> > > > >         Expansion ROM at fb400000 [disabled] [size=4M]
>> > > > 
>> > > > BTW, I think this is the issue, the Expansion ROM. qemu-xen will
>> > > > allocate memory for it. Will have maybe have to find another way.
>> > > > qemu-trad those not seems to allocate memory, but I haven't been very
>> > > > far in trying to check that.
>> > > 
>> > > And indeed that is the case. The "Fix" below fixes it.
>> > > 
>> > > 
>> > > Based on that and this guest config:
>> > > disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
>> > > memory = 2048
>> > > boot="d"
>> > > maxvcpus=32
>> > > vcpus=1
>> > > serial='pty'
>> > > vnclisten="0.0.0.0"
>> > > name="latest"
>> > > vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
>> > > pci = ["01:00.0"]
>> > > 
>> > > I can boot the guest.
>> > 
>> > And can you access the ROM from the guest ?
>> 
>> I hadn't tried it. This is with a NIC and I just wanted to see if it
>> could do PCI passthrough without using the Option ROM.
>> > 
>> > 
>> > Also, I have another patch, it will initialize the PCI ROM BAR like any
>> > other BAR. In this case, if qemu is envolved in the access to ROM, it
>> > will print an error, like it the case for other BAR. 
>> > 
>> > I tried to test it, but it was with an embedded VGA card. When I dump
>> > the ROM, I got the same one as the emulated card instead of the ROM from
>> > the device.
>> 
>> Oddly enough for me with your patch the NIC's BIOS was invoked and
>> it tried to PXE boot:
>> 
>> (d1) [2014-01-10 03:20:29] Running option rom at ca00:0003
>> 
>> (d1) [2014-01-10 03:20:47] Booting from DVD/CD...
>> (d1) [2014-01-10 03:20:47] Booting from 0000:7c00
>> ..
>> and I did see the PXE boot menu in my guest - so even
>> better!

> Perfect, look like it is the fix for PCI passthrough.

Hi Konrad,

Are you sure it's the rom of the NIC, and not the iPXE rom from the emulated 
device that
gets run ?

With this patch and VGA devices it still points to another rom for me.
(it looks like it is pointing to the rom of the device with a BDF just one 
lower than the passed through one)

Do you by any chance know if there is a difference in how lspci and the linux 
kernel scan / list pci devices ?
(for example one by reading acpi tables .. the other one by real probing .. ?)

--
Sander


>> I have not yet done the GPU - this issue was preventing me from using
>> qemu-xen as it would always blow up before SeaBIOS was in the picture.
>> 
>> If you would like to put 'Reported-and-Tested-by: Konrad Rzeszutek Wilk
>> <konrad.wilk@xxxxxxxxxx>' please do.

> Will do.

>> Thank you!
>> > 
>> > 
>> > diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
>> > index 6dd7a68..2bbdb6d 100644
>> > --- a/hw/xen/xen_pt.c
>> > +++ b/hw/xen/xen_pt.c
>> > @@ -440,8 +440,8 @@ static int 
>> > xen_pt_register_regions(XenPCIPassthroughState *s)
>> >  
>> >          s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>> >  
>> > -        memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
>> > -                                      "xen-pci-pt-rom", d->rom.size);
>> > +        memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
>> > +                              "xen-pci-pt-rom", d->rom.size);
>> >          pci_register_bar(&s->dev, PCI_ROM_SLOT, 
>> > PCI_BASE_ADDRESS_MEM_PREFETCH,
>> >                           &s->rom);
>> >  
>> > 
>> > -- 
>> > Anthony PERARD



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel