Re: [Xen-devel] qemu-xen-dir + PCI passthrough = BOOM

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

Thursday, January 9, 2014, 3:56:24 PM, you wrote:

> On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
>> On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
>> > On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
>> > > On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
>> > > > On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
>> > > > [...]
>> > > > > > Those Xen report something like:
>> > > > > > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329 >
>> > > > > > 131328
>> > > > > > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46
>> > > > > > memflags=0 (62 of 64)
>> > > > > > 
>> > > > > > ?
>> > > > > > 
>> > > > > > (I tryied to reproduce the issue by simply add many emulated e1000 
>> > > > > > in
>> > > > > > QEMU :) )
>> > > > > > 
>> > 
>> > > -bash-4.1# lspci -s 01:00.0 -v 
>> > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network 
>> > > Connection (rev 01)
>> > >         Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
>> > >         Flags: fast devsel, IRQ 16
>> > >         Memory at fbc20000 (32-bit, non-prefetchable) [disabled] 
>> > > [size=128K]
>> > >         Memory at fb800000 (32-bit, non-prefetchable) [disabled] 
>> > > [size=4M]
>> > >         I/O ports at e020 [disabled] [size=32]
>> > >         Memory at fbc44000 (32-bit, non-prefetchable) [disabled] 
>> > > [size=16K]
>> > >         Expansion ROM at fb400000 [disabled] [size=4M]
>> > 
>> > BTW, I think this is the issue, the Expansion ROM. qemu-xen will
>> > allocate memory for it. Will have maybe have to find another way.
>> > qemu-trad those not seems to allocate memory, but I haven't been very
>> > far in trying to check that.
>> 
>> And indeed that is the case. The "Fix" below fixes it.
>> 
>> 
>> Based on that and this guest config:
>> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
>> memory = 2048
>> boot="d"
>> maxvcpus=32
>> vcpus=1
>> serial='pty'
>> vnclisten="0.0.0.0"
>> name="latest"
>> vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
>> pci = ["01:00.0"]
>> 
>> I can boot the guest.

> And can you access the ROM from the guest ?


> Also, I have another patch, it will initialize the PCI ROM BAR like any
> other BAR. In this case, if qemu is envolved in the access to ROM, it
> will print an error, like it the case for other BAR. 

> I tried to test it, but it was with an embedded VGA card. When I dump
> the ROM, I got the same one as the emulated card instead of the ROM from
> the device.

Ah this is what i reported earlier ..

If you would like it more funky .. use Dario's patches to be able to use 
vga="none" and use:
vga="none"
nographic=1
xen_platform_pci=1

And when you dump the rombar of the passedthrough vga card, you will end up 
with the iPXE rom of the emulated NIC.

So it is pointing at the first / last / a random rom ... but at least it 
doesn't seem directly tied to pointing at
another VGA rom (which was my first assumption some time ago).

When i go one step further .. by also disabling the xen platform pci device .. 
it doesn't boot any more.


Strange thing is that all the addresses in debug messages (host kernel .. 
hvmloader seabios qemu and guest kernel) for the rom bar seem to correspond 
when the translation is taken into account ..
so nothing obvious there ...

--
Sander

> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
> index 6dd7a68..2bbdb6d 100644
> --- a/hw/xen/xen_pt.c
> +++ b/hw/xen/xen_pt.c
> @@ -440,8 +440,8 @@ static int xen_pt_register_regions(XenPCIPassthroughState 
> *s)
>  
>          s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>  
> -        memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
> -                                      "xen-pci-pt-rom", d->rom.size);
> +        memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
> +                              "xen-pci-pt-rom", d->rom.size);
>          pci_register_bar(&s->dev, PCI_ROM_SLOT, 
> PCI_BASE_ADDRESS_MEM_PREFETCH,
>                           &s->rom);
>  



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel