Re: [Xen-devel] qemu-xen-dir + PCI passthrough = BOOM

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

Friday, January 10, 2014, 5:35:44 PM, you wrote:

>> >> (d1) [2014-01-10 03:20:29] Running option rom at ca00:0003
>> >> 
>> >> (d1) [2014-01-10 03:20:47] Booting from DVD/CD...
>> >> (d1) [2014-01-10 03:20:47] Booting from 0000:7c00
>> >> ..
>> >> and I did see the PXE boot menu in my guest - so even
>> >> better!
>> 
>> > Perfect, look like it is the fix for PCI passthrough.
>> 
>> Hi Konrad,
>> 
>> Are you sure it's the rom of the NIC, and not the iPXE rom from the emulated 
>> device that
>> gets run ?

> Yes. I double checked that the MAC address that was given an DHCP
> address was indeed for the physical hardware. And it was.

OK

>> 
>> With this patch and VGA devices it still points to another rom for me.
>> (it looks like it is pointing to the rom of the device with a BDF just one 
>> lower than the passed through one)

> That all sounds to me like a bug in QEMU which constructs
> the 'world'. Then 'hvmloader' and the kernel ingest this to
> create their view of what the PCI configuration/slots/etc should look like.
> They use 'inb' and 'outb' instructions on the 0xcf8/0xcfc port.

> Perhaps the 'vga=none' is having an hard time dealing with 
> 'no-VGA-but-wait-there
> is-and-PT-VGA'?

> Just to make sure I am not forgetting a crucial fact - if you don't
> have vga=none, does the lspci output look sane?

It does .. except when using an PV NIC, that one doesn't show in lspci, but it 
does occupy a slot (the numbers are not
consecutive anymore, one is "hidden"), but the PV nic itself is working ok.
This doesn't seem to happen for disk so that seems a bit strange to me ..

Apart from that it looks sane.

But I'm rebuilding everything now from scratch and will try again .. just too 
damned many parameters :-)

Will see if i can make a complete post again with all data, although the 
previous time i tried that,
it was a little bit to intimidating i guess.

>> 
>> Do you by any chance know if there is a difference in how lspci and the 
>> linux kernel scan / list pci devices ?
>> (for example one by reading acpi tables .. the other one by real probing .. 
>> ?)

> The kernel and hvmloader all use the 'inb' and 'outb' to figure out
> what the PCI space looks like.

> 'lspci' uses /sysfs. Thought if you use '-xxxx' I think it also does
> 'inb' and 'outb'.

Hmm ok .. so i would expect that dumping with the:
echo 1 > rom; cat rom > rom.bin; echo 0 > rom;
sequence in /sys/bus/pci/devices/<BDF> would do everything according to the 
addresses that lspci gives for the rombar ..
and when i do that the resulting rom.bin differs according to the emulated 
devices i put it.

The devices that have a rom in the guest are always in the order (with there 
BDF):
NIC
Emulated VGA
Passthroughed VGA

When all enabled, i end up with the rom of the emulated VGA
When i disable that with vga="none", i end up with the rom of the NIC
When i also disable that .. by not specifying any vif and using 
xen_platform_pci=0 ..
     the kernel complains when trying to dump that it's not a valid rom (and 
when forced too it's all zero's so it's
     right because it can't find the start signature of a rom)

Is there an easy tool to dump the content of arbitrary mem addresses ?
(except enabling the /dev/kmem and trying to do some voodoo calculations and 
use "dd" ?)

Hmm when i make a core dump using "xl dump-core" would/should that also contain 
the content of the passed throughed rom ?
Then at least i could search that dump for the strings of the real vga rom of 
the passedtroughed card  and see if it is there .. perhaps on some other
address as expected .. or at least know it isn't.

Because at the moment i'm not very well in progressing with ruling things out 
(which seems to be the only method),
i only have some symptoms that differ. Wish i had another NIC device with a rom 
to see if that does work.


> So it all seems to point to QEMU.
>> 
>> --
>> Sander
>> 
>> 
>> >> I have not yet done the GPU - this issue was preventing me from using
>> >> qemu-xen as it would always blow up before SeaBIOS was in the picture.
>> >> 
>> >> If you would like to put 'Reported-and-Tested-by: Konrad Rzeszutek Wilk
>> >> <konrad.wilk@xxxxxxxxxx>' please do.
>> 
>> > Will do.
>> 
>> >> Thank you!
>> >> > 
>> >> > 
>> >> > diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
>> >> > index 6dd7a68..2bbdb6d 100644
>> >> > --- a/hw/xen/xen_pt.c
>> >> > +++ b/hw/xen/xen_pt.c
>> >> > @@ -440,8 +440,8 @@ static int 
>> >> > xen_pt_register_regions(XenPCIPassthroughState *s)
>> >> >  
>> >> >          s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>> >> >  
>> >> > -        memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
>> >> > -                                      "xen-pci-pt-rom", d->rom.size);
>> >> > +        memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
>> >> > +                              "xen-pci-pt-rom", d->rom.size);
>> >> >          pci_register_bar(&s->dev, PCI_ROM_SLOT, 
>> >> > PCI_BASE_ADDRESS_MEM_PREFETCH,
>> >> >                           &s->rom);
>> >> >  
>> >> > 
>> >> > -- 
>> >> > Anthony PERARD
>> 
>> 



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel