|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] qemu-xen-dir + PCI passthrough = BOOM
Friday, January 10, 2014, 1:17:08 AM, you wrote:
> Friday, January 10, 2014, 12:54:43 AM, you wrote:
>> Thursday, January 9, 2014, 3:56:24 PM, you wrote:
>>> On Wed, Jan 08, 2014 at 02:44:51PM -0500, Konrad Rzeszutek Wilk wrote:
>>>> On Wed, Dec 18, 2013 at 02:48:24PM +0000, Anthony PERARD wrote:
>>>> > On Mon, Dec 16, 2013 at 10:08:16AM -0500, Konrad Rzeszutek Wilk wrote:
>>>> > > On Fri, Dec 06, 2013 at 04:03:10PM +0000, Wei Liu wrote:
>>>> > > > On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote:
>>>> > > > [...]
>>>> > > > > > Those Xen report something like:
>>>> > > > > > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329
>>>> > > > > > >
>>>> > > > > > 131328
>>>> > > > > > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46
>>>> > > > > > memflags=0 (62 of 64)
>>>> > > > > >
>>>> > > > > > ?
>>>> > > > > >
>>>> > > > > > (I tryied to reproduce the issue by simply add many emulated
>>>> > > > > > e1000 in
>>>> > > > > > QEMU :) )
>>>> > > > > >
>>>> >
>>>> > > -bash-4.1# lspci -s 01:00.0 -v
>>>> > > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
>>>> > > Connection (rev 01)
>>>> > > Subsystem: Intel Corporation Gigabit ET Dual Port Server
>>>> > > Adapter
>>>> > > Flags: fast devsel, IRQ 16
>>>> > > Memory at fbc20000 (32-bit, non-prefetchable) [disabled]
>>>> > > [size=128K]
>>>> > > Memory at fb800000 (32-bit, non-prefetchable) [disabled]
>>>> > > [size=4M]
>>>> > > I/O ports at e020 [disabled] [size=32]
>>>> > > Memory at fbc44000 (32-bit, non-prefetchable) [disabled]
>>>> > > [size=16K]
>>>> > > Expansion ROM at fb400000 [disabled] [size=4M]
>>>> >
>>>> > BTW, I think this is the issue, the Expansion ROM. qemu-xen will
>>>> > allocate memory for it. Will have maybe have to find another way.
>>>> > qemu-trad those not seems to allocate memory, but I haven't been very
>>>> > far in trying to check that.
>>>>
>>>> And indeed that is the case. The "Fix" below fixes it.
>>>>
>>>>
>>>> Based on that and this guest config:
>>>> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
>>>> memory = 2048
>>>> boot="d"
>>>> maxvcpus=32
>>>> vcpus=1
>>>> serial='pty'
>>>> vnclisten="0.0.0.0"
>>>> name="latest"
>>>> vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
>>>> pci = ["01:00.0"]
>>>>
>>>> I can boot the guest.
>>> And can you access the ROM from the guest ?
>>> Also, I have another patch, it will initialize the PCI ROM BAR like any
>>> other BAR. In this case, if qemu is envolved in the access to ROM, it
>>> will print an error, like it the case for other BAR.
>>> I tried to test it, but it was with an embedded VGA card. When I dump
>>> the ROM, I got the same one as the emulated card instead of the ROM from
>>> the device.
>> Ah this is what i reported earlier ..
>> If you would like it more funky .. use Dario's patches to be able to use
>> vga="none" and use:
>> vga="none"
>> nographic=1
>> xen_platform_pci=1
>> And when you dump the rombar of the passedthrough vga card, you will end up
>> with the iPXE rom of the emulated NIC.
>> So it is pointing at the first / last / a random rom ... but at least it
>> doesn't seem directly tied to pointing at
>> another VGA rom (which was my first assumption some time ago).
>> When i go one step further .. by also disabling the xen platform pci device
>> .. it doesn't boot any more.
>> Strange thing is that all the addresses in debug messages (host kernel ..
>> hvmloader seabios qemu and guest kernel) for the rom bar seem to correspond
>> when the translation is taken into account ..
>> so nothing obvious there ...
> Perhaps i have to correct myself here ... with:
> vga="none"
> nographic=1
> xen_platform_pci=1
> There seems to be a discrepancy between what the guest kernel reports at boot
> and the guest lspci output:
> [ 0.000000] e820: [mem 0x40000000-0xfbffffff] available for PCI devices
> <snip>
> [ 1.453530] PCI host bridge to bus 0000:00
> [ 1.460018] pci_bus 0000:00: root bus resource [bus 00-ff]
> [ 1.466690] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7]
> [ 1.473355] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff]
> [ 1.480024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
> [ 1.490023] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xfbffffff]
> [ 1.500013] pci_bus 0000:00: scanning bus
> [ 1.500504] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
> [ 1.500600] pci 0000:00:00.0: calling quirk_mmio_always_on+0x0/0x10
> [ 1.508515] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
> [ 1.515216] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
> [ 1.531714] pci 0000:00:01.1: reg 0x20: [io 0xc240-0xc24f]
> [ 1.540859] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
> [ 1.540859] pci 0000:00:01.3: calling acpi_pm_check_blacklist+0x0/0x40
> [ 1.545776] pci 0000:00:01.3: calling quirk_piix4_acpi+0x0/0x140
> [ 1.545883] pci 0000:00:01.3: quirk: [io 0xb000-0xb03f] claimed by PIIX4
> ACPI
> [ 1.546916] pci 0000:00:01.3: quirk: [io 0xb100-0xb10f] claimed by PIIX4
> SMB
> [ 1.551173] pci 0000:00:01.3: calling pci_fixup_piix4_acpi+0x0/0x10
> [ 1.554237] pci 0000:00:02.0: [5853:0001] type 00 class 0xff8000
> [ 1.556666] pci 0000:00:02.0: reg 0x10: [io 0xc000-0xc0ff]
> [ 1.566373] pci 0000:00:02.0: reg 0x14: [mem 0xf0000000-0xf0ffffff pref]
> [ 1.596780] pci 0000:00:04.0: [1002:6759] type 00 class 0x030000
> [ 1.656719] pci 0000:00:04.0: reg 0x10: [mem 0xe0000000-0xefffffff 64bit
> pref]
> [ 1.693355] pci 0000:00:04.0: reg 0x18: [mem 0xf1060000-0xf107ffff 64bit]
> [ 1.743361] pci 0000:00:04.0: reg 0x20: [io 0xc100-0xc1ff]
> [ 1.826692] pci 0000:00:04.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
> [ 1.828722] pci 0000:00:04.0: supports D1 D2
> 00:04.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Turks
> [Radeon HD 6570] (prog-if 00 [VGA controller])
> Subsystem: PC Partner Limited Device e193
> Physical Slot: 4
> Flags: fast devsel, IRQ 32
> Memory at e0000000 (64-bit, prefetchable) [size=256M]
> Memory at f1060000 (64-bit, non-prefetchable) [size=128K]
> I/O ports at c100 [size=256]
> [virtual] Expansion ROM at f1000000 [disabled] [size=128K]
> Capabilities: [50] Power Management version 3
> Capabilities: [58] Express Legacy Endpoint, MSI 00
> Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
> Capabilities: [100] #1002
> At least these 2 don't seem to match up:
> [ 1.826692] pci 0000:00:04.0: reg 0x30: [mem 0x00000000-0x0001ffff pref]
> [virtual] Expansion ROM at f1000000 [disabled] [size=128K]
And from xl dmesg:
(d23) [2014-01-10 00:07:47] pci dev 03:0 bar 30 size 000040000: 0f1000000
which is .. probably the NIC ... but which is mysteriously never shown in lspci
(thought i had reported that earlier as well):
lspci -v
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
Subsystem: Red Hat, Inc Qemu virtual machine
Flags: bus master, fast devsel, latency 0
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
Subsystem: Red Hat, Inc Qemu virtual machine
Physical Slot: 1
Flags: bus master, medium devsel, latency 0
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
(prog-if 80 [Master])
Subsystem: Red Hat, Inc Qemu virtual machine
Physical Slot: 1
Flags: bus master, medium devsel, latency 0
[virtual] Memory at 000001f0 (32-bit, non-prefetchable) [size=8]
[virtual] Memory at 000003f0 (type 3, non-prefetchable) [size=1]
[virtual] Memory at 00000170 (32-bit, non-prefetchable) [size=8]
[virtual] Memory at 00000370 (type 3, non-prefetchable) [size=1]
I/O ports at c240 [size=16]
Kernel driver in use: PIIX_IDE
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
Subsystem: Red Hat, Inc Qemu virtual machine
Physical Slot: 1
Flags: bus master, medium devsel, latency 0, IRQ 9
00:02.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)
Subsystem: XenSource, Inc. Xen Platform Device
Physical Slot: 2
Flags: bus master, fast devsel, latency 0, IRQ 24
I/O ports at c000 [size=256]
Memory at f0000000 (32-bit, prefetchable) [size=16M]
Kernel driver in use: xen-platform-pci
00:04.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Turks
[Radeon HD 6570] (prog-if 00 [VGA controller])
Subsystem: PC Partner Limited Device e193
Physical Slot: 4
Flags: fast devsel, IRQ 32
Memory at e0000000 (64-bit, prefetchable) [size=256M]
Memory at f1060000 (64-bit, non-prefetchable) [size=128K]
I/O ports at c100 [size=256]
[virtual] Expansion ROM at f1000000 [disabled] [size=128K]
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [100] #1002
which is defined in the guest config as:
vif = [ 'bridge=xen_bridge,ip=192.168.1.44,mac=00:16:3A:C6:76:65, model=e1000'
]
>> --
>> Sander
>>> diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
>>> index 6dd7a68..2bbdb6d 100644
>>> --- a/hw/xen/xen_pt.c
>>> +++ b/hw/xen/xen_pt.c
>>> @@ -440,8 +440,8 @@ static int
>>> xen_pt_register_regions(XenPCIPassthroughState *s)
>>>
>>> s->bases[PCI_ROM_SLOT].access.maddr = d->rom.base_addr;
>>>
>>> - memory_region_init_rom_device(&s->rom, OBJECT(s), NULL, NULL,
>>> - "xen-pci-pt-rom", d->rom.size);
>>> + memory_region_init_io(&s->rom, OBJECT(s), &ops, &s->dev,
>>> + "xen-pci-pt-rom", d->rom.size);
>>> pci_register_bar(&s->dev, PCI_ROM_SLOT,
>>> PCI_BASE_ADDRESS_MEM_PREFETCH,
>>> &s->rom);
>>>
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |