[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Bug: Limitation of <=2GB RAM in domU persists with 4.3.0



On Tue, Jul 23, 2013 at 11:34:00PM +0100, Gordan Bobic wrote:
> I just built 4.3.0 in order to get > 2GB of RAM in domU with GPU
> passthrough without crashes. Unfortunately, the same crashes still
> happen. Massive frame buffer corruption on domU before it locks up
> solid. It seems the PCI memory stomp is still happening.
> 

If you boot Xen with guest_loglvl=all

and then run the guest the consoel (xl dmesg) should also have
the output from QEMU - that will help in seeing how it constructs
the E820 (which was the problem last time).

Are you also able to get the serial log from the guest? (IF this is
Linux?) I usually have this in my guest config:

serial='pty'

and when Linux boots up I add 'console=ttyS0,115200 loglevel=8 debug'
which will output everything to the 'xl console <guest> | tee /tmp/log'.

> I am using qemu-dm, as I did on Xen 4.2.x.
> 
> So whatever fix for this went into 4.3.0 didn't fix it for me.
> Passing less than 2GB of RAM to domU till works fine.
> 
> I have attached:
> 
> qemu-dm log for domU
> xl dmesg

> domid: 1
> Using file /dev/zvol/ssd/edi in read-write mode
> Watching /local/domain/0/device-model/1/logdirty/cmd
> Watching /local/domain/0/device-model/1/command
> Watching /local/domain/1/cpu
> char device redirected to /dev/pts/3
> qemu_map_cache_init nr_buckets = 10000 size 4194304
> shared page at pfn feffd
> buffered io page at pfn feffb
> Guest uuid = a57e6840-e9f5-4a14-a822-b2cc662c177f
> populating video RAM at ff000000
> mapping video RAM from ff000000
> Register xen platform.
> Done register platform.
> platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw 
> state.
> xs_read(/local/domain/0/device-model/1/xen_extended_power_mgmt): read error
> xs_read(): vncpasswd get error. 
> /vm/a57e6840-e9f5-4a14-a822-b2cc662c177f/vncpasswd.
> Log-dirty: no command yet.
> I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
> I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
> vcpu-set: watch node error.
> [xenstore_process_vcpu_set_event]: /local/domain/1/cpu has no CPU!
> I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
> xs_read(/local/domain/1/log-throttling): read error
> qemu: ignoring not-understood drive `/local/domain/1/log-throttling'
> medium change watch on `/local/domain/1/log-throttling' - unknown device, 
> ignored
> I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
> I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
> I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
> I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
> I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
> dm-command: hot insert pass-through pci dev 
> register_real_device: Assigning real physical device 00:1a.1 ...
> register_real_device: Enable MSI translation via per device option
> register_real_device: Disable power management
> pt_iomul_init: Error: pt_iomul_init can't open file /dev/xen/pci_iomul: No 
> such file or directory: 0x0:0x1a.0x1
> pt_register_regions: IO region registered (size=0x00000020 
> base_addr=0x00009a01)
> pci_intx: intx=2
> register_real_device: Real physical device 00:1a.1 registered successfuly!
> IRQ type = INTx
> dm-command: hot insert pass-through pci dev 
> register_real_device: Assigning real physical device 0d:00.0 ...
> register_real_device: Enable MSI translation via per device option
> register_real_device: Disable power management
> pt_iomul_init: Error: pt_iomul_init can't open file /dev/xen/pci_iomul: No 
> such file or directory: 0xd:0x0.0x0
> pt_register_regions: IO region registered (size=0x00004000 
> base_addr=0xd7efc000)
> pci_intx: intx=1
> register_real_device: Real physical device 0d:00.0 registered successfuly!
> IRQ type = INTx
> dm-command: hot insert pass-through pci dev 
> register_real_device: Assigning real physical device 08:00.0 ...
> register_real_device: Enable MSI translation via per device option
> register_real_device: Disable power management
> pt_iomul_init: Error: pt_iomul_init can't open file /dev/xen/pci_iomul: No 
> such file or directory: 0x8:0x0.0x0
> pt_register_regions: IO region registered (size=0x02000000 
> base_addr=0xf8000000)
> pt_register_regions: IO region registered (size=0x08000000 
> base_addr=0xb800000c)
> pt_register_regions: IO region registered (size=0x04000000 
> base_addr=0xb400000c)
> pt_register_regions: IO region registered (size=0x00000080 
> base_addr=0x0000df81)
> pt_register_regions: Expansion ROM registered (size=0x00080000 
> base_addr=0xfbd00000)
> pt_msi_setup: msi mapped with pirq 4f
> pci_intx: intx=1
> register_real_device: Real physical device 08:00.0 registered successfuly!
> IRQ type = MSI-INTx
> dm-command: hot insert pass-through pci dev 
> register_real_device: Assigning real physical device 08:00.1 ...
> register_real_device: Enable MSI translation via per device option
> register_real_device: Disable power management
> pt_iomul_init: Error: pt_iomul_init can't open file /dev/xen/pci_iomul: No 
> such file or directory: 0x8:0x0.0x1
> pt_register_regions: IO region registered (size=0x00004000 
> base_addr=0xfbdfc000)
> pt_msi_setup: msi mapped with pirq 4e
> pci_intx: intx=2
> register_real_device: Real physical device 08:00.1 registered successfuly!
> IRQ type = MSI-INTx
> pt_iomem_map: e_phys=e0000000 maddr=b8000000 type=8 len=134217728 index=1 
> first_map=1
> pt_iomem_map: e_phys=e8000000 maddr=b4000000 type=8 len=67108864 index=3 
> first_map=1
> pt_iomem_map: e_phys=ec000000 maddr=f8000000 type=0 len=33554432 index=0 
> first_map=1
> vga s->lfb_addr = ef000000 s->lfb_end = ef800000 
> pt_iomem_map: e_phys=ef8a0000 maddr=d7efc000 type=0 len=16384 index=0 
> first_map=1
> pt_iomem_map: e_phys=ef8a4000 maddr=fbdfc000 type=0 len=16384 index=0 
> first_map=1
> pt_ioport_map: e_phys=c100 pio_base=df80 len=128 index=5 first_map=1
> pt_ioport_map: e_phys=c1e0 pio_base=9a00 len=32 index=4 first_map=1
> platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw 
> state.
> platform_fixed_ioport: changed ro/rw state of ROM memory area. now is ro 
> state.
> Unknown PV product 2 loaded in guest
> PV driver build 1
> region type 0 at [ef880000,ef8a0000).
> squash iomem [ef880000, ef8a0000).
> region type 1 at [c180,c1c0).
> vga s->lfb_addr = ef000000 s->lfb_end = ef800000 
> pt_ioport_map: e_phys=ffff pio_base=9a00 len=32 index=4 first_map=0
> pt_pci_write_config: [00:05:0] Warning: Guest attempt to set address to 
> unused Base Address Register. [Offset:30h][Length:4]
> pt_ioport_map: e_phys=c1e0 pio_base=9a00 len=32 index=4 first_map=0
> pt_iomem_map: e_phys=ffffffff maddr=d7efc000 type=0 len=16384 index=0 
> first_map=0
> pt_pci_write_config: [00:06:0] Warning: Guest attempt to set address to 
> unused Base Address Register. [Offset:30h][Length:4]
> pt_iomem_map: e_phys=ef8a0000 maddr=d7efc000 type=0 len=16384 index=0 
> first_map=0
> pt_iomem_map: e_phys=ffffffff maddr=f8000000 type=0 len=33554432 index=0 
> first_map=0
> pt_iomem_map: e_phys=ffffffff maddr=b8000000 type=8 len=134217728 index=1 
> first_map=0
> pt_iomem_map: e_phys=ffffffff maddr=b4000000 type=8 len=67108864 index=3 
> first_map=0
> pt_ioport_map: e_phys=ffff pio_base=df80 len=128 index=5 first_map=0
> pt_iomem_map: e_phys=ec000000 maddr=f8000000 type=0 len=33554432 index=0 
> first_map=0
> pt_iomem_map: e_phys=e0000000 maddr=b8000000 type=8 len=134217728 index=1 
> first_map=0
> pt_iomem_map: e_phys=e8000000 maddr=b4000000 type=8 len=67108864 index=3 
> first_map=0
> pt_ioport_map: e_phys=c100 pio_base=df80 len=128 index=5 first_map=0
> pt_iomem_map: e_phys=ffffffff maddr=fbdfc000 type=0 len=16384 index=0 
> first_map=0
> pt_pci_write_config: [00:08:0] Warning: Guest attempt to set address to 
> unused Base Address Register. [Offset:30h][Length:4]
> pt_iomem_map: e_phys=ef8a4000 maddr=fbdfc000 type=0 len=16384 index=0 
> first_map=0
> pt_ioport_map: e_phys=ffff pio_base=9a00 len=32 index=4 first_map=0
> pt_ioport_map: e_phys=c1e0 pio_base=9a00 len=32 index=4 first_map=0
> pt_iomem_map: e_phys=ffffffff maddr=fbdfc000 type=0 len=16384 index=0 
> first_map=0
> pt_iomem_map: e_phys=ef8a4000 maddr=fbdfc000 type=0 len=16384 index=0 
> first_map=0
> pt_iomem_map: e_phys=ffffffff maddr=d7efc000 type=0 len=16384 index=0 
> first_map=0
> pt_iomem_map: e_phys=ef8a0000 maddr=d7efc000 type=0 len=16384 index=0 
> first_map=0
> pt_iomem_map: e_phys=ffffffff maddr=f8000000 type=0 len=33554432 index=0 
> first_map=0
> pt_iomem_map: e_phys=ffffffff maddr=b8000000 type=8 len=134217728 index=1 
> first_map=0
> pt_iomem_map: e_phys=ffffffff maddr=b4000000 type=8 len=67108864 index=3 
> first_map=0
> pt_ioport_map: e_phys=ffff pio_base=df80 len=128 index=5 first_map=0
> pt_iomem_map: e_phys=ec000000 maddr=f8000000 type=0 len=33554432 index=0 
> first_map=0
> pt_iomem_map: e_phys=e0000000 maddr=b8000000 type=8 len=134217728 index=1 
> first_map=0
> pt_iomem_map: e_phys=e8000000 maddr=b4000000 type=8 len=67108864 index=3 
> first_map=0
> pt_ioport_map: e_phys=c100 pio_base=df80 len=128 index=5 first_map=0

>  __  __            _  _    _____  ___     _       _  __   
>  \ \/ /___ _ __   | || |  |___ / / _ \   / |  ___| |/ /_  
>   \  // _ \ '_ \  | || |_   |_ \| | | |__| | / _ \ | '_ \ 
>   /  \  __/ | | | |__   _| ___) | |_| |__| ||  __/ | (_) |
>  /_/\_\___|_| |_|    |_|(_)____(_)___/   |_(_)___|_|\___/ 
>                                                           
> (XEN) Xen version 4.3.0 (root@xxxxxxxxxxxxxxxxxxxx) (gcc (GCC) 4.4.5 20110214 
> (Red Hat 4.4.5-6)) debug=n Tue Jul 23 14:28:40 BST 2013
> (XEN) Latest ChangeSet: 
> (XEN) Bootloader: GNU GRUB 0.97
> (XEN) Command line: noreboot dom0_vcpus_pin
> (XEN) Video information:
> (XEN)  VGA is text mode 80x25, font 8x16
> (XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
> (XEN) Disc information:
> (XEN)  Found 4 MBR signatures
> (XEN)  Found 4 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN)  0000000000000000 - 000000000009d400 (usable)
> (XEN)  000000000009d400 - 00000000000a0000 (reserved)
> (XEN)  00000000000e0000 - 0000000000100000 (reserved)
> (XEN)  0000000000100000 - 000000003f790000 (usable)
> (XEN)  000000003f790000 - 000000003f79e000 (ACPI data)
> (XEN)  000000003f79e000 - 000000003f7d0000 (ACPI NVS)
> (XEN)  000000003f7d0000 - 000000003f7e0000 (reserved)
> (XEN)  000000003f7e7000 - 0000000040000000 (reserved)
> (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> (XEN)  00000000ffc00000 - 0000000100000000 (reserved)
> (XEN)  0000000100000000 - 0000000cc0000000 (usable)
> (XEN) ACPI: RSDP 000F9F70, 0024 (r2 ACPIAM)
> (XEN) ACPI: XSDT 3F790100, 0064 (r1 042413 XSDT1438 20130424 MSFT       97)
> (XEN) ACPI: FACP 3F790290, 00F4 (r4 042413 FACP1438 20130424 MSFT       97)
> (XEN) ACPI: DSDT 3F7904F0, 58A3 (r2  1W555 1W555A58      A58 INTL 20051117)
> (XEN) ACPI: FACS 3F79E000, 0040
> (XEN) ACPI: APIC 3F790390, 0118 (r2 042413 APIC1438 20130424 MSFT       97)
> (XEN) ACPI: MCFG 3F7904B0, 003C (r1 042413 OEMMCFG  20130424 MSFT       97)
> (XEN) ACPI: OEMB 3F79E040, 0082 (r1 042413 OEMB1438 20130424 MSFT       97)
> (XEN) ACPI: SRAT 3F79A4F0, 0250 (r2 042413 OEMSRAT         1 INTL        1)
> (XEN) ACPI: HPET 3F79A740, 0038 (r1 042413 OEMHPET  20130424 MSFT       97)
> (XEN) ACPI: DMAR 3F79E0D0, 0120 (r1    AMI  OEMDMAR        1 MSFT       97)
> (XEN) ACPI: SSDT 3F7A4C70, 0363 (r1 DpgPmm    CpuPm       12 INTL 20051117)
> (XEN) System RAM: 49143MB (50322612kB)
> (XEN) Domain heap initialised DMA width 32 bits
> (XEN) Processor #0 6:12 APIC version 21
> (XEN) Processor #2 6:12 APIC version 21
> (XEN) Processor #4 6:12 APIC version 21
> (XEN) Processor #16 6:12 APIC version 21
> (XEN) Processor #18 6:12 APIC version 21
> (XEN) Processor #20 6:12 APIC version 21
> (XEN) Processor #32 6:12 APIC version 21
> (XEN) Processor #34 6:12 APIC version 21
> (XEN) Processor #36 6:12 APIC version 21
> (XEN) Processor #48 6:12 APIC version 21
> (XEN) Processor #50 6:12 APIC version 21
> (XEN) Processor #52 6:12 APIC version 21
> (XEN) Processor #1 6:12 APIC version 21
> (XEN) Processor #3 6:12 APIC version 21
> (XEN) Processor #5 6:12 APIC version 21
> (XEN) Processor #17 6:12 APIC version 21
> (XEN) Processor #19 6:12 APIC version 21
> (XEN) Processor #21 6:12 APIC version 21
> (XEN) Processor #33 6:12 APIC version 21
> (XEN) Processor #35 6:12 APIC version 21
> (XEN) Processor #37 6:12 APIC version 21
> (XEN) Processor #49 6:12 APIC version 21
> (XEN) Processor #51 6:12 APIC version 21
> (XEN) Processor #53 6:12 APIC version 21
> (XEN) IOAPIC[0]: apic_id 6, version 32, address 0xfec00000, GSI 0-23
> (XEN) IOAPIC[1]: apic_id 7, version 32, address 0xfec8a000, GSI 24-47
> (XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 3321.755 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
> (XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
> (XEN) Intel VT-d Snoop Control enabled.
> (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
> (XEN) Intel VT-d Queued Invalidation enabled.
> (XEN) Intel VT-d Interrupt Remapping not enabled.
> (XEN) Intel VT-d Shared EPT tables not enabled.
> (XEN) I/O virtualisation enabled
> (XEN)  - Dom0 mode: Relaxed
> (XEN) Interrupt remapping disabled
> (XEN) Enabled directed EOI with ioapic_ack_old on!
> (XEN) ENABLING IO-APIC IRQs
> (XEN)  -> Using old ACK method
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Allocated console ring of 64 KiB.
> (XEN) VMX: Supported advanced features:
> (XEN)  - APIC MMIO access virtualisation
> (XEN)  - APIC TPR shadow
> (XEN)  - Extended Page Tables (EPT)
> (XEN)  - Virtual-Processor Identifiers (VPID)
> (XEN)  - Virtual NMI
> (XEN)  - MSR direct-access bitmap
> (XEN)  - Unrestricted Guest
> (XEN) HVM: ASIDs enabled.
> (XEN) HVM: VMX enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) detected
> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
> (XEN) Brought up 24 CPUs
> (XEN) verify_tsc_reliability: TSC warp detected, disabling TSC_RELIABLE
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN)  Xen  kernel: 64-bit, lsb, compat32
> (XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1f70000
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN)  Dom0 alloc.:   0000000420000000->0000000430000000 (12302085 pages to 
> be allocated)
> (XEN)  Init. ramdisk: 0000000cbbdc3000->0000000cbffff400
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN)  Loaded kernel: ffffffff81000000->ffffffff81f70000
> (XEN)  Init. ramdisk: ffffffff81f70000->ffffffff861ac400
> (XEN)  Phys-Mach map: ffffffff861ad000->ffffffff8c029a10
> (XEN)  Start info:    ffffffff8c02a000->ffffffff8c02a4b4
> (XEN)  Page tables:   ffffffff8c02b000->ffffffff8c090000
> (XEN)  Boot stack:    ffffffff8c090000->ffffffff8c091000
> (XEN)  TOTAL:         ffffffff80000000->ffffffff8c400000
> (XEN)  ENTRY ADDRESS: ffffffff818091e0
> (XEN) Dom0 has maximum 24 VCPUs
> (XEN) Scrubbing Free RAM: .done.
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Std. Loglevel: Errors and warnings
> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to 
> Xen)
> (XEN) Freed 272kB init memory.
> (XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 
> 0x0000000000000002 to 0x0000000000000000.
> (XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 
> 0x0000000000000002 to 0x0000000000000000.
> (XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 
> 0x0000000000000002 to 0x0000000000000000.
> (XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 
> 0x0000000000000002 to 0x0000000000000000.
> (XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 
> 0x0000000000000002 to 0x0000000000000000.
> (XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 
> 0x0000000000000002 to 0x0000000000000000.
> (XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 
> 0x0000000000000002 to 0x0000000000000000.
> (XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 
> 0x0000000000000002 to 0x0000000000000000.
> (XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 
> 0x0000000000000002 to 0x0000000000000000.
> (XEN) traps.c:2503:d0 Domain attempted WRMSR 00000000000001fc from 
> 0x0000000000000002 to 0x0000000000000000.

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.