RE: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build

["Ian Pratt" <m+Ian.Pratt@xxxxxxxxxxxx>]

> During boot of dom0, I get the following.  Box is remote, so 
> I'm now waiting for proxy power button pusher ;-)  tg3 init 
> fails doing dma_alloc_coherent (I assume this is from 
> tg3_init_one -> tg3_test_dma -> pci_alloc_consistent -> 
> dma_alloc_coherent).  Looks like an issue with the amount of 
> memory in dom0.  Is this known?

Is this with a debug=y build of Xen? I can see how that might provoke
the situation more than other builds.

Do you know how large a contiguous chunk the tg3 needs? 

I guess we could hold onto a few large chunks before making the dom0
memory allocation.

Alternatively, if we get a failure, we should round up to the next power
of two and then balloon out that much memory. Repeat until success...

Ian



> thanks,
> -chris
> 
> root (hd0,0)
>  Filesystem type is ext2fs, partition type 0x83 kernel 
> /xen.gz com1=115200,8n1 console=com1,vga
>    [Multiboot-elf, <0x100000:0x7c3e0:0x29c20>, 
> shtab=0x1a6078, entry=0x100000] module /vmlinuz-2.6.12-xen0 
> root=/dev/VolGroup00/LogVol00 ro console=tty0,ttyS0
>    [Multiboot-module @ 0x1a7000, 0x6b43f8 bytes] module 
> /initrd-generic-2.6.img
>    [Multiboot-module @ 0x85c000, 0x2ab400 bytes]
> 
>  __  __            _____  ___         _                _ 
>  \ \/ /___ _ __   |___ / / _ \     __| | _____   _____| |
>   \  // _ \ '_ \    |_ \| | | |__ / _` |/ _ \ \ / / _ \ |
>   /  \  __/ | | |  ___) | |_| |__| (_| |  __/\ V /  __/ |
>  /_/\_\___|_| |_| |____(_)___/    \__,_|\___| \_/ \___|_|
>                                                          
>  http://www.cl.cam.ac.uk/netos/xen
>  University of Cambridge Computer Laboratory
> 
>  Xen version 3.0-devel (chrisw@xxxxxxxxxxxx) (gcc version 
> 3.4.3 20041125 (Red Hat 3.4.3-7)) Thu Jul 28 10:54:44 PDT 
> 2005  Latest ChangeSet: Thu Jul 28 05:34:45 2005 
> 501a70f3ae968e46e27b9003febf05253f4cf949
> 
> (XEN) Physical RAM map:
> (XEN)  0000000000000000 - 000000000009dc00 (usable)
> (XEN)  000000000009dc00 - 00000000000a0000 (reserved)
> (XEN)  00000000000d0000 - 0000000000100000 (reserved)
> (XEN)  0000000000100000 - 000000007ff60000 (usable)
> (XEN)  000000007ff60000 - 000000007ff72000 (ACPI data)
> (XEN)  000000007ff72000 - 000000007ff80000 (ACPI NVS)
> (XEN)  000000007ff80000 - 0000000080000000 (reserved)
> (XEN)  00000000fec00000 - 00000000fec00400 (reserved)
> (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> (XEN)  00000000fff80000 - 0000000100000000 (reserved)
> (XEN) System RAM: 2046MB (2096116kB)
> (XEN) Xen heap: 14MB (14632kB)
> (XEN) found SMP MP-table at 000f7270
> (XEN) DMI present.
> (XEN) Using APIC driver default
> (XEN) ACPI: RSDP (v002 PTLTD                                 
> ) @ 0x00000000000f7210
> (XEN) ACPI: XSDT (v001 PTLTD           XSDT   0x06040000  LTP 
> 0x00000000) @ 0x000000007ff6d5a7
> (XEN) ACPI: FADT (v003 AMD    HAMMER   0x06040000 PTEC 
> 0x000f4240) @ 0x000000007ff71d0f
> (XEN) ACPI: SRAT (v001 AMD    HAMMER   0x06040000 AMD  
> 0x00000001) @ 0x000000007ff71e03
> (XEN) ACPI: MADT (v001 PTLTD           APIC   0x06040000  LTP 
> 0x00000000) @ 0x000000007ff71ecb
> (XEN) ACPI: ASF! (v016    MBI     CETP 0x06040000 PTL  
> 0x00000001) @ 0x000000007ff71f59
> (XEN) ACPI: DSDT (v001 AMD-K8  AMDACPI 0x06040000 MSFT 
> 0x0100000e) @ 0x0000000000000000
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> (XEN) Processor #0 15:5 APIC version 16
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> (XEN) Processor #1 15:5 APIC version 16
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
> (XEN) ACPI: IOAPIC (id[0x03] address[0xc0000000] gsi_base[24])
> (XEN) IOAPIC[1]: apic_id 3, version 17, address 0xc0000000, GSI 24-27
> (XEN) ACPI: IOAPIC (id[0x04] address[0xc0001000] gsi_base[28])
> (XEN) IOAPIC[2]: apic_id 4, version 17, address 0xc0001000, GSI 28-31
> (XEN) ACPI: IOAPIC (id[0x05] address[0xc0600000] gsi_base[32])
> (XEN) IOAPIC[3]: apic_id 5, version 17, address 0xc0600000, GSI 32-35
> (XEN) ACPI: IOAPIC (id[0x06] address[0xc0601000] gsi_base[36])
> (XEN) IOAPIC[4]: apic_id 6, version 17, address 0xc0601000, GSI 36-39
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) Enabling APIC mode:  Flat.  Using 5 I/O APICs
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) Initializing CPU#0
> (XEN) Detected 1994.377 MHz processor.
> (XEN) Using scheduler: Simple EDF Scheduler (sedf)
> (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 
> bytes/line)
> (XEN) CPU: L2 Cache: 1024K (64 bytes/line)
> (XEN) CPU0: AMD Opteron(tm) Processor 246 stepping 08
> (XEN) Booting processor 1/1 eip 90000
> (XEN) Initializing CPU#1
> (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 
> bytes/line)
> (XEN) CPU: L2 Cache: 1024K (64 bytes/line)
> (XEN) CPU1: AMD Opteron(tm) Processor 246 stepping 08
> (XEN) Total of 2 processors activated.
> (XEN) ENABLING IO-APIC IRQs
> (XEN) ..TIMER: vector=0x31 pin1=2 pin2=-1
> (XEN) checking TSC synchronization across 2 CPUs: passed.
> (XEN) Platform timer is 1.193MHz PIT
> (XEN) Brought up 2 CPUs
> (XEN) mtrr: v2.0 (20020519)
> (XEN) mtrr: your CPUs had inconsistent fixed MTRR settings
> (XEN) mtrr: probably your BIOS does not setup all CPUs.
> (XEN) mtrr: corrected configuration.
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Xen-ELF header found: 
> 'GUEST_OS=linux,GUEST_VER=2.6,XEN_VER=3.0,VIRT_BASE=0xffffffff
80000000,LOADER=generic'
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN)  Dom0 alloc.:   0000000020000000->0000000040000000 
> (382799 pages to be allocated)
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN)  Loaded kernel: ffffffff80100000->ffffffff805ab086
> (XEN)  Init. ramdisk: ffffffff805ac000->ffffffff80857400
> (XEN)  Phys-Mach map: ffffffff80858000->ffffffff80a4dd3c
> (XEN)  Page tables:   ffffffff80a4e000->ffffffff80a57000
> (XEN)  Start info:    ffffffff80a57000->ffffffff80a58000
> (XEN)  Boot stack:    ffffffff80a58000->ffffffff80a59000
> (XEN)  TOTAL:         ffffffff80000000->ffffffff80c00000
> (XEN)  ENTRY ADDRESS: ffffffff80100000
> (XEN) Initrd len 0x2ab400, start at 0xffffffff805ac000
> (XEN) Scrubbing Free RAM: .....................done.
> (XEN) Xen trace buffers: initialised
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to 
> switch input to Xen).
> Linux version 2.6.12.2-xen0 (chrisw@xxxxxxxxxxxxxxxxxx) (gcc 
> version 3.4.3 20041125 (Red Hat 3.4.3-7)) #6 Thu Jul 28 
> 10:53:46 PDT 2005 kernel direct mapping tables upto 
> ffff88007d74f000 @ c00000-fef000 Registering memory for 
> bootmem: from  1000000, size = 7c74f000
> ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
> ACPI: IOAPIC (id[0x03] address[0xc0000000] gsi_base[24])
> IOAPIC[1]: apic_id 3, version 17, address 0xc0000000, GSI 24-27
> ACPI: IOAPIC (id[0x04] address[0xc0001000] gsi_base[28])
> IOAPIC[2]: apic_id 4, version 17, address 0xc0001000, GSI 28-31
> ACPI: IOAPIC (id[0x05] address[0xc0600000] gsi_base[32])
> IOAPIC[3]: apic_id 5, version 17, address 0xc0600000, GSI 32-35
> ACPI: IOAPIC (id[0x06] address[0xc0601000] gsi_base[36])
> IOAPIC[4]: apic_id 6, version 17, address 0xc0601000, GSI 36-39
> ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge) 
> Setting APIC routing to flat Using ACPI (MADT) for SMP 
> configuration information Allocating PCI resources starting 
> at 7d800000 (gap: 7d74f000:828b1000) Built 1 zonelists Kernel 
> command line: root=/dev/VolGroup00/LogVol00 ro 
> console=tty0,ttyS0 Initializing CPU#0 PID hash table entries: 
> 4096 (order: 12, 131072 bytes) Xen reported: 1994.377 MHz processor.
> Console: colour VGA+ 80x25
> Dentry cache hash table entries: 262144 (order: 9, 2097152 
> bytes) Inode-cache hash table entries: 131072 (order: 8, 
> 1048576 bytes)
> Memory: 2007552k/2055484k available (2820k kernel code, 
> 31364k reserved, 1119k data, 272k init) Mount-cache hash 
> table entries: 256
> CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
> CPU: L2 Cache: 1024K (64 bytes/line)
> CPU: AMD Opteron(tm) Processor 246 stepping 08 checking if 
> image is initramfs... it is
> NET: Registered protocol family 16
> PCI: Using configuration type 1
> ACPI: Subsystem revision 20050309
> ACPI: Interpreter enabled
> ACPI: Using IOAPIC for interrupt routing
> ACPI: PCI Root Bridge [PCI0] (0000:00)
> PCI: Probing PCI hardware (bus 00)
> ACPI: PCI Interrupt Link [LNKA] (IRQs 3 *5 10 11)
> ACPI: PCI Interrupt Link [LNKB] (IRQs 3 5 10 *11)
> ACPI: PCI Interrupt Link [LNKC] (IRQs 3 5 *10 11)
> ACPI: PCI Interrupt Link [LNKD] (IRQs 3 5 10 *11)
> ACPI: PCI Root Bridge [PCI1] (0000:08)
> PCI: Probing PCI hardware (bus 08)
> xen_mem: Initialising balloon driver.
> SCSI subsystem initialized
> PCI: Using ACPI for IRQ routing
> PCI: If a device doesn't work, try "pci=routeirq".  If it 
> helps, post a report Grant table initialized
> IA-32 Microcode Update Driver: v1.14 <tigran@xxxxxxxxxxx>
> IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak 
> Exp $ Initializing Cryptographic API
> PCI: MSI quirk detected. pci_msi_quirk set.
> PCI: MSI quirk detected. pci_msi_quirk set.
> PCI: MSI quirk detected. pci_msi_quirk set.
> PCI: MSI quirk detected. pci_msi_quirk set.
> Real Time Clock Driver v1.12
> serio: i8042 AUX port at 0x60,0x64 irq 12
> serio: i8042 KBD port at 0x60,0x64 irq 1 io scheduler noop 
> registered io scheduler anticipatory registered io scheduler 
> deadline registered io scheduler cfq registered
> floppy0: no floppy controllers found
> RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
> loop: loaded (max 8 devices)
> HP CISS Driver (v 2.6.6)
> Intel(R) PRO/1000 Network Driver - version 6.0.54-k2 
> Copyright (c) 1999-2004 Intel Corporation.
> pcnet32.c:v1.30j 29.04.2005 tsbogend@xxxxxxxxxxxxxxxx
> e100: Intel(R) PRO/100 Network Driver, 3.4.8-k2-NAPI
> e100: Copyright(c) 1999-2005 Intel Corporation
> tg3.c:v3.31 (June 8, 2005)
> ACPI: PCI Interrupt 0000:03:02.0[A] -> GSI 28 (level, low) -> IRQ 28
> (XEN) (file=dom_mem_ops.c, line=61) Could not allocate a frame
> ----------- [cut here ] --------- [please bite here ] 
> --------- Kernel BUG at "arch/xen/x86_64/mm/hypervisor.c":299
> invalid operand: 0000 [1]
> CPU 0
> Modules linked in:
> Pid: 1, comm: swapper Not tainted 2.6.12.2-xen0
> RIP: e030:[<ffffffff8011a32a>] 
> <ffffffff8011a32a>{xen_contig_memory+586}
> RSP: e02b:ffff880002ee1bf8  EFLAGS: 00010296
> RAX: 0000000000000000 RBX: 000000000000000c RCX: ffffffff8011a324
> RDX: 0000000000000001 RSI: ffff880002ee1c28 RDI: 00000000ffffffff
> RBP: 0000000000000001 R08: 0000000000007ff0 R09: 0000000000000001
> R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000
> R13: 0000000000000002 R14: 0000000000000001 R15: ffff800000000000
> FS:  0000000000000000(0000) GS:ffffffff80533a80(0000) 
> knlGS:0000000000000000
> CS:  e033 DS: 0000 ES: 0000
> Process swapper (pid: 1, threadinfo ffff880002ee0000, task 
> ffff880002edf410)
> Stack: 000000001002029a ffff880078cc6000 0000000202edf410 
> 0000000052aa4d54 
>        0000000000000000 ffff880078cc4000 0000000000044a89 
> 0000000000000001 
>        ffff880078cc4000 0000000000000000 Call 
> Trace:<ffffffff80114920>{dma_alloc_coherent+224} 
> <ffffffff802a8096>{tg3_init_one+7302}
>        <ffffffff803bdafa>{wait_for_completion+170} 
> <ffffffff80126dd0>{default_wake_function+0}
>        <ffffffff802212d6>{pci_device_probe+134} 
> <ffffffff8025ac0d>{driver_probe_device+77}
>        <ffffffff8025ad2b>{driver_attach+75} 
> <ffffffff8025b118>{bus_add_driver+152}
>        <ffffffff80220fe5>{pci_register_driver+117} 
> <ffffffff801090f2>{init+178}
>        <ffffffff8010e567>{child_rip+8} <ffffffff80109040>{init+0}
>        <ffffffff8010e55f>{child_rip+0} 
> 
> Code: 0f 0b 08 91 3d 80 ff ff ff ff 2b 01 b8 01 00 00 00 44 
> 89 f1 RIP <ffffffff8011a32a>{xen_contig_memory+586} RSP 
> <ffff880002ee1bf8>  <0>Kernel panic - not syncing: Attempted 
> to kill init!
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel