[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] Xen hypervisor failed to startup when booting CPUs


  • To: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
  • From: "Lv, Hui" <hui.lv@xxxxxxxxx>
  • Date: Fri, 11 Feb 2011 15:04:46 +0800
  • Accept-language: en-US
  • Acceptlanguage: en-US
  • Delivery-date: Thu, 10 Feb 2011 23:08:13 -0800
  • List-id: Xen developer discussion <xen-devel.lists.xensource.com>
  • Thread-index: AcvJufdyxQdWkJRUSjC5BYrmP3Np8w==
  • Thread-topic: Xen hypervisor failed to startup when booting CPUs

Hi Folks:

 

         I run into a problem when enabling Xen in next generation server platforms with Xen c/s is 21380.

         Xen reported “CPU Not responding” when booting up 32 CPUs( 2 sockets with 8 cores/16threads total). The log files belonging showed something wrong with APCI. So I added x2apic=0 in the xen grub line, but the symptom remained.

         However, Native RHEL5.5 can normally started in this platforms.

 

         Does anyone meet this problem and know how to solve it or give some hints?

         Thanks very much!

 

         Btw, I also tried the latest xen (version 4.1 rc5). Hypervisor still hang when booting CPUs with “error code -5”?. (if someone need the log files, I’ll post it)

                  __  __            _____  ____                     _        _     _     

 \ \/ /___ _ __   |___ / | ___|    _   _ _ __  ___| |_ __ _| |__ | | ___

  \  // _ \ '_ \    |_ \ |___ \ __| | | | '_ \/ __| __/ _` | '_ \| |/ _ \

  /  \  __/ | | |  ___) | ___) |__| |_| | | | \__ \ || (_| | |_) | |  __/

 /_/\_\___|_| |_| |____(_)____/    \__,_|_| |_|___/\__\__,_|_.__/|_|\___|

                                                                        

(XEN) Xen version 3.5-unstable (root@) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) Thu Feb 10 14:40:15 EST 2011

(XEN) Latest ChangeSet: Wed Dec 09 10:59:31 2009 +0000 20605:8f304c003af4

(XEN) Command line: com1=115200,8n1 console=com1 loglvl=all

(XEN) Video information:

(XEN)  VGA is text mode 80x25, font 8x16

(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds

(XEN) Disc information:

(XEN)  Found 0 MBR signatures

(XEN)  Found 1 EDD information structures

(XEN) Xen-e820 RAM map:

(XEN)  0000000000000000 - 000000000009bc00 (usable)

(XEN)  000000000009bc00 - 00000000000a0000 (reserved)

(XEN)  00000000000e0000 - 0000000000100000 (reserved)

(XEN)  0000000000100000 - 00000000ab70f000 (usable)

(XEN)  00000000ab70f000 - 00000000ab88d000 (ACPI NVS)

(XEN)  00000000ab88d000 - 00000000bed0d000 (usable)

(XEN)  00000000bed0d000 - 00000000bede7000 (reserved)

(XEN)  00000000bede7000 - 00000000bee70000 (ACPI NVS)

(XEN)  00000000bee70000 - 00000000beeae000 (ACPI data)

(XEN)  00000000beeae000 - 00000000beeaf000 (ACPI NVS)

(XEN)  00000000beeaf000 - 00000000beeca000 (ACPI data)

(XEN)  00000000beeca000 - 00000000befd2000 (ACPI NVS)

(XEN)  00000000befd2000 - 00000000bf000000 (usable)

(XEN)  00000000bf000000 - 00000000d0000000 (reserved)

(XEN)  00000000fec00000 - 00000000fec01000 (reserved)

(XEN)  00000000fed19000 - 00000000fed1a000 (reserved)

(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)

(XEN)  00000000fee00000 - 00000000fee01000 (reserved)

(XEN)  00000000ffc00000 - 0000000100000000 (reserved)

(XEN)  0000000100000000 - 0000000840000000 (usable)

(XEN) ACPI: RSDP 000F0410, 0024 (r2  INTEL)

(XEN) ACPI: XSDT BEEC8E18, 008C (r1  INTEL   ROMLEY  6222004 INTL 20090903)

(XEN) ACPI: FACP BEEC7D98, 00F4 (r4  INTEL   ROMLEY  6222004 INTL 20090903)

(XEN) ACPI: DSDT BEEAF018, 1660E (r2  INTEL   ROMLEY       27 INTL 20100331)

(XEN) ACPI: FACS BEEC8D40, 0040

(XEN) ACPI: APIC BEEC6718, 066A (r3  INTEL   ROMLEY  6222004 INTL 20090903)

(XEN) ACPI: SPMI BEEC9F18, 0040 (r5  INTEL   ROMLEY  6222004 INTL 20090903)

(XEN) ACPI: TCPA BEEC9E98, 0032 (r2                        0             0)

(XEN) ACPI: MCFG BEEC9E18, 003C (r1 INTEL  ROMLEY    6222004 INTL 20090903)

(XEN) ACPI: SRAT BEEADC18, 02A8 (r3  INTEL   ROMLEY  6222004 INTL 20090903)

(XEN) ACPI: SLIT BEEC9D18, 0030 (r1  INTEL   ROMLEY  6222004 INTL 20090903)

(XEN) ACPI: MSCT BEEC9C98, 004E (r1  INTEL   ROMLEY  6222004 INTL 20090903)

(XEN) ACPI: HPET BEEC9D98, 0038 (r1 INTL   ROMLEY    6222004 INTL 20090903)

(XEN) ACPI: SSDT BEEC9C18, 002B (r2  INTEL PtidDevc     1000 INTL 20100331)

(XEN) ACPI: BOOT BEEC9B98, 0028 (r1 INTEL  ROMLEY    6222004 INTL 20090903)

(XEN) ACPI: SSDT BEE70018, 3CFC4 (r2  INTEL    CpuPm     4000 INTL 20100331)

(XEN) ACPI: DMAR BEEC7798, 0130 (r1 INTEL      RML         1 INTL        1)

(XEN) System RAM: 32731MB (33517028kB)

(XEN) SRAT: PXM 0 -> APIC 0 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 1 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 2 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 3 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 4 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 5 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 6 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 7 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 8 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 9 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 10 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 11 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 12 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 13 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 14 -> Node 0

(XEN) SRAT: PXM 0 -> APIC 15 -> Node 0

(XEN) SRAT: PXM 1 -> APIC 32 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 33 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 34 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 35 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 36 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 37 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 38 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 39 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 40 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 41 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 42 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 43 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 44 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 45 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 46 -> Node 1

(XEN) SRAT: PXM 1 -> APIC 47 -> Node 1

(XEN) SRAT: Node 0 PXM 0 0-c0000000

(XEN) SRAT: Node 0 PXM 0 100000000-440000000

(XEN) SRAT: Node 1 PXM 1 440000000-840000000

(XEN) NUMA: Using 18 for the hash shift.

(XEN) Domain heap initialised DMA width 32 bits

(XEN) found SMP MP-table at 000fcdd0

(XEN) DMI 2.6 present.

(XEN) Using APIC driver default

(XEN) ACPI: PM-Timer IO Port: 0x408

(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0]

(XEN) ACPI:                  wakeup_vec[beec8d4c], vec_size[20]

(XEN) ACPI: Local APIC address 0xfee00000

(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)

(XEN) Processor #0 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)

(XEN) Processor #2 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)

(XEN) Processor #4 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)

(XEN) Processor #6 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] enabled)

(XEN) Processor #8 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x0a] enabled)

(XEN) Processor #10 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x0c] enabled)

(XEN) Processor #12 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x0e] enabled)

(XEN) Processor #14 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x20] enabled)

(XEN) Processor #32 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x09] lapic_id[0x22] enabled)

(XEN) Processor #34 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x24] enabled)

(XEN) Processor #36 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x26] enabled)

(XEN) Processor #38 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x28] enabled)

(XEN) Processor #40 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x2a] enabled)

(XEN) Processor #42 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x2c] enabled)

(XEN) Processor #44 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x2e] enabled)

(XEN) Processor #46 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x10] lapic_id[0x01] enabled)

(XEN) Processor #1 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x11] lapic_id[0x03] enabled)

(XEN) Processor #3 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x12] lapic_id[0x05] enabled)

(XEN) Processor #5 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x13] lapic_id[0x07] enabled)

(XEN) Processor #7 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x14] lapic_id[0x09] enabled)

(XEN) Processor #9 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x15] lapic_id[0x0b] enabled)

(XEN) Processor #11 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x16] lapic_id[0x0d] enabled)

(XEN) Processor #13 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x17] lapic_id[0x0f] enabled)

(XEN) Processor #15 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x18] lapic_id[0x21] enabled)

(XEN) Processor #33 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x19] lapic_id[0x23] enabled)

(XEN) Processor #35 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x25] enabled)

(XEN) Processor #37 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x27] enabled)

(XEN) Processor #39 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x29] enabled)

(XEN) Processor #41 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x2b] enabled)

(XEN) Processor #43 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x2d] enabled)

(XEN) Processor #45 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x2f] enabled)

(XEN) Processor #47 6:13 APIC version 21

(XEN) ACPI: LAPIC (acpi_id[0x20] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x21] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x22] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x23] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x24] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x25] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x26] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x27] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x28] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x29] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x2a] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x2b] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x2c] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x2d] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x2e] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x2f] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x30] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x31] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x32] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x33] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x34] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x35] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x36] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x37] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x38] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x39] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x3a] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x3b] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x3c] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x3d] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x3e] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC (acpi_id[0x3f] lapic_id[0xff] disabled)

(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])

(XEN) Overriding APIC driver with bigsmp

(XEN) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])

(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23

(XEN) ACPI: IOAPIC (id[0x01] address[0xfec3f000] gsi_base[24])

(XEN) IOAPIC[1]: apic_id 1, version 32, address 0xfec3f000, GSI 24-47

(XEN) ACPI: IOAPIC (id[0x02] address[0xfec7f000] gsi_base[48])

(XEN) IOAPIC[2]: apic_id 2, version 32, address 0xfec7f000, GSI 48-71

(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)

(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)

(XEN) ACPI: IRQ0 used by override.

(XEN) ACPI: IRQ2 used by override.

(XEN) ACPI: IRQ9 used by override.

(XEN) Enabling APIC mode:  Phys.  Using 3 I/O APICs

(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000

(XEN) PCI: MCFG configuration 0: base c0000000 segment 0 buses 0 - 255

(XEN) PCI: MCFG area at c0000000 reserved in E820

(XEN) x2APIC would not be enabled without EIM.

(XEN) Using ACPI (MADT) for SMP configuration information

(XEN) Using scheduler: SMP Credit Scheduler (credit)

(XEN) Initializing CPU#0

(XEN) Detected 1600.049 MHz processor.

(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K

(XEN) CPU: L2 cache: 256K

(XEN) CPU: L3 cache: 20480K

(XEN) CPU: Physical Processor ID: 0

(XEN) CPU: Processor Core ID: 0

(XEN) VMX: Supported advanced features:

(XEN)  - APIC MMIO access virtualisation

(XEN)  - APIC TPR shadow

(XEN)  - Extended Page Tables (EPT)

(XEN)  - Virtual-Processor Identifiers (VPID)

(XEN)  - Virtual NMI

(XEN)  - MSR direct-access bitmap

(XEN)  - Unrestricted Guest

(XEN) HVM: ASIDs enabled.

(XEN) HVM: VMX enabled

(XEN) HVM: Hardware Assisted Paging detected.

(XEN) xsave_init: cpu0: cntxt_max_size: 0x340 and states: 00000000:00000007

(XEN) xsave_init: using cntxt_size: 0x340 and states: 00000000:00000007

(XEN) Intel machine check reporting enabled on CPU#0.

(XEN) CPU0: Thermal monitoring enabled (TM1)

(XEN) I/O virtualisation disabled

(XEN) CPU0: Intel Genuine Intel(R) CPU            0     @ 1.60GHz stepping 01

(XEN) Enabled directed EOI with ioapic_ack_old on!

(XEN) Booting processor 1/2 eip 8c000

(XEN) Initializing CPU#1

(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K

(XEN) CPU: L2 cache: 256K

(XEN) CPU: L3 cache: 20480K

(XEN) CPU: Physical Processor ID: 0

(XEN) CPU: Processor Core ID: 1

(XEN) HVM: ASIDs enabled.

(XEN) xsave_init: cpu1: cntxt_max_size: 0x340 and states: 00000000:00000007

(XEN) Intel machine check reporting enabled on CPU#1.

(XEN) CPU1: Thermal monitoring enabled (TM1)

(XEN) CPU1: Intel Genuine Intel(R) CPU            0     @ 1.60GHz stepping 01

(XEN) Booting processor 2/4 eip 8c000

(XEN) Initializing CPU#2

(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K

(XEN) CPU: L2 cache: 256K

(XEN) CPU: L3 cache: 20480K

(XEN) CPU: Physical Processor ID: 0

(XEN) CPU: Processor Core ID: 2

(XEN) HVM: ASIDs enabled.

(XEN) xsave_init: cpu2: cntxt_max_size: 0x340 and states: 00000000:00000007

(XEN) Intel machine check reporting enabled on CPU#2.

(XEN) CPU2: Thermal monitoring enabled (TM1)

(XEN) CPU2: Intel Genuine Intel(R) CPU            0     @ 1.60GHz stepping 01

(XEN) Booting processor 3/6 eip 8c000

(XEN) Not responding.

(XEN) Inquiring remote APIC #6...

(XEN) ... APIC #6 ID: failed

(XEN) ... APIC #6 VERSION: failed

(XEN) ... APIC #6 SPIV: failed

(XEN) CPU #6 not responding - cannot use it.

(XEN) Booting processor 4/8 eip 8c000

(XEN) Not responding.

(XEN) Inquiring remote APIC #8...

(XEN) ... APIC #8 ID: failed

(XEN) ... APIC #8 VERSION: failed

(XEN) ... APIC #8 SPIV: failed

(XEN) CPU #8 not responding - cannot use it.

(XEN) Booting processor 5/10 eip 8c000

(XEN) Not responding.

(XEN) Inquiring remote APIC #10...

(XEN) ... APIC #10 ID: failed

(XEN) ... APIC #10 VERSION: failed

(XEN) ... APIC #10 SPIV: failed

(XEN) CPU #10 not responding - cannot use it.

(XEN) Booting processor 6/12 eip 8c000

(XEN) Not responding.

(XEN) Inquiring remote APIC #12...

(XEN) ... APIC #12 ID: failed

(XEN) ... APIC #12 VERSION: failed

(XEN) ... APIC #12 SPIV: failed

(XEN) CPU #12 not responding - cannot use it.

(XEN) Booting processor 7/14 eip 8c000

(XEN) Not responding.

(XEN) Inquiring remote APIC #14...

(XEN) ... APIC #14 ID: failed

(XEN) ... APIC #14 VERSION: failed

(XEN) ... APIC #14 SPIV: failed

(XEN) CPU #14 not responding - cannot use it.

(XEN) Booting processor 8/32 eip 8c00

 

Best regards,

 

Lv, Hui

 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.