Hi Folks:
I
run into a problem when enabling Xen in next generation server platforms with Xen
c/s is 21380.
Xen
reported “CPU Not responding” when booting up 32 CPUs( 2 sockets
with 8 cores/16threads total). The log files belonging showed something wrong
with APCI. So I added x2apic=0 in the xen grub line, but the symptom remained.
However,
Native RHEL5.5 can normally started in this platforms.
Does
anyone meet this problem and know how to solve it or give some hints?
Thanks
very much!
Btw,
I also tried the latest xen (version 4.1 rc5). Hypervisor still hang when
booting CPUs with “error code -5”?. (if someone need the log files,
I’ll post it)
__ __
_____ ____
_ _
_
\ \/ /___ _ __ |___ / |
___| _ _ _ __ ___| |_ __ _| |__ | | ___
\ // _ \ '_
\ |_ \ |___ \ __| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
/ \ __/ | | | ___)
| ___) |__| |_| | | | \__ \ || (_| | |_) | | __/
/_/\_\___|_| |_|
|____(_)____/ \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
(XEN) Xen version 3.5-unstable (root@) (gcc
version 4.1.2 20080704 (Red Hat 4.1.2-48)) Thu Feb 10 14:40:15 EST 2011
(XEN) Latest ChangeSet: Wed Dec 09 10:59:31
2009 +0000 20605:8f304c003af4
(XEN) Command line: com1=115200,8n1
console=com1 loglvl=all
(XEN) Video information:
(XEN) VGA is text mode 80x25, font
8x16
(XEN) VBE/DDC methods: V2; EDID
transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 0 MBR signatures
(XEN) Found 1 EDD information
structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 -
000000000009bc00 (usable)
(XEN) 000000000009bc00 -
00000000000a0000 (reserved)
(XEN) 00000000000e0000 -
0000000000100000 (reserved)
(XEN) 0000000000100000 -
00000000ab70f000 (usable)
(XEN) 00000000ab70f000 -
00000000ab88d000 (ACPI NVS)
(XEN) 00000000ab88d000 -
00000000bed0d000 (usable)
(XEN) 00000000bed0d000 -
00000000bede7000 (reserved)
(XEN) 00000000bede7000 -
00000000bee70000 (ACPI NVS)
(XEN) 00000000bee70000 -
00000000beeae000 (ACPI data)
(XEN) 00000000beeae000 -
00000000beeaf000 (ACPI NVS)
(XEN) 00000000beeaf000 -
00000000beeca000 (ACPI data)
(XEN) 00000000beeca000 -
00000000befd2000 (ACPI NVS)
(XEN) 00000000befd2000 -
00000000bf000000 (usable)
(XEN) 00000000bf000000 -
00000000d0000000 (reserved)
(XEN) 00000000fec00000 -
00000000fec01000 (reserved)
(XEN) 00000000fed19000 -
00000000fed1a000 (reserved)
(XEN) 00000000fed1c000 -
00000000fed20000 (reserved)
(XEN) 00000000fee00000 -
00000000fee01000 (reserved)
(XEN) 00000000ffc00000 -
0000000100000000 (reserved)
(XEN) 0000000100000000 -
0000000840000000 (usable)
(XEN) ACPI: RSDP 000F0410, 0024 (r2
INTEL)
(XEN) ACPI: XSDT BEEC8E18, 008C (r1
INTEL ROMLEY 6222004 INTL 20090903)
(XEN) ACPI: FACP BEEC7D98, 00F4 (r4
INTEL ROMLEY 6222004 INTL 20090903)
(XEN) ACPI: DSDT BEEAF018, 1660E (r2
INTEL ROMLEY 27 INTL 20100331)
(XEN) ACPI: FACS BEEC8D40, 0040
(XEN) ACPI: APIC BEEC6718, 066A (r3
INTEL ROMLEY 6222004 INTL 20090903)
(XEN) ACPI: SPMI BEEC9F18, 0040 (r5
INTEL ROMLEY 6222004 INTL 20090903)
(XEN) ACPI: TCPA BEEC9E98, 0032
(r2
0 0)
(XEN) ACPI: MCFG BEEC9E18, 003C (r1
INTEL ROMLEY 6222004 INTL 20090903)
(XEN) ACPI: SRAT BEEADC18, 02A8 (r3
INTEL ROMLEY 6222004 INTL 20090903)
(XEN) ACPI: SLIT BEEC9D18, 0030 (r1
INTEL ROMLEY 6222004 INTL 20090903)
(XEN) ACPI: MSCT BEEC9C98, 004E (r1
INTEL ROMLEY 6222004 INTL 20090903)
(XEN) ACPI: HPET BEEC9D98, 0038 (r1
INTL ROMLEY 6222004 INTL 20090903)
(XEN) ACPI: SSDT BEEC9C18, 002B (r2
INTEL PtidDevc 1000 INTL 20100331)
(XEN) ACPI: BOOT BEEC9B98, 0028 (r1
INTEL ROMLEY 6222004 INTL 20090903)
(XEN) ACPI: SSDT BEE70018, 3CFC4 (r2
INTEL CpuPm 4000 INTL 20100331)
(XEN) ACPI: DMAR BEEC7798, 0130 (r1
INTEL
RML 1
INTL 1)
(XEN) System RAM: 32731MB (33517028kB)
(XEN) SRAT: PXM 0 -> APIC 0 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 1 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 2 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 3 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 4 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 5 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 6 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 7 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 8 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 9 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 10 -> Node
0
(XEN) SRAT: PXM 0 -> APIC 11 -> Node
0
(XEN) SRAT: PXM 0 -> APIC 12 -> Node
0
(XEN) SRAT: PXM 0 -> APIC 13 -> Node
0
(XEN) SRAT: PXM 0 -> APIC 14 -> Node
0
(XEN) SRAT: PXM 0 -> APIC 15 -> Node
0
(XEN) SRAT: PXM 1 -> APIC 32 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 33 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 34 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 35 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 36 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 37 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 38 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 39 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 40 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 41 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 42 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 43 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 44 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 45 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 46 -> Node
1
(XEN) SRAT: PXM 1 -> APIC 47 -> Node
1
(XEN) SRAT: Node 0 PXM 0 0-c0000000
(XEN) SRAT: Node 0 PXM 0
100000000-440000000
(XEN) SRAT: Node 1 PXM 1
440000000-840000000
(XEN) NUMA: Using 18 for the hash shift.
(XEN) Domain heap initialised DMA width 32
bits
(XEN) found SMP MP-table at 000fcdd0
(XEN) DMI 2.6 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0],
pm1x_evt[400,0]
(XEN)
ACPI:
wakeup_vec[beec8d4c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00]
lapic_id[0x00] enabled)
(XEN) Processor #0 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x01]
lapic_id[0x02] enabled)
(XEN) Processor #2 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02]
lapic_id[0x04] enabled)
(XEN) Processor #4 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03]
lapic_id[0x06] enabled)
(XEN) Processor #6 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x04]
lapic_id[0x08] enabled)
(XEN) Processor #8 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x05]
lapic_id[0x0a] enabled)
(XEN) Processor #10 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x06]
lapic_id[0x0c] enabled)
(XEN) Processor #12 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x07]
lapic_id[0x0e] enabled)
(XEN) Processor #14 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x08]
lapic_id[0x20] enabled)
(XEN) Processor #32 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x09]
lapic_id[0x22] enabled)
(XEN) Processor #34 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x0a]
lapic_id[0x24] enabled)
(XEN) Processor #36 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x0b]
lapic_id[0x26] enabled)
(XEN) Processor #38 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x0c]
lapic_id[0x28] enabled)
(XEN) Processor #40 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x0d]
lapic_id[0x2a] enabled)
(XEN) Processor #42 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x0e]
lapic_id[0x2c] enabled)
(XEN) Processor #44 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x0f]
lapic_id[0x2e] enabled)
(XEN) Processor #46 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x10]
lapic_id[0x01] enabled)
(XEN) Processor #1 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x11]
lapic_id[0x03] enabled)
(XEN) Processor #3 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x12]
lapic_id[0x05] enabled)
(XEN) Processor #5 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x13]
lapic_id[0x07] enabled)
(XEN) Processor #7 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x14]
lapic_id[0x09] enabled)
(XEN) Processor #9 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x15]
lapic_id[0x0b] enabled)
(XEN) Processor #11 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x16]
lapic_id[0x0d] enabled)
(XEN) Processor #13 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x17]
lapic_id[0x0f] enabled)
(XEN) Processor #15 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x18]
lapic_id[0x21] enabled)
(XEN) Processor #33 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x19]
lapic_id[0x23] enabled)
(XEN) Processor #35 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x1a]
lapic_id[0x25] enabled)
(XEN) Processor #37 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x1b]
lapic_id[0x27] enabled)
(XEN) Processor #39 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x1c]
lapic_id[0x29] enabled)
(XEN) Processor #41 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x1d]
lapic_id[0x2b] enabled)
(XEN) Processor #43 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x1e]
lapic_id[0x2d] enabled)
(XEN) Processor #45 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x1f]
lapic_id[0x2f] enabled)
(XEN) Processor #47 6:13 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x20] lapic_id[0xff]
disabled)
(XEN) ACPI: LAPIC (acpi_id[0x21]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x22]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x23]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x24]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x25]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x26]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x27]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x28]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x29]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x2a]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x2b]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x2c]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x2d]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x2e]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x2f]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x30]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x31]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x32]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x33]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x34]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x35]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x36]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x37]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x38]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x39]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x3a]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x3b]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x3c]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x3d]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x3e] lapic_id[0xff]
disabled)
(XEN) ACPI: LAPIC (acpi_id[0x3f]
lapic_id[0xff] disabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] dfl
dfl lint[0x1])
(XEN) Overriding APIC driver with bigsmp
(XEN) ACPI: IOAPIC (id[0x00]
address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 0, version 32,
address 0xfec00000, GSI 0-23
(XEN) ACPI: IOAPIC (id[0x01]
address[0xfec3f000] gsi_base[24])
(XEN) IOAPIC[1]: apic_id 1, version 32,
address 0xfec3f000, GSI 24-47
(XEN) ACPI: IOAPIC (id[0x02]
address[0xfec7f000] gsi_base[48])
(XEN) IOAPIC[2]: apic_id 2, version 32,
address 0xfec7f000, GSI 48-71
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0
global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9
global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Phys.
Using 3 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base:
0xfed00000
(XEN) PCI: MCFG configuration 0: base
c0000000 segment 0 buses 0 - 255
(XEN) PCI: MCFG area at c0000000 reserved
in E820
(XEN) x2APIC would not be enabled without
EIM.
(XEN) Using ACPI (MADT) for SMP
configuration information
(XEN) Using scheduler: SMP Credit Scheduler
(credit)
(XEN) Initializing CPU#0
(XEN) Detected 1600.049 MHz processor.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 20480K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access
virtualisation
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers
(VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging
detected.
(XEN) xsave_init: cpu0: cntxt_max_size:
0x340 and states: 00000000:00000007
(XEN) xsave_init: using cntxt_size: 0x340
and states: 00000000:00000007
(XEN) Intel machine check reporting enabled
on CPU#0.
(XEN) CPU0: Thermal monitoring enabled
(TM1)
(XEN) I/O virtualisation disabled
(XEN) CPU0: Intel Genuine Intel(R)
CPU
0 @ 1.60GHz stepping 01
(XEN) Enabled directed EOI with
ioapic_ack_old on!
(XEN) Booting processor 1/2 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 20480K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) HVM: ASIDs enabled.
(XEN) xsave_init: cpu1: cntxt_max_size:
0x340 and states: 00000000:00000007
(XEN) Intel machine check reporting enabled
on CPU#1.
(XEN) CPU1: Thermal monitoring enabled
(TM1)
(XEN) CPU1: Intel Genuine Intel(R)
CPU
0 @ 1.60GHz stepping 01
(XEN) Booting processor 2/4 eip 8c000
(XEN) Initializing CPU#2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 20480K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 2
(XEN) HVM: ASIDs enabled.
(XEN) xsave_init: cpu2: cntxt_max_size:
0x340 and states: 00000000:00000007
(XEN) Intel machine check reporting enabled
on CPU#2.
(XEN) CPU2: Thermal monitoring enabled
(TM1)
(XEN) CPU2: Intel Genuine Intel(R)
CPU
0 @ 1.60GHz stepping 01
(XEN) Booting processor 3/6 eip 8c000
(XEN) Not responding.
(XEN) Inquiring remote APIC #6...
(XEN) ... APIC #6 ID: failed
(XEN) ... APIC #6 VERSION: failed
(XEN) ... APIC #6 SPIV: failed
(XEN) CPU #6 not responding - cannot use
it.
(XEN) Booting processor 4/8 eip 8c000
(XEN) Not responding.
(XEN) Inquiring remote APIC #8...
(XEN) ... APIC #8 ID: failed
(XEN) ... APIC #8 VERSION: failed
(XEN) ... APIC #8 SPIV: failed
(XEN) CPU #8 not responding - cannot use
it.
(XEN) Booting processor 5/10 eip 8c000
(XEN) Not responding.
(XEN) Inquiring remote APIC #10...
(XEN) ... APIC #10 ID: failed
(XEN) ... APIC #10 VERSION: failed
(XEN) ... APIC #10 SPIV: failed
(XEN) CPU #10 not responding - cannot use
it.
(XEN) Booting processor 6/12 eip 8c000
(XEN) Not responding.
(XEN) Inquiring remote APIC #12...
(XEN) ... APIC #12 ID: failed
(XEN) ... APIC #12 VERSION: failed
(XEN) ... APIC #12 SPIV: failed
(XEN) CPU #12 not responding - cannot use
it.
(XEN) Booting processor 7/14 eip 8c000
(XEN) Not responding.
(XEN) Inquiring remote APIC #14...
(XEN) ... APIC #14 ID: failed
(XEN) ... APIC #14 VERSION: failed
(XEN) ... APIC #14 SPIV: failed
(XEN) CPU #14 not responding - cannot use
it.
(XEN) Booting processor 8/32 eip 8c00
Best regards,
Lv, Hui
