RE: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32 permanently

["Li, Xin" <xin.li@xxxxxxxxx>]

> Okay. I think after 4.1 is branched we should simply remove 32-bit host
> support entirely. There's really no good reason for it to persist imo.

You're much more aggressive than me, but I totally agree :)
Thanks!
-Xin

> On 21/01/2011 15:19, "Li, Xin" <xin.li@xxxxxxxxx> wrote:
> 
> > Below is original log.  We found Xen hypervisor needs to map
> > VT-d/x2apci/MSI/MSI-x resources using map_domain_page.  And on 32bit, the
> map
> > page pool is trending to be used up.  Maybe we can fix it, but for large
> > machines with many devices and CPUs, we probably will still meet the 32bit
> > limitation.  So for new features which need map page in Xen hypervisor, I 
> > tend
> > to enable on 64 bit only.  I'm not sure how important 32bit Xen hypervisor 
> > is,
> > but I think 64 bit is becoming more and more popular.  Simply disabling is 
> > not
> > technically elegant, but for longer term, I think it's the trend because it
> > makes things simpler :).  Comments?
> > Thanks!
> > -Xin
> >
> > root (hd0,2)
> >  Filesystem type is ext2fs, partition type 0x83
> > kernel (hd0,2)/boot/xen.gz dom0_mem=512M iommu=1 loglvl=all guest_loglvl=all
> > un
> > restricted_guest=1  console=com1 sync_console iommu=1 msi=1
> conring_size=4M
> >    [Multiboot-elf, <0x100000:0x14433c:0x34cc4>, shtab=0x279078,
> > entry=0x100000]
> > module (hd0,2)/boot/vmlinuz-2.6-xen ro root=/dev/sda3 console=hvc0
> > earlyprintk=
> > xen ignore_loglevel pci=assign-busses
> >    [Multiboot-module @ 0x27a000, 0x3ede40 bytes]
> > module (hd0,2)/boot/initrd-2.6-xen.img
> >    [Multiboot-module @ 0x668000, 0x369800 bytes]
> >
> >  __  __            _  _    _                    _        _     _
> >  \ \/ /___ _ __   | || |  / |   _   _ _ __  ___| |_ __ _| |__ | | ___
> >   \  // _ \ '_ \  | || |_ | |__| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
> >   /  \  __/ | | | |__   _|| |__| |_| | | | \__ \ || (_| | |_) | |  __/
> >  /_/\_\___|_| |_|    |_|(_)_|   \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
> >
> > (XEN) Xen version 4.1-unstable (build@xxxxxxxxxxxx) (gcc version 4.1.2
> > 20080704 (Red Hat 4.1.2-44)) Sat Sep 11 06:03:28 CST 2010
> > (XEN) Latest ChangeSet: Fri Sep 10 19:06:33 2010 +0100 22132:3985fea87987
> > (XEN) Console output is synchronous.
> > (XEN) Bootloader: GNU GRUB 0.97
> > (XEN) Command line: dom0_mem=512M iommu=1 loglvl=all guest_loglvl=all
> > unrestricted_guest=1  console=com1 sync_console iommu=1 msi=1
> conring_size=4M
> > (XEN) Video information:
> > (XEN)  VGA is text mode 80x25, font 8x16
> > (XEN)  VBE/DDC methods: none; EDID transfer time: 0 seconds
> > (XEN)  EDID info not retrieved because no DDC retrieval method detected
> > (XEN) Disc information:
> > (XEN)  Found 1 MBR signatures
> > (XEN)  Found 1 EDD information structures
> > (XEN) Xen-e820 RAM map:
> > (XEN)  0000000000000000 - 000000000009bc00 (usable)
> > (XEN)  000000000009bc00 - 00000000000a0000 (reserved)
> > (XEN)  00000000000e0000 - 0000000000100000 (reserved)
> > (XEN)  0000000000100000 - 0000000020050000 (usable)
> > (XEN)  0000000020050000 - 0000000020250000 (reserved)
> > (XEN)  0000000020250000 - 0000000040000000 (usable)
> > (XEN)  0000000040000000 - 0000000040200000 (reserved)
> > (XEN)  0000000040200000 - 000000007f553000 (usable)
> > (XEN)  000000007f553000 - 000000007f5e7000 (reserved)
> > (XEN)  000000007f5e7000 - 000000007f7e7000 (ACPI NVS)
> > (XEN)  000000007f7e7000 - 000000007f7ff000 (ACPI data)
> > (XEN)  000000007f7ff000 - 000000007f800000 (usable)
> > (XEN)  000000007f800000 - 0000000080000000 (reserved)
> > (XEN)  00000000f8000000 - 00000000fc000000 (reserved)
> > (XEN)  00000000fec00000 - 00000000fec01000 (reserved)
> > (XEN)  00000000fed10000 - 00000000fed14000 (reserved)
> > (XEN)  00000000fed18000 - 00000000fed1a000 (reserved)
> > (XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
> > (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> > (XEN)  00000000ff980000 - 00000000ffc00000 (reserved)
> > (XEN)  00000000ffd80000 - 0000000100000000 (reserved)
> > (XEN) System RAM: 2032MB (2081724kB)
> > (XEN) ACPI: RSDP 000F0410, 0024 (r2  INTEL)
> > (XEN) ACPI: XSDT 7F7FDE18, 006C (r1  INTEL  SNB-CPT  6222004 MSFT
> 10013)
> > (XEN) ACPI: FACP 7F752D98, 00F4 (r4  INTEL  SNB-CPT  6222004 MSFT
> 10013)
> > (XEN) ACPI: DSDT 7F725018, EACA (r2  INTEL  SNB-CPT        0 INTL
> 20100331)
> > (XEN) ACPI: FACS 7F7E6D40, 0040
> > (XEN) ACPI: APIC 7F7FCF18, 0092 (r2  INTEL  SNB-CPT  6222004 MSFT
> 10013)
> > (XEN) ACPI: TCPA 7F79FD18, 0032 (r2                        0
> 0)
> > (XEN) ACPI: MCFG 7F79FC98, 003C (r1 INTEL  SNDYBRDG  6222004 MSFT
> 97)
> > (XEN) ACPI: HPET 7F79FC18, 0038 (r1 A M I   PCHHPET  6222004 AMI.
> 3)
> > (XEN) ACPI: SSDT 7F753018, 1068 (r1 TrmRef PtidDevc     1000 INTL 20100331)
> > (XEN) ACPI: BOOT 7F79FB98, 0028 (r1 INTEL  SNB-CPT   6222004 AMI
> 10013)
> > (XEN) ACPI: SSDT 7F751018, 0996 (r1  PmRef    CpuPm     3000 INTL
> 20100331)
> > (XEN) ACPI: DMAR 7F752F18, 00B0 (r1 INTEL      SNB         1 INTL
> 1)
> > (XEN) No NUMA configuration found
> > (XEN) Faking a node at 0000000000000000-000000007f800000
> > (XEN) Xen heap: 9MB (9716kB)
> > (XEN) Domain heap initialised
> > (XEN) found SMP MP-table at 000fcaa0
> > (XEN) DMI 2.6 present.
> > (XEN) Using APIC driver default
> > (XEN) ACPI: PM-Timer IO Port: 0x408
> > (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0]
> > (XEN) ACPI: 32/64X FACS address mismatch in FADT -
> 7f79df40/000000007f7e6d40,
> > using 32
> > (XEN) ACPI:                  wakeup_vec[7f79df4c], vec_size[20]
> > (XEN) ACPI: Local APIC address 0xfee00000
> > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> > (XEN) Processor #0 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
> > (XEN) Processor #2 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
> > (XEN) Processor #4 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
> > (XEN) Processor #6 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
> > (XEN) Processor #1 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
> > (XEN) Processor #3 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
> > (XEN) Processor #5 6:10 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
> > (XEN) Processor #7 6:10 APIC version 21
> > (XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
> > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
> > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> > (XEN) ACPI: IRQ0 used by override.
> > (XEN) ACPI: IRQ2 used by override.
> > (XEN) ACPI: IRQ9 used by override.
> > (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
> > (XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
> > (XEN) Table is not found!
> > (XEN) Using ACPI (MADT) for SMP configuration information
> > (XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X
> > (XEN) Early fatal page fault at e008:ff1da1c6 (cr2=ffffffb8, ec=0000)
> > (XEN) Stack dump: ff279610 00000080 ff24bfb0 fffff000 00000000 ff24bdd4
> > ffffc000 00000000
> > ff1da1c6 0000e008 00010082 00000000 00000000 ff24bdd4 ff130e89 ff21828a
> > 00000006 ffbfec64 ff279610 00bde0a0 00000000 ff24be04 ff1384f7 0007e95c
> > 00000002 00000000 ff11d4ef 00000096 00000096 f73de0a0 ff279610 ff279638
> > ff2795b8 ff24be34 ff13c740 ff279568 00000004 c9008020 00000000 ff21ac1c
> > ff2795e4 00000096 ff279568 ff21a7bc 00000008 ff24be54 ff13e038 ff2795b8
> > 00010970 02fa0000 ff279910 ff279910 00000007 ff24be74 ff144b0a ff279910
> > ff24be74 ff232570 00000007 00000000 00100800 ff24bfa4 ff234bf7 ff21dc0c
> > 00000080 00000008 00000000 00000000 00000000 00000000 00000000
> 00000000
> > 00000000 00000000 00000000 00000000 00000000 0007bef0 00000000
> 00000000
> > ff220260 00000000 00000000 1f000000 00000000 00000000 ff07bc50 ff07bfb0
> > ff07bef0 00757640 00848000 00000015 ffffffff 00000000 00000000 00000000
> > 00000000 00000000 00000000 7f7ff000 00000000 40000000 00000000 00000000
> > ff07bef0 00000000 00000000 00000000 00000000 00000007 00000007 00000008
> > 000001a4 000001a4 ff07bee0 01000000 00000000 ffd80000 00000000 ffd80000
> > 00000000 00000015 ffffffff ff07bf24 0000097d 00000000 00000000 00000000
> > 00000008 0000006e 00000001 00000003 000002f8 00000000 00000000
> 00220d70
> > ff2737c0 00067ebc ff10006c 0007bfb0 00000000 00000000 00000000 00000000
> > 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000
> > 00000000 00000000 00000000 00000000 00000000 00000000 fffff000 00000000
> >
> >> -----Original Message-----
> >> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> >> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Keir Fraser
> >> Sent: Friday, January 21, 2011 9:09 PM
> >> To: Zhang, Fengzhe; xen-devel@xxxxxxxxxxxxxxxxxxx
> >> Subject: Re: [Xen-devel] [PATCH]x86:x2apic: Disable x2apic on x86-32
> >> permanently
> >>
> >> Could you give more info on the bug, such as a backtrace? I wonder whether
> >> this might be better fixed in some more elegant way. Also I can't eyeball
> >> where map_domain_page() is being called during x2apic setup.
> >>
> >>  Thanks,
> >>  Keir
> >>
> >> On 21/01/2011 12:21, "Zhang, Fengzhe" <fengzhe.zhang@xxxxxxxxx> wrote:
> >>
> >>> x86:x2apic: Disable x2apic on x86-32 permanently
> >>>
> >>> x2apic initialization on x86_32 uses vcpu pointer before it is 
> >>> initialized.
> >>> As
> >>> x2apic is unlikely to be used on x86_32, this patch disables x2apic
> >>> permanently on x86_32. It also asserts the sanity of vcpu pointer before
> >>> dereference to prevent further misuse.
> >>>
> >>> Signed-off-by: Fengzhe Zhang <fengzhe.zhang@xxxxxxxxx>
> >>>
> >>> diff -r 02c0af2bf280 xen/arch/x86/apic.c
> >>> --- a/xen/arch/x86/apic.c Mon Jan 17 18:05:52 2011 +0000
> >>> +++ b/xen/arch/x86/apic.c Wed Jan 19 03:24:16 2011 -0500
> >>> @@ -961,6 +961,22 @@
> >>>      if ( !cpu_has_x2apic )
> >>>          return;
> >>>
> >>> +#ifdef __i386__
> >>> +    clear_bit(X86_FEATURE_X2APIC, boot_cpu_data.x86_capability);
> >>> +    if (x2apic_enabled) {
> >>> +        uint64_t msr_content;
> >>> +
> >>> +        rdmsrl(MSR_IA32_APICBASE, msr_content);
> >>> +        msr_content &= ~(MSR_IA32_APICBASE_ENABLE |
> >> MSR_IA32_APICBASE_EXTD);
> >>> +        wrmsrl(MSR_IA32_APICBASE, msr_content);
> >>> +        msr_content |= MSR_IA32_APICBASE_ENABLE;
> >>> +        wrmsrl(MSR_IA32_APICBASE, msr_content);
> >>> +        x2apic_enabled = 0;
> >>> +    }
> >>> +    printk("x2APIC disabled permanently on x86_32.\n");
> >>> +    return;
> >>> +#endif
> >>> +
> >>>      if ( !opt_x2apic )
> >>>      {
> >>>          if ( !x2apic_enabled )
> >>> diff -r 02c0af2bf280 xen/arch/x86/x86_32/domain_page.c
> >>> --- a/xen/arch/x86/x86_32/domain_page.c Mon Jan 17 18:05:52 2011 +0000
> >>> +++ b/xen/arch/x86/x86_32/domain_page.c Wed Jan 19 03:24:16 2011 -0500
> >>> @@ -53,6 +53,8 @@
> >>>      perfc_incr(map_domain_page_count);
> >>>
> >>>      v = mapcache_current_vcpu();
> >>> +    /* Prevent vcpu pointer being used before initialize. */
> >>> +    ASSERT((unsigned long)v != 0xfffff000);
> >>>
> >>>      dcache = &v->domain->arch.mapcache;
> >>>      vcache = &v->arch.mapcache;
> >>> _______________________________________________
> >>> Xen-devel mailing list
> >>> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>> http://lists.xensource.com/xen-devel
> >>
> >>
> >>
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@xxxxxxxxxxxxxxxxxxx
> >> http://lists.xensource.com/xen-devel
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel