[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [Xen-devel] RE: [Xen-users] Why my system only use 3.3G memory ?



It is certainly not a bug as such. It's a "Missing feature". Xen 2.0 does not 
support PAE (Page Address Extension). The PAE extension allows the processor to 
address memory beyond 4GB when accessing memory through the paged memory 
management unit (MMU or PMMU depending on who you talk to). This means that 
although the processor has a virtual limit of 4GB, it can map memory from 
beyond 4GB to map that into the virtual address space of the kernel. However, 
without PAE, the processor can only access 4GB and that's it, anything beyond 
that is outside it's reach. 

The other part of the problem is that the BIOS maps PCI devices into the first 
4GB of memory. Depending on how clever (or not) the BIOS is, it may eat more 
memory than it MUST. But there are several factors to take into account here. 
For example, some devices that require large address spaces will often also 
require that the device is evenly aligned to it's mapped size, so something 
that has a 64MB address space, should also be aligned to 64MB. This means that 
there may well be a hole of almost 64MB after this map. Some devices, such as 
graphics cards, can easily use up 256 or 512MB this way. [Of course, if the 
BIOS is clever, it would use the remaining part for some other mappings, but 
it's not always that the BIOS is clever enough to do this]. 

So, your 700 or so MB that are "missing" is caused by the PCI mappings. 
Generally speaking, this space is "non-RAM", so any access to the unused areas 
in this space will just be lost.

You can work around this by running Xen 3.0, but the Xen 3.0 is yet to be 
released, so it's not suitable for production environments. It does, however, 
support both 64-bit and PAE addressing of the memory, depending on the type of 
processor you have. 

Obviously, even with PAE, you will not be able to use ALL of the 4GB, because 
some of it will be mapped to the PCI hardware devices. But any "hole" between 
devices in the PCI map can be mapped in as RAM, so the kernel could possibly 
use up the gaps in the space with memory from above 4GB, rather than just 
"loosing" it. 

I hope this clarifies the situation.

--
Mats

________________________________

        From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx 
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of she11c0de she11c0de
        Sent: 07 July 2005 11:00
        To: m+Ian.Pratt@xxxxxxxxxxxx; xen-users@xxxxxxxxxxxxxxxxxxx
        Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
        Subject: [Xen-devel] RE: [Xen-users] Why my system only use 3.3G memory 
?
        
        

        
        I think it's maybe a bug, because if Xen 2.0 has 4G memory limit, even 
my Dell 2850 server has 6G memory , Xen 2.0 would use 4G memory on it, but it's 
just use 3327MB memory.

        I have another server Dell 2650 with 4G memory dual Xeon 3.0G CPU, had 
install Redhat 9.0 and Xen 2.0, it's not have any problem, i has running 8 
virtual server on it.

        Here is my Dell2650 XenLinux system info.

        [tongjian@XenServer43 11:59:19 ~ 350]$ xm info
        system                 : Linux
        host                   : XenServer43.testgroups.sina.com.cn
        release                : 2.6.9-xen0
        version                : #4 Fri Dec 31 19:09:27 CST 2004
        machine                : i686
        cores                  : 2
        hyperthreads_per_core  : 2
        cpu_mhz                : 3052
        memory                 : 3967
        free_memory            : 1757
        

        [tongjian@XenServer43 17:35:18 ~ 351]$ xm dmesg | less
        ERROR: cannot use unconfigured serial port COM1
         __  __            ____    ___  
         \ \/ /___ _ __   |___ \  / _ \ 
          \  // _ \ '_ \    __) || | | |
          /  \  __/ | | |  / __/ | |_| |
         /_/\_\___|_| |_| |_____(_)___/ 
                                        
         http://www.cl.cam.ac.uk/netos/xen
         University of Cambridge Computer Laboratory

         Xen version 2.0 (root@(none)) (gcc version 3.2.2 20030222 (Red Hat 
Linux 3.2.2-5)) Fri Dec 31 19:16:41 CST 2004
         Latest ChangeSet: information unavailable

        (XEN) Initialised 3967MB memory (1015776 pages) on a 3967MB machine
        (XEN) Xen heap size is 10684KB
        (XEN) CPU0: Before vendor init, caps: bfebfbff 00000000 00000000, 
vendor = 0
        (XEN) CPU#0: Physical ID: 0, Logical ID: 0
        (XEN) CPU caps: bfebfbff 00000000 00000000 00000000
        (XEN) found SMP MP-table at 000fe710
        (XEN) Memory Reservation 0xfe710, 4096 bytes
        (XEN) Memory Reservation 0xf0000, 4096 bytes
        (XEN) ACPI: RSDP (v000 DELL                                      ) @ 
0x000fdc40
        (XEN) ACPI: RSDT (v001 DELL   PE2650   0x00000001 MSFT 0x0100000a) @ 
0x000fdc54
        (XEN) ACPI: FADT (v001 DELL   PE2650   0x00000001 MSFT 0x0100000a) @ 
0x000fdc84
        (XEN) ACPI: MADT (v001 DELL   PE2650   0x00000001 MSFT 0x0100000a) @ 
0x000fdcf8
        (XEN) ACPI: SPCR (v001 DELL   PE2650   0x00000001 MSFT 0x0100000a) @ 
0x000fdd80
        (XEN) ACPI: DSDT (v001 DELL   PE2650   0x00000001 MSFT 0x0100000a) @ 
0x00000000
        (XEN) ACPI: Local APIC address 0xfee00000
        (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
        (XEN) Processor #0 Pentium 4(tm) XEON(tm) APIC version 20
        (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x06] enabled)
        (XEN) Processor #6 Pentium 4(tm) XEON(tm) APIC version 20
        (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)
        (XEN) Processor #1 Pentium 4(tm) XEON(tm) APIC version 20
        (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x07] enabled)
        (XEN) Processor #7 Pentium 4(tm) XEON(tm) APIC version 20

         

         

        
         




        >From: "Ian Pratt" <m+Ian.Pratt@xxxxxxxxxxxx>
        >To: "she11c0de she11c0de" 
<she11c0de@xxxxxxxxxxx>,<xen-users@xxxxxxxxxxxxxxxxxxx>
        >CC: <xen-devel@xxxxxxxxxxxxxxxxxxx>,<ian.pratt@xxxxxxxxxxxx>
        >Subject: RE: [Xen-users] Why my system only use 3.3G memory ?
        >Date: Thu, 7 Jul 2005 09:05:01 +0100
        >
        >(XEN) Physical RAM map:
        >(XEN)  0000000000000000 - 00000000000a0000 (usable)
        >(XEN)  0000000000100000 - 00000000cffc0000 (usable)
        >(XEN)  00000000cffc0000 - 00000000cffcfc00 (ACPI data)
        >(XEN)  00000000cffcfc00 - 00000000cffff000 (reserved)
        >(XEN)  00000000e0000000 - 00000000fec90000 (reserved)
        >(XEN)  00000000fed00000 - 00000000fed00400 (reserved)
        >(XEN)  00000000fee00000 - 00000000fee10000 (reserved)
        >(XEN)  00000000ffb00000 - 0000000100000000 (reserved)
        >(XEN) System RAM: 3327MB (3407232kB)
        >
        >Your BIOS has created a whole at the top of the memory map into which 
the memory mapped PCI devices have been placed, effectively hiding the memory 
from use.
        >
        >What is slightly odd is that the whole is larger than I've seen 
before. I presume its the area between cffff000 and e0000000 which isn't listed 
in the e820 map (everything above cffc0000 is unusable).
        >
        >You'll have to wait until xen3 and use x86_32pae or x86_64 to use more 
memory.
        >
        >Ian
        >
        > > -----Original Message-----
        > > From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
        > > [mailto:xen-users-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of
        > > she11c0de she11c0de
        > > Sent: 07 July 2005 06:50
        > > To: xen-users@xxxxxxxxxxxxxxxxxxx
        > > Subject: [Xen-users] Why my system only use 3.3G memory ?
        > >
        > > Hi, all
        > >
        > > I have a server used to install Xen-2.0.6 on CentOS 4.1 i386,
        > > hardware is DELL 2850, dual Xeon 3.2G CPU, 6G memory, use
        > > myself custom compiled xen kernel(dom0 and domU), i know xen
        > > 2.0 on i386 only use 4G memory, but i found on my server,
        > > it's only use 3327M memory, whit's problem, who can help me?
        > >
        > > Here is some info.
        > >
        > > [root@CentOS4 ~]#  xm info
        > > system                 : Linux
        > > host                   : CentOS4
        > > release                : 2.6.11.10-xen0
        > > version                : #3 Fri Jul 1 02:54:48 CST 2005
        > > machine                : i686
        > > cores                  : 2
        > > hyperthreads_per_core  : 2
        > > cpu_mhz                : 3192
        > > memory                 : 3327
        > > free_memory            : 2913
        > >
        > >
        > > [root@CentOS4 ~]#  xm dmesg | less
        > > ERROR: cannot use unconfigured serial port COM1
        > >  __  __            ____    ___   __
        > >  \ \/ /___ _ __   |___ \  / _ \ / /_
        > >   \  // _ \ '_ \    __) || | | | '_ \
        > >   /  \  __/ | | |  / __/ | |_| | (_) |
        > >  /_/\_\___|_| |_| |_____(_)___(_)___/
        > >
        > >  http://www.cl.cam.ac.uk/netos/xen
        > >  University of Cambridge Computer Laboratory
        > >
        > >  Xen version 2.0.6 (tongjian@(none)) (gcc version 3.4.3
        > > 20050227 (Red Hat 3.4.3-22.1)) Fri Jul  1 02:53:42 CST 2005
        > > Latest ChangeSet: information unavailable
        > >
        > > (XEN) WARNING: Only the first 4GB of the physical memory map
        > > can be accessed
        > > (XEN)          by Xen in 32-bit mode. Truncating the memory map...
        > > (XEN) Physical RAM map:
        > > (XEN)  0000000000000000 - 00000000000a0000 (usable)
        > > (XEN)  0000000000100000 - 00000000cffc0000 (usable)
        > > (XEN)  00000000cffc0000 - 00000000cffcfc00 (ACPI data)
        > > (XEN)  00000000cffcfc00 - 00000000cffff000 (reserved)
        > > (XEN)  00000000e0000000 - 00000000fec90000 (reserved)
        > > (XEN)  00000000fed00000 - 00000000fed00400 (reserved)
        > > (XEN)  00000000fee00000 - 00000000fee10000 (reserved)
        > > (XEN)  00000000ffb00000 - 0000000100000000 (reserved)
        > > (XEN) System RAM: 3327MB (3407232kB)
        > > (XEN) Xen heap: 10MB (10680kB)
        > > (XEN) CPU0: Before vendor init, caps: bfebfbff 20100000
        > > 00000000, vendor = 0
        > > (XEN) CPU#0: Physical ID: 0, Logical ID: 0
        > > (XEN) CPU caps: bfebfbff 20100000 00000000 00000000
        > > (XEN) found SMP MP-table at 000fe710
        > > (XEN) ACPI: RSDP (v000 DELL
        > >    ) @ 0x000fd650
        > > (XEN) ACPI: RSDT (v001 DELL   PE BKC   0x00000001 MSFT
        > > 0x0100000a) @ 0x000fd664
        > > (XEN) ACPI: FADT (v001 DELL   PE BKC   0x00000001 MSFT
        > > 0x0100000a) @ 0x000fd6b0
        > > (XEN) ACPI: MADT (v001 DELL   PE BKC   0x00000001 MSFT
        > > 0x0100000a) @ 0x000fd724
        > > (XEN) ACPI: SPCR (v001 DELL   PE BKC   0x00000001 MSFT
        > > 0x0100000a) @ 0x000fd7cc
        > > (XEN) ACPI: HPET (v001 DELL   PE BKC   0x00000001 MSFT
        > > 0x0100000a) @ 0x000fd81c
        > > (XEN) ACPI: MCFG (v001 DELL   PE BKC   0x00000001 MSFT
        > > 0x0100000a) @ 0x000fd854
        > > (XEN) ACPI: DSDT (v001 DELL   PE BKC   0x00000001 MSFT
        > > 0x0100000e) @ 0x00000000
        > > (XEN) ACPI: Local APIC address 0xfee00000
        > > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
        > > (XEN) Processor #0 Unknown CPU [15:4] APIC version 20
        > > (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x06] enabled)
        > > (XEN) Processor #6 Unknown CPU [15:4] APIC version 20
        > > (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)
        > > (XEN) Processor #1 Unknown CPU [15:4] APIC version 20
        > > [root@CentOS4 ~]#
        > >
        > >
        > >
        > >
        > >
        > >
        > > ________________________________
        > >
        > > æèæ MSN Messenger <http://g.msn.com/8HMBCN/2734??PS=47575>  
æåäéèæéååææîæãç
        > >
        

________________________________

        äç MSN Messenger <http://g.msn.com/8HMACN/2737??PS=47575>  äèæçæåèèäæ 

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.