[Xen-users] Xen 3 not seeing the right amount of RAM on x86_64

["Chuck McIntyre" <chuck.mcintyre@xxxxxxxxx>]

Hi,

I am having a problem where Xen is not seeing the appropriate amount
of memory allocated. At first I thought this was similar to previously
described issues with the BIOS remapped hole for PCI devices (such as
in a post here 
http://lists.xensource.com/archives/html/xen-users/2005-07/msg00177.html),
however, we only see about 3.5GB (see below) of RAM regardless of
whether we have a system with 4GB of RAM or 8GB of RAM (as seen by the
BIOS). Regardless, when we boot into the non-xen kernel available on
the same box, the OS then sees all the available RAM.

I have tried to provide as much relevant information as I could
without being spammy. Please let me know if I need to provide
additional details. I would appreciate any help here, as this one has
us stumped, or do we have a bug with our hardware version?

__4GB system___

[me@molotov ~]$ uname -a
Linux molotov.mydomain 2.6.18-8.1.15.el5.centos.plusxen #1 SMP Thu Oct
25 07:21:34 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
[me@molotov ~]$ rpm -qa | grep xen
xen-3.0.3-25.0.4.el5
xen-libs-3.0.3-25.0.4.el5
xen-libs-3.0.3-25.0.4.el5
kernel-xen-2.6.18-8.1.15.el5.centos.plus
[me@molotov ~]$ grep MemTotal /proc/meminfo
MemTotal:      3428400 kB
top gives me:
Mem:   3428400k total,   422916k used,  3005484k free,    21648k buffers
[root@molotov ~]# xm info 2>&1 | grep memory
total_memory           : 3535
free_memory            : 129
xentop gives me:
Mem: 3620404k total, 3488052k used, 132352k free    CPUs: 4 @ 2200MHz
dmesg
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 00000000d1d4c000 (usable)
On node 0 totalpages: 859468
  DMA zone: 859468 pages, LIFO batch:31
DMI present.

-=After reboot=-

BIOS gives me:
Initializing USB Controllers .. Done.
4096MB OK
[me@molotov ~]$ grep MemTotal /proc/meminfo
MemTotal:      4050520 kB
top gives
Mem:   4050520k total,   264668k used,  3785852k free,    20612k buffers
dmesg
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009dc00 (usable)
 BIOS-e820: 000000000009dc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000dcff0000 (usable)
 BIOS-e820: 00000000dcff0000 - 00000000dcffe000 (ACPI data)
 BIOS-e820: 00000000dcffe000 - 00000000dd000000 (ACPI NVS)
 BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
 BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fef00000 (reserved)
 BIOS-e820: 00000000ff700000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000123000000 (usable)
DMI present.
Bootmem setup node 0 0000000000000000-0000000080000000
Bootmem setup node 1 0000000080000000-0000000123000000
On node 0 totalpages: 515808
  DMA zone: 2728 pages, LIFO batch:0
  DMA32 zone: 513080 pages, LIFO batch:31
On node 1 totalpages: 515144
  DMA32 zone: 373744 pages, LIFO batch:31
  Normal zone: 141400 pages, LIFO batch:31



__ 8GB System __

[me@molotov ~]$ uname -a
Linux benelli.mydomain 2.6.18-8.1.15.el5.centos.plusxen #1 SMP Thu Oct
25 07:21:34 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
[me@molotov ~]$ rpm -qa |  grep xen
kernel-xen-2.6.18-8.1.15.el5.centos.plus
xen-libs-3.0.3-25.0.4.el5
xen-libs-3.0.3-25.0.4.el5
xen-3.0.3-25.0.4.el5
[me@benelli ~]$ grep MemTotal /proc/meminfo
MemTotal:      3428336 kB
top gives me:
Mem:   3428336k total,   437888k used,  2990448k free,    16580k buffers
[root@benelli ~]# xm info 2>&1| grep memory
total_memory           : 3535
free_memory            : 0
xentop gives me:
Mem: 3620336k total, 3620064k used, 272k free    CPUs: 4 @ 2200MHz
dmesg
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 00000000d1d3c000 (usable)
DMI present.

-=After Reboot=-

BIOS gives me:
Initializing USB Controllers .. Done.
8192MB OK

After reboot into non-xen kernel:

[root@benelli ~]# grep MemTotal /proc/meminfo
MemTotal:      8181124 kB
Top gives
Mem:   8181124k total,   284732k used,  7896392k free,    16944k buffers
dmesg
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009c400 (usable)
 BIOS-e820: 000000000009c400 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000dcfe0000 (usable)
 BIOS-e820: 00000000dcfe0000 - 00000000dcfee000 (ACPI data)
 BIOS-e820: 00000000dcfee000 - 00000000dcff0000 (ACPI NVS)
 BIOS-e820: 00000000dcff0000 - 00000000dcffe000 (reserved)
 BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
 BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fef00000 (reserved)
 BIOS-e820: 00000000ff700000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000223000000 (usable)
DMI present.
Bootmem setup node 0 0000000000000000-0000000123000000
Bootmem setup node 1 0000000123000000-0000000223000000
On node 0 totalpages: 1030931
  DMA zone: 2723 pages, LIFO batch:0
  DMA32 zone: 886808 pages, LIFO batch:31
  Normal zone: 141400 pages, LIFO batch:31
On node 1 totalpages: 1034240
  Normal zone: 1034240 pages, LIFO batch:31


__Both Systems__

Here is grub.conf (equivalent on both systems):
default=0
timeout=5
serial --unit=0 --speed=9600
terminal --timeout=5 serial console
title CentOS (2.6.18-8.1.15.el5.centos.plusxen)
        root (hd0,0)
        kernel /boot/xen.gz-2.6.18-8.1.15.el5.centos.plus console=vga
        module /boot/vmlinuz-2.6.18-8.1.15.el5.centos.plusxen ro
root=/dev/md0 console=ttyS0 console=ttyS0,9600n8 console=tty0
xencons=tty
        module /boot/initrd-2.6.18-8.1.15.el5.centos.plusxen.img
title CentOS (2.6.18-8.1.15.el5.centos.plus)
        root (hd0,0)
        kernel /boot/vmlinuz-2.6.18-8.1.15.el5.centos.plus ro
root=/dev/md0 console=ttyS0 console=ttyS0,9600n8 console=tty0
xencons=tty
        initrd /boot/initrd-2.6.18-8.1.15.el5.centos.plus.img

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users