|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] [PATCH v2 1/1] tools: Handle xc_maxmem adjustments
On 04/15/15 07:48, Stefano Stabellini wrote:
> On Wed, 15 Apr 2015, Andrew Cooper wrote:
>> On 15/04/15 10:57, Stefano Stabellini wrote:
>>> On Wed, 15 Apr 2015, Andrew Cooper wrote:
>>>> On 14/04/15 23:06, Don Slutz wrote:
>>>>> This fixes an issue where "xl save" followed by "xl restore" reports:
>>>>> "xc: error: Failed to allocate memory for batch.!: Internal error"
>>>>>
>>>>> One of the ways to get into this state is to have more then 4 e1000
>>>>> nics configured.
>>>>>
>>>>> Signed-off-by: Don Slutz <dslutz@xxxxxxxxxxx>
>>>> I still don't think this is the correct solution, although I will
>>>> concede that this is a far better patch than v1.
>>>>
>>>> Going back to the original problem, why does Qemu need to change maxmem
>>>> in the first place? You talk about e1000 option roms, but the option
>>>> roms themselves must be allocated in an appropriate PCI bridge window.
>>> QEMU allocates actual memory to store the option roms using
>>> xc_domain_populate_physmap_exact, increasing the total amount of ram
>>> allocated to the guest. QEMU needs to increase maxmem too, otherwise
>>> memory allocations will fail after a few option roms.
>>
>> It doesn't need to. The domain already has that much ram allocated to
>> itself and otherwise wasted. Qemu should reuse that ram rather than
>> wasting more.
This "otherwise wasted" I do not understand at all.
for:
maxmem = "2056"
memory = "2056"
mmio_hole = "3072"
videoram = "8"
vif = [
"model=e1000,bridge=xenbr0,mac=00:0c:29:86:44:a0",
"model=e1000,bridge=xenbr0,mac=00:0c:29:86:44:aa",
"model=e1000,bridge=xenbr0,mac=00:0c:29:86:44:b4",
"model=e1000,bridge=xenbr0,mac=00:0c:29:86:44:be",
]
xc: detail: VIRTUAL MEMORY ARRANGEMENT:
xc: detail: Loader: 0000000000100000->00000000001c4a1c
xc: detail: Modules: 0000000000000000->0000000000000000
xc: detail: TOTAL: 0000000000000000->0000000080000000
xc: detail: ENTRY: 0000000000100038
xc: detail: PHYSICAL MEMORY ALLOCATION:
xc: detail: 4KB PAGES: 0x0000000000000200
xc: detail: 2MB PAGES: 0x00000000000001ff
xc: detail: 1GB PAGES: 0x0000000000000001
xl list:
e1000x4-2g 36 2057 1 --p---
0.0
2057 is 2056 + LIBXL_MAXMEM_CONSTANT
As reported by xc_domain_getinfo()
info.max_memkb == 2106432 ==> pages 526608
info.nr_pages == 526592 ==> 2106368 KiB
So there are 16 "wasted" pages, part of the pages from physical address
0xa0000 to 0xbffff.
hvmloader:
(d36) [2015-04-15 21:03:45] BIOS map:
(d36) [2015-04-15 21:03:45] 10000-100d3: Scratch space
(d36) [2015-04-15 21:03:45] c0000-fffff: Main BIOS
(d36) [2015-04-15 21:03:45] E820 table:
(d36) [2015-04-15 21:03:45] [00]: 00000000:00000000 -
00000000:000a0000: RAM
(d36) [2015-04-15 21:03:45] HOLE: 00000000:000a0000 - 00000000:000c0000
(d36) [2015-04-15 21:03:45] [01]: 00000000:000c0000 -
00000000:00100000: RESERVED
(d36) [2015-04-15 21:03:45] [02]: 00000000:00100000 -
00000000:40000000: RAM
(d36) [2015-04-15 21:03:45] HOLE: 00000000:40000000 - 00000000:fc000000
(d36) [2015-04-15 21:03:45] [03]: 00000000:fc000000 -
00000001:00000000: RESERVED
(d36) [2015-04-15 21:03:45] [04]: 00000001:00000000 -
00000001:40000000: RAM
Linux reports:
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000003ffff000 (usable)
BIOS-e820: 000000003ffff000 - 0000000040000000 (reserved)
BIOS-e820: 00000000fc000000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000140000000 (usable)
Note: the e820 map is missing the VGA hole of 0xa0000 - 0xc0000.
So I see 16 pages < 1GiB below 4GiB and 1GiB above 4Gib
8 MiB of video RAM, which is real close to the 2056 KiB
of ram the guest was configured with.
If I drop all the e1000 nics (the QEMU option ROMs
that contain code that is executed by the guest to PXE boot
from SeaBIOS):
xc: detail: VIRTUAL MEMORY ARRANGEMENT:
xc: detail: Loader: 0000000000100000->00000000001c4a1c
xc: detail: Modules: 0000000000000000->0000000000000000
xc: detail: TOTAL: 0000000000000000->0000000080000000
xc: detail: ENTRY: 0000000000100038
xc: detail: PHYSICAL MEMORY ALLOCATION:
xc: detail: 4KB PAGES: 0x0000000000000200
xc: detail: 2MB PAGES: 0x00000000000001ff
xc: detail: 1GB PAGES: 0x0000000000000001
(d2) [2015-04-15 21:35:18] c0000-fffff: Main BIOS
(d2) [2015-04-15 21:35:18] E820 table:
(d2) [2015-04-15 21:35:18] [00]: 00000000:00000000 - 00000000:000a0000: RAM
(d2) [2015-04-15 21:35:18] HOLE: 00000000:000a0000 - 00000000:000c0000
(d2) [2015-04-15 21:35:18] [01]: 00000000:000c0000 - 00000000:00100000:
RESERVED
(d2) [2015-04-15 21:35:18] [02]: 00000000:00100000 - 00000000:40000000: RAM
(d2) [2015-04-15 21:35:18] HOLE: 00000000:40000000 - 00000000:fc000000
(d2) [2015-04-15 21:35:18] [03]: 00000000:fc000000 - 00000001:00000000:
RESERVED
(d2) [2015-04-15 21:35:18] [04]: 00000001:00000000 - 00000001:40000000: RAM
(d2) [2015-04-15 21:35:18] Invoking SeaBIOS ...
...
(d2) [2015-04-15 21:35:21] Returned 258048 bytes of ZoneHigh
(d2) [2015-04-15 21:35:21] e820 map has 7 items:
(d2) [2015-04-15 21:35:21] 0: 0000000000000000 - 000000000009fc00 = 1 RAM
(d2) [2015-04-15 21:35:21] 1: 000000000009fc00 - 00000000000a0000 = 2
RESERVED
(d2) [2015-04-15 21:35:21] 2: 00000000000f0000 - 0000000000100000 = 2
RESERVED
(d2) [2015-04-15 21:35:21] 3: 0000000000100000 - 000000003ffff000 = 1 RAM
(d2) [2015-04-15 21:35:21] 4: 000000003ffff000 - 0000000040000000 = 2
RESERVED
(d2) [2015-04-15 21:35:21] 5: 00000000fc000000 - 0000000100000000 = 2
RESERVED
(d2) [2015-04-15 21:35:21] 6: 0000000100000000 - 0000000140000000 = 1 RAM
And linux:
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000003ffff000 (usable)
BIOS-e820: 000000003ffff000 - 0000000040000000 (reserved)
BIOS-e820: 00000000fc000000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000140000000 (usable)
max_pages=526592 (2106368(0x202400) KiB) nr_pages=526336
(2105344(0x202000) KiB) free_pages=256
And with LIBXL_MAXMEM_CONSTANT=1024, the 256 "free (wasted?)"
pages are this.
>
> QEMU has no knowledge about what the e820 contains and considers the
> memory relocated from the PCI hole to the top of ram as valid guest
> memory. Libxl tells QEMU how big the PCI hole is by passing
> max-ram-below-4g to QEMU.
>
> I think that we should fix the guest e820 as it is actually valid guest
> memory.
>
If you are talking about the 1GiB over 4GiB in my example, it is already
in the e820 map.
So I do not understand the "wasted" ram issue.
-Don Slutz
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |