Re: [Xen-devel] Converting heap page_infos to contiguous virtual

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 14/07/16 13:42, Julien Grall wrote:
> Hi,
>
> On 14/07/16 11:34, Andrew Cooper wrote:
>> On 14/07/16 11:25, George Dunlap wrote:
>>> On 13/07/16 21:57, Boris Ostrovsky wrote:
>>>> On 07/13/2016 04:34 PM, Andrew Cooper wrote:
>>>>> On 13/07/2016 21:17, Boris Ostrovsky wrote:
>>>>>> On 07/13/2016 04:02 PM, Andrew Cooper wrote:
>>>>>>> On 13/07/16 20:44, Boris Ostrovsky wrote:
>>>>>>>> I would like to clear a bunch of Xen heap pages at once (i.e. not
>>>>>>>> page-by-page).
>>>>>>>>
>>>>>>>> Greatly simplifying things, let's say I grab (in
>>>>>>>> common/page_alloc.c)
>>>>>>>>      pg = page_list_remove_head(&heap(node, zone, order)
>>>>>>>>
>>>>>>>> and then
>>>>>>>>
>>>>>>>>      mfn_t mfn =
>>>>>>>> _mfn(page_to_mfn(pg));
>>>>>>>>      char *va = mfn_to_virt(mfn_x(mfn));
>>>>>>>>      memset(va, 0, 4096 * (1 << order));
>>>>>>>>
>>>>>>>>
>>>>>>>> Would it be valid to this?
>>>>>>> In principle, yes.  The frame_table is in order.
>>>>>>>
>>>>>>> However, mfn_to_virt() will blow up for RAM above the 5TB
>>>>>>> boundary.  You
>>>>>>> need to map_domain_page() to get a mapping.
>>>>>> Right, but that would mean going page-by-page, which I want to
>>>>>> avoid.
>>>>>>
>>>>>> Now, DIRECTMAP_SIZE is ~128TB (if my math is correct) --- doesn't it
>>>>>> imply that it maps this big a range contiguously (modulo PDX hole)?
>>>>> Your maths is correct, and yet you will end up with problems if you
>>>>> trust it.
>>>>>
>>>>> That is the magic mode for the idle and monitor pagetables.  In the
>>>>> context of a 64bit PV guest, the cutoff is at 5TB, at which point you
>>>>> venture into the virtual address space reserved for guest kernel use.
>>>>> (It is rather depressing that the 64bit PV guest ABI is the factor
>>>>> limiting Xen's maximum RAM usage.)
>>>> I don't know whether it would make any difference but the pages
>>>> that I am
>>>> talking about are not in use by any guest, they are free. (This
>>>> question
>>>> is for scrubbing rewrite that I am working on. Which apparently you
>>>> figured out judged by what you are saying below)
>>> Is this start-of-day scrubbing (when there are no guests), or scrubbing
>>> on guest destruction?
>>>
>>> If the former, it seems like it might not be too difficult to arrange
>>> that we're in a context that has all the RAM mapped.
>>
>> This will be runtime scrubbing of pages.  This topic has come up at
>> several hackathons.
>
> Is it a feature that will be implemented in common code? If so, bear
> in mind that ARM 32-bit hypervisor does not have all the memory mapped
> (actually only Xen heap is mapped).

Nor does x86, on boxes with more than 5TB of RAM.

Where possible, we should try to make it common, but it will depend on
exactly how similar the heap implementations are.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel