RE: [Xen-devel] [Question] How to support page offline in Xen environment

["Jiang, Yunhong" <yunhong.jiang@xxxxxxxxx>]

Tim Deegan <mailto:Tim.Deegan@xxxxxxxxxx> wrote:
> Hi,
> 
> At 11:24 +0800 on 02 Dec (1228217091), Jiang, Yunhong wrote:
>>      We are considering utilize the live migration mechanism to
>> achieve the two steps. The user space tools will firstly mark page
>> offline_pending through hypercall, this hyercall will also return the
>> owners of the pages. secondly, if all pages can be offlined, user
>> space tools should live migrate the domains owning those
>> pages.
> 
> What do you mean by "live migrate" here?  Presuambly you can do
> something a lot more lightweight, just pausing the guest for a
> checkpoint, changing one or two p2m entries, and letting it resume.

Thanks for your reply very much. Yes, we do consider this before (it is in fac 
the "other option" in my mail) and it is ok for HVM domain without any foreign 
mapping, however, considering following situation:

a) A page is foreign mapped by another guest (e.g. dom0), change p2m entries is 
not enough.
b) A page is assigned to a domain with device assigned, we can't simply change 
the p2m entry because of DMA operation may on-going. (this in fact can't 
resolve cleanly through live migration, although the tools do hot remove in 
advance).
c) If a page is used like shadow page table or, virutal local apic's page, 
currently we can't simply exchange these pages.
d) For PV guest, can this be done without co-operation from guest? (Or we need 
change the paging mode to shadow before page offline?)

Of course, we can simplify the request, for example, no support for page in 
item a), b) and c) and that will be ok. 
That's the reason we hope to get suggestion on next step.

> 
>>      Following hypercall will be added:
>>      int xen_page_offline_pending(int start_pfn,  int
> end_pfn, void *result, void *owners)
>>      IN: start_pfn/end_pfn:
>>              the range of pages to be offlined.
>>      OUT: result:
>>              A buffer contain the page status for each page, it can be:
>>                      offlined: the page is offlined already
> (e.g. the page is already freed when the hypercall happen)
>>                      offline_pending: the page will be
> offline when freed
>>                      offline_fail: The page can't be
> offline, may because it is used by xen/dom0. Notice is, if any
> page is marked offline_fail, this hypercall will not change
> any page's status (i.e. no page will be marked offline_pending
> or offlined) to make sure atomic operation.
>>                      other status: Other status to be
> defined in future.
> 
> Should this buffer be IN/OUT?  The caller has to allocate it anyway and
> it would give a more general interface than the start/end arguments.
> 
>>      OUT: owners:
>>              A buffer contains the domains owning of the
> pages. Because of security consideration, it will not state
> which domain owning which page.
> 
> Why not?  Presumably the caller needs to have privilege over all those
> domains anyway in order to mark their frames pending-offline.

Hmm, that depends on how we define the privilege, and yes it is ture for dom0 
now.

> 
>>      Need notice is, issue exists for the live migrate mechanism:
>>      a) the domain ID will be changed after live migrate
>>      b) live migration will fail for a domain with device
> assigned, so user space tools have to hot remove the device,
> or fail the page offline requirement
> 
> Both of those issues go away if you don't use a full migrate.
> 
> Cheers,
> 
> Tim.
> 
> --
> Tim Deegan <Tim.Deegan@xxxxxxxxxx>
> Principal Software Engineer, Citrix Systems (R&D) Ltd.
> [Company #02300071, SL9 0DZ, UK.]
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel