[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Design session report: Live-Updating Xen


  • To: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
  • From: Jan Beulich <JBeulich@xxxxxxxx>
  • Date: Wed, 17 Jul 2019 07:09:55 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=suse.com;dmarc=pass action=none header.from=suse.com;dkim=pass header.d=suse.com;arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OTlrE4K7kTVJfMAX5Di3gU2IrMca0jp+2OsedzJl3bI=; b=Nt7T8XPCETREInaNUiAFqGAfOaIwH3MvGSBlPm9PKa1eoPzbAiahpoguAy2wvALvOz6LgOXuaqajHXjnPLMo1h2CXDTMzdjYr584E+RYZbl6OwxX8DB03ewB2bOa1FiUrfUVsblJpauAFyVPdy9UPCYmHfQVKhlfngAnWUdGNi9XfRQZHjGwekYi+MZcYtIVvXPki+Gu0rtMujZGOL7IF6DfhppKS3q07QbXAGWjTuUI9kR3/HUCI0iszeTjRWsBgXmGVFOdW0FdrTKv5tEzyiHk+9EtVF5sIv1nqR/FAyK7PKQ0FvGQmbZty8wuOil6pUcNJYwRoyGGeUJANmyJ4g==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GkoK6K+x9IApmS8eih32dIrqXsQGCB9IUMRKNLGovqZBQ1k2luOs9RrsYyO0HhGdstDzPWCTt+sXJ/W2voXf+sWgIbs5lFb/UzjzlpbNGfX3z2mpnX0imGdwxnOSJ4WEWnDAkipYQU1oPJzNUud3wdFM6SKSlepzU7srUHYu8EqFs7JGT8swLHyXCzXkjgJEQ5QJd2h9CnOpifOsqTPGxgJxih/TiyLem+WDaCQFsdyoSUocQgYzFIOJR72qp0RSbEqRu5Gy12KWUNh++NBKmpiaCettrGUU5mezJEh266fcozIRz/8c/O5Uv0Rfv/Si1lVkl+QPRdNNazXoJ4JosQ==
  • Authentication-results: spf=none (sender IP is ) smtp.mailfrom=JBeulich@xxxxxxxx;
  • Cc: "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Leonard Foerster <foersleo@xxxxxxxxxx>
  • Delivery-date: Wed, 17 Jul 2019 07:10:49 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Thread-index: AQHVOz84rrbZg3jr7UWo9Co44cwhi6bN7RdMgAB5rAA=
  • Thread-topic: [Xen-devel] Design session report: Live-Updating Xen

On 17.07.2019 01:51, Andrew Cooper wrote:
> On 15/07/2019 19:57, Foerster, Leonard wrote:
>>      * dom0less: bootstrap domains without the involvement of dom0
>>              -> this might come in handy to at least setup and continue dom0 
>> on target xen
>>              -> If we have this this might also enable us to de-serialize 
>> the state for
>>                      other guest-domains in xen and not have to wait for 
>> dom0 to do this
> 
> Reconstruction of dom0 is something which Xen will definitely need to
> do.  With the memory still in place, its just a fairly small of register
> state which needs restoring.
> 
> That said, reconstruction of the typerefs will be an issue.  Walking
> over a fully populated L4 tree can (in theory) take minutes, and its not
> safe to just start executing without reconstruction.
> 
> Depending on how bad it is in practice, one option might be to do a
> demand validate of %rip and %rsp, along with a hybrid shadow mode which
> turns faults into typerefs, which would allow the gross cost of
> revalidation to be amortised while the vcpus were executing.  We would
> definitely want some kind of logic to aggressively typeref outstanding
> pagetables so the shadow mode could be turned off.

Neither walking the page table trees nor and on-demand re-creation can
possibly work, as pointed out during (partly informal) discussion: At
the very least the allocated and pinned states of pages can only be
transferred. Hence we seem to have come to agreement that struct
page_info instances have to be transformed (in place if possible, i.e.
when the sizes match, otherwise by copying).
>>              -> We might have to go and re-inject certain interrupts
> 
> What hardware are you targeting here?  IvyBridge and later has a posted
> interrupt descriptor which can accumulate pending interrupts (at least
> manually), and newer versions (Broadwell?) can accumulate interrupts
> directly from hardware.

For HVM/PVH perhaps that's good enough. What about PV though?

>> A key cornerstone for Live-update is guest transparent live migration
>>      -> This means we are using a well defined ABI for saving/restoring 
>> domain state
>>              -> We do only rely on domain state and no internal xen state
> 
> Absolutely.  One issue I discussed with David a while ago is that even
> across an upgrade of Xen, the format of the EPT/NPT pagetables might
> change, at least in terms of the layout of software bits.  (Especially
> for EPT where we slowly lose software bits to new hardware features we
> wish to use.)

Right, and therefore a similar transformation like for struct page_info
may be unavoidable here too.

Re-using large data structures (or arrays thereof) may also turn out
useful in terms of latency until the new Xen actually becomes ready to
resume.

Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.