Re: [Xen-devel] Essay on an important Xen decision (long)

[Mark Williamson <mark.williamson@xxxxxxxxxxxx>]

Dan,

Thanks for the summary, it's nice to see all the arguments presented together.

> 3) virtual physical (VP): The guest is given max_pages of
>    contiguous physical memory starting at zero.  Xen provides
>    the illusion to the guest that this is machine memory;
>    any physical-to-machine translation required for functional
>    correctness is handled invisibly by Xen.  VP cannot be used
>    by guests that directly program DMA-based I/O devices
>    because a DMA device requires a machine address and, by
>    definition, the guest knows only about physical addresses.
>
> Xen/x86 and Xen/x86_64 use P2M, but switch to VP (aka "shadow
> mode") for an unprivileged guest when a migration is underway.
> Xen/ia64 currently uses P==M for domain0 and VP for unprivileged
> guests.  Xen/ppc intends to use VP only.

NB. the shadow mode for migration (logdirty) doesn't actually virtualise the 
physical <-> machine mapping - a paravirt guest on x86 always knows where all 
its pages are in machine memory.  All that's being hidden in this case is 
that the pagetables are being shadowed (so that pages can be transparently 
write protected).

> Driver domains are "coming soon" and support of driver domains is a
> "must", however support for hybrid driver domains (i.e. domains that
> utilize both backend and frontend drivers) is open to debate.  It can
> be assumed however that all driver domains will require DMA access.
>
> P2M should make driver domains easier to implement (once the initial
> Xenlinux/ia64 work is completed) and able to support a broader range
> of functionality.  P==M may disallow hybrid driver domains and
> create other restrictions, though some creative person may be able
> to solve these.

I'd think that driver domains themselves would be quite attractive on IA64 - 
for big boxes, it allows you to partition the hardware devices *and* 
potentially improve uptime by isolating driver faults.

For what you call "hybrid" domains, there are people using this for virtual 
DMZ functionality...  I guess it'd be nice to enable it.  Presumably the 
problem is that the backend does some sort of P-to-M translation itself?

Do you have a plan for how you would implement P==M driver domains?

Cheers,
Mark

> FUTURE XEN FEATURE SUPPORT
>
> None of the approaches have been "design-tested" significantly for
> support or compatibility with future Xen functionality such as
> oversubscription or machine-memory hot-plug, nor for exotic
> machine memory topologies such as NUMA or discontig (sparsely
> populated).  Such functionalities and topologies are much more
> likely to be encountered in high-end server architectures rather
> than widely-available PCs and low-end servers.  There is some
> debate as to whether the existing Xen memory architecture will easily
> evolve to accommodate these future changes or if more fundamental
> changes will be required.  Architectural decisions and restrictions
> should be made with these uncertainties in mind.
>
> Some believe that discovery and policy for machine memory will
> eventually need to move out of Xen into domain0, leaving only
> enforcement mechanism in Xen.  For example, oversubscription, NUMA
> or hot-plug memory support are likely to be fairly complicated
> and a commonly stated goal is to move unnecessary complexity out
> of Xen.  And the plethora of recent changes in Linux/ia64
> involving machine memory models indicates there are still many
> unknowns.  P==M more easily supports a model where domain0
> owns ALL of machine memory *except* a small amount reserved for
> and protected by Xen itself.  If this is all true, Xen/x86 may
> eventually need to move to a dom0 P==M model, in which case it
> would be silly for Xen/ia64 to move to P2M and then back to P==M.
>
> Others think these features will be easy to implement in Xen and,
> with minor changes, entirely compatible with P2M.  And that
> P2M is the once and future model for domain0.
>
> SUMMARY
>
> I'm sure there are more issues and tradeoffs that will come up
> in discussion, but let me summarize these:
>
> Move domain0 to P2M:
> + Fewer differences in Xen drivers between Xen/x86 and Xen/ia64
> + Fewer differences in Xen drivers between Xen/VT-x and Xen/VT-i
> + Easier to implement remaining Xen drivers for Xen/ia64
> - Major changes may require months for Xen/ia64 to regain stability
> - Many more changes to Xenlinux/ia64; more difficulty pushing upstream
> - No attempt to make Xen more resilient for future architectures
>
> Leave domain0 as P==M:
> + Fewer changes in Xenlinux; easier to push upstream
> + Making Xen more flexible is a good thing
> ? May provide better foundation for future features (oversubscr, NUMA)
> - More restrictions on driver domains
> - More hacks required for some Xen drivers, or
> - More work to better abstract and define a portable driver
>   architecture abstract
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel