Re: [Xen-devel] Shadow mode

[Steven Hand <Steven.Hand@xxxxxxxxxxxx>]

>I have seen many references in the code to shadow mode. It seems to be
>related to sharing pages among guests. Could some explain (briefly) what
>it is, and how it works?

"Shadow mode" is a mode of operation in which guest operating systems 
write and read directly from 'shadow' rather than 'real' page tables. 
This is the mode of operation used by e.g. vmware and has the advantage 
that the guest operating system doesn't need to be modified to understand
that physical memory is sparse. In particular, the guest operating system
can in principle believe that it owns physical memory from 0 to e.g. 32Mb 
while in practice it has some set of discontiguous physical pages drawn 
from anywhere in the real e.g. 2GB of memory. 

It operates by having Xen transparently translate from the guest page 
tables to the real page tables -- whenever a guest operating system 
modifies a page table, the access traps to Xen which then makes the 
corresponding update to the real page table (after doing the relevant
validity checks, etc). From time to time, Xen propagates the accessed
and modified bits from the real page tables back to the guest's private
page tables to that reads from the guest work ok. Note that this has
nothing to do with sharing pages among guests (there is support for 
this in Xen, but it is completely unrelated to shadow mode). 

Shadow page tables provide a level of indirection and, in particular, 
allow Xen to change the physical pages assigned to the guest without
the guest's knowledge. This is particularly useful for applications
like virtual machine migration (moving a running guest operating 
system from one physical machine to another). 

On the down sides: shadow mode involves an extra copy of every page 
table that the guest uses (since there is the private and the 'real'
copy) and so doubles the memory usage for these, and increases the 
cache footprint etc. It also involves (in the simple case) a trap on 
every write to a page table -- you lose the benefits of batching which
you get with the default paravirtualized interface. Plus you end up
lying to the guest which means it may make poor decisions about physical
memory allocation (e.g. thinking it's chosen pages not to alias in a 
cache but getting it wrong). And there's some additional complexity too. 

Shadow mode is work in progress; it works to a certain extent but has
not been widely tested. We intend to do a thorough evaluation of shadow
mode and see how much it costs overall. Not really something for anyone
but the serious hacker out there to play with ... but if you are keen,
have a go! Just don't expect too much hand holding at this stage :-) 

cheers,

S.