Re: [Xen-devel] [VMI] Possible race-condition in altp2m APIs

[Tamas K Lengyel <tamas@xxxxxxxxxxxxx>]

> And some questions.
>
> 1) I'm guessing the drakvuf_inject_trap(drakvuf, 0x293e6a0, 0) call is 
> specific to the exact windows kernel in use?
>
> 2) In vmi_init(), what is the purpose of fmask and zero_page_gfn?  You add 
> one extra gfn to the guest, called zero_page, and fill it with 1's from fmask.
>
> 3) You create two altp2m's, but both have the same default access.  Is this 
> deliberate, or a bug?  If deliberate, why?
>
> Finally, and probably the source of the memory corruption...
>
> 4) When injecting a trap, you allocate a new gfn, memcpy() the contents and 
> insert a 0xcc (so far so good).  You then remap the executable view to point 
> at the new gfn with a breakpoint in (fine), and remap the readable view to 
> point at the zero_page, which is full of 1's (uh-oh).
>
> What is this final step trying to achieve?  It guarantees that patch-guard 
> will eventually notice and BSOD your VM for critical structure corruption.  
> The read-only view needs to point to the original gfn with only read 
> permissions, so when Windows reads the gfn back, it sees what it expects.  
> You also need to prohibit writes to either gfn so you can spot writes 
> (unlikely in this case but important for general introspection) so you can 
> propagate the change to both copies.

I can answer these questions based on how I've set it up in DRAKVUF
(haven't checked Mathiue's implementation and I won't be able for a
while as I'm traveling at the moment). The reason we have multiple EPT
views is to make it all multi-vCPU safe. Swapping the views around can
be done per-vCPU and we don't have to remove/reapply breakpoints all
the time and pause the entire domain while doing so.

There are three views being used: the default (hostp2m); the
execute-view which is active by default and has the remapped
shadow-copies of the pages with breakpoints injected at the very end
of the guests' physmap; and the read-only view that is only used when
someone is trying to read the actual address of a shadow-copy at the
end of the physmap (ie. not via the remapped gfn).

The read-only view has all shadow-copy gfn's remapped to that one page
full of 1's, because if you read random large gfn's in the guests'
memory space that's what Xen's emulator returns. It is called
zero_page because I originally guessed that those pages should be all
0 but it turned out I was wrong. Just haven't change the name of it
yet. This page is there because we want to avoid someone being able to
find out that there are shadow pages present. It would be quite
obvious something is "odd" when you can find copies of the Windows
kernel memory pages at the end of the memory space. So the shadow
pages' real GFN mem_access is restricted in the execute view, which
allows us to switch to the read-only view with MTF enabled and then
back afterwards. That way the shadow pages are not visible to the
guest, if someone tries to read them they return the same value and
behave the same as all other unmapped gfn's in that memory region.

So since the read-only view with all the 1's is rarely used, let's
talk about why patchguard can't notice changes to the kernel: the
execute-view has all pages that were breakpointed remapped and marked
execute-only. When patchguard tries to read these pages, the view is
swapped back to the hostp2m with MTF enabled. Then in the MTF callback
the view is swapped back to the execute-view. This means that
patchguard only ever reads the original page from the hostp2m. If the
page is being written to, the same dance happens with the addition of
the whole page being re-copied to the shadow location and the
breakpoints being reapplied on the shadow copy. This copy happens
while the whole domain is paused to avoid race-condition.

I hope this makes sense.

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel