Re: [Xen-devel] device-mmio emulation in Xen

[Tim Deegan <Tim.Deegan@xxxxxxxxxx>]

Hi, 

At 15:32 +0000 on 05 Jan (1262705557), Abhinav Srivastava wrote:
> I noticed that during a network copy operation Xen page faults a lot and 
> control goes to sh_page_fault function. When I printed some debugging info, 
> it showed me gmfn = -1. Then the execution goes through the regular path of 
> the page fault handler code, which means it creates an entry using 
> shadow_get_and_create_l1e, propagates it using l1e_propagate_from_guest, and 
> finally updates the entry using shadow_set_l1e. It finally goes into the 
> device-model mmio condition. In this condition, it extracts a guest physical 
> address and calls "goto mmio", which in turn calls handle_mmio function that 
> emulates the instruction.
> 
> However with the gmfn = -1 condition, the execution sometimes directly goto
> to handle_mmio function using the fast_fault_path with going through the 
> regular path. It seems like there are two possible execution paths, and I did 
> not understand which one is chosen when?
> 

/******************************************************************************
 * We implement a "fast path" for two special cases: faults that require
 * MMIO emulation, and faults where the guest PTE is not present.  We
 * record these as shadow l1 entries that have reserved bits set in
 * them, so we can spot them immediately in the fault handler and handle
 * them without needing to hold the shadow lock or walk the guest
 * pagetables.
 *
 * This is only feasible for PAE and 64bit Xen: 32-bit non-PAE PTEs don't
 * have reserved bits that we can use for this.
 */

> I have some questions related to this behavior:
> 
> 1. Why are there so many faults duing network copy operation?

The faults are a HVM guest doing MMIO operations to control its emulated
network card.

> 2. What does gmfn = -1 signify? Is it reserved for mmio addresses?

It's INVALID_MFN.  It means there is no memory mapped at the address the
guest accessed.  Xen treats that as MMIO, though there's a plan to have
MMIO regions explicitly registered instead.

> 3. How does Xen handle this gmfn = -1? It seems like on the regular path
> it still creates, propagates, and updates entries for gmfn = -1. How does Xen 
> handles this at the shadow page table level?

Yes, it creates a deliberately invalid shadow PTE; then in the pagefault
handler it can use the pagefault error code and the contents of the PTE 
to skip the bulk of the shadow fault handling and go straight to handle_mmio.

> 4. What are these two code execution paths, and when does Xen decide which
> path to choose?
> 
> 5. Finally, is there anyway these faults can be reduced?

Yes, use PV drivers in the guest.  That eliminates the whole
emulated-MMIO system, and all the pagefaults associated with it. 

Cheers,

Tim.

> I would very appreciate any help in this regard.
> 
> Thanks,
> Abhinav
> 
> 
> 
>       The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. 
> http://in.yahoo.com/
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

-- 
Tim Deegan <Tim.Deegan@xxxxxxxxxx>
Principal Software Engineer, Citrix Systems (R&D) Ltd.
[Company #02300071, SL9 0DZ, UK.]

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel