Re: [Xen-devel] PML (Page Modification Logging) design for Xen

["Tian, Kevin" <kevin.tian@xxxxxxxxx>]

> From: Kai Huang [mailto:kai.huang@xxxxxxxxxxxxxxx]
> Sent: Thursday, February 12, 2015 2:46 PM
> 
> On 02/12/2015 02:25 PM, Tian, Kevin wrote:
> >> From: Kai Huang [mailto:kai.huang@xxxxxxxxxxxxxxx]
> >> Sent: Thursday, February 12, 2015 10:35 AM
> >>
> >> On 02/11/2015 09:13 PM, Jan Beulich wrote:
> >>>>>> On 11.02.15 at 12:52, <andrew.cooper3@xxxxxxxxxx> wrote:
> >>>> On 11/02/15 08:28, Kai Huang wrote:
> >>>>> With PML, we don't have to use write protection but just clear D-bit
> >>>>> of EPT entry of guest memory to do dirty logging, with an additional
> >>>>> PML buffer full VMEXIT for 512 dirty GPAs. Theoretically, this can
> >>>>> reduce hypervisor overhead when guest is in dirty logging mode, and
> >>>>> therefore more CPU cycles can be allocated to guest, so it's expected
> >>>>> benchmarks in guest will have better performance comparing to
> >> non-PML.
> >>>> One issue with basic EPT A/D tracking was the scan of the EPT tables.
> >>>> Here, hardware will give us a list of affected gfns, but how is Xen
> >>>> supposed to efficiently clear the dirty bits again?  Using EPT
> >>>> misconfiguration is no better than the existing fault path.
> >>> Why not? The misconfiguration exit ought to clear the D bit for all
> >>> 511 entries in the L1 table (and set it for the one entry that is
> >>> currently serving the access). All further D bit handling will then
> >>> be PML based.
> >> Indeed, we clear D-bit in EPT misconfiguration. In my understanding, the
> >> sequences are as follows:
> >>
> >> 1) PML enabled for the domain.
> >> 2) ept_invalidate_emt (or ept_invalidate_emt_range) is called.
> >> 3) Guest accesses specific GPA (which has been invalidated by step 2),
> >> and EPT misconfig is triggered.
> >> 4) Then resolve_misconfig is called, which fixes up GFN (above GPA >>
> >> 12) to p2m_ram_logdirty, and calls ept_p2m_type_to_flags, in which we
> >> clear D-bit of EPT entry (instead of clear W-bit) if p2m type is
> >> p2m_ram_logdirty. Then dirty logging of this GFN will be handled by PML.
> >>
> >> The above 2) ~ 4) will be repeated when log-dirty radix tree is cleared.
> > is ept_invalidate_emt required by existing logdirty mode or by PML enable?
> It's in existing logdirty code.
> > can we clear D bits directly when log-dirty radix tree is cleared to reduce
> > EPT misconfig exits for repeatedly dirtied pages?
> Theoretically we can, and looks logdirty for video ram is done in this
> way (logdirty for the page is re-enabled while it is reported to
> dirty_bitmap). One thing is looks video ram logdirty only exists for HAP
> mode.
> But in current log dirty implementation for global logdirty, at common
> paging layer, the log-dirty radix tree is cleaned in single step after
> reporting all dirty pages to userspace. And it just calls
> ept_invalidate_emt essentially. Therefore we need to modify logdirty
> common code at paging layer to achieve this, which is more like logdirty
> enhancement but not related to PML enabling directly. And any change of
> interface in paging layer requires modification in shadow mode
> accordingly, so currently I just choose not to do it.
> 

for general log dirty, ept_invalidate_emt is required because there is 
access permission change (dirtied page becomes rw after 1st fault,
so need to change them back to ro again for the new dirty tracking
round). But for PML, there's no permission change at all (always rw),
so such behavior should be noted by general logdirty layer for better
optimization. I'm OK not doing so for initial enabling patch, but it's
something you can think about later. :-)

Thanks
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel