Re: [Xen-devel] [PATCH v2 19/20] x86/mem_sharing: reset a fork

[Julien Grall <julien@xxxxxxx>]

Hi Tamas,

On 18/12/2019 19:40, Tamas K Lengyel wrote:
> Implement hypercall that allows a fork to shed all memory that got allocated
> for it during its execution and re-load its vCPU context from the parent VM.
> This allows the forked VM to reset into the same state the parent VM is in a
> faster way then creating a new fork would be. Measurements show about a 2x
> speedup during normal fuzzing operations. Performance may vary depending how
> much memory got allocated for the forked VM. If it has been completely
> deduplicated from the parent VM then creating a new fork would likely be more
> performant.
>
> Signed-off-by: Tamas K Lengyel <tamas.lengyel@xxxxxxxxx>
> ---
>   xen/arch/x86/mm/mem_sharing.c | 105 ++++++++++++++++++++++++++++++++++
>   xen/include/public/memory.h   |   1 +
>   2 files changed, 106 insertions(+)
>
> diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c
> index e93ad2ec5a..4735a334b9 100644
> --- a/xen/arch/x86/mm/mem_sharing.c
> +++ b/xen/arch/x86/mm/mem_sharing.c
> @@ -1622,6 +1622,87 @@ static int mem_sharing_fork(struct domain *d, struct 
> domain *cd)
>       return 0;
>   }
>   
> +struct gfn_free;
> +struct gfn_free {
> +    struct gfn_free *next;
> +    struct page_info *page;
> +    gfn_t gfn;
> +};
> +
> +static int mem_sharing_fork_reset(struct domain *d, struct domain *cd)
> +{
> +    int rc;
> +
> +    struct p2m_domain* p2m = p2m_get_hostp2m(cd);
> +    struct gfn_free *list = NULL;
> +    struct page_info *page;
> +
> +    page_list_for_each(page, &cd->page_list)
AFAICT, your domain is not paused, so it would be possible to have page 
added/remove in that list behind your back.
You also have multiple loop on the page_list in this function. Given the 
number of page_list can be quite big, this is a call for hogging the 
pCPU and an RCU lock on the domain vCPU running this call.
> +    {
> +        mfn_t mfn = page_to_mfn(page);
> +        if ( mfn_valid(mfn) )
> +        {
> +            p2m_type_t p2mt;
> +            p2m_access_t p2ma;
> +            gfn_t gfn = mfn_to_gfn(cd, mfn);
> +            mfn = __get_gfn_type_access(p2m, gfn_x(gfn), &p2mt, &p2ma,
> +                                        0, NULL, false);
> +            if ( p2m_is_ram(p2mt) )
> +            {
> +                struct gfn_free *gfn_free;
> +                if ( !get_page(page, cd) )
> +                    goto err_reset;
> +
> +                /*
> +                 * We can't free the page while iterating over the page_list
> +                 * so we build a separate list to loop over.
> +                 *
> +                 * We want to iterate over the page_list instead of checking
> +                 * gfn from 0 to max_gfn because this is ~10x faster.
> +                 */
> +                gfn_free = xmalloc(struct gfn_free);
If I did the math right, for a 4G guest this will require at ~24MB of 
memory. Actually, is it really necessary to do the allocation for a 
short period of time?
What are you trying to achieve by iterating twice on the GFN? Wouldn't 
it be easier to pause the domain?
> +                if ( !gfn_free )
> +                    goto err_reset;
> +
> +                gfn_free->gfn = gfn;
> +                gfn_free->page = page;
> +                gfn_free->next = list;
> +                list = gfn_free;
> +            }
> +        }
> +    }
> +
> +    while ( list )
> +    {
> +        struct gfn_free *next = list->next;
> +
> +        rc = p2m->set_entry(p2m, list->gfn, INVALID_MFN, PAGE_ORDER_4K,
> +                            p2m_invalid, p2m_access_rwx, -1);
> +        put_page_alloc_ref(list->page);
> +        put_page(list->page);
> +
> +        xfree(list);
> +        list = next;
> +
> +        ASSERT(!rc);
> +    }
> +
> +    if ( (rc = fork_hvm(d, cd)) )
> +        return rc;
> +
> + err_reset:
> +    while ( list )
> +    {
> +        struct gfn_free *next = list->next;
> +
> +        put_page(list->page);
> +        xfree(list);
> +        list = next;
> +    }
> +
> +    return 0;
> +}
> +
Cheers,

--
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel