[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v2 2/5] xen/arm32: mm: Consolidate the domheap mappings initialization


  • To: Julien Grall <julien@xxxxxxx>
  • From: Bertrand Marquis <Bertrand.Marquis@xxxxxxx>
  • Date: Thu, 21 Jul 2022 08:36:09 +0000
  • Accept-language: en-GB, en-US
  • Arc-authentication-results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com])
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
  • Arc-message-signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DuZ4M4gHTq91mRmAjaPbTMmhEJh5TIjlu09hJWHueSA=; b=WyKHJpELvJ1Z1q9bifoMu+X4F9yXFXInCU5hkDEx9zMBV8yQa6IKwxvtx8hQKK34ZpkYk5cArGjYrebL8F5e/952pWyoBIrzB+26ChfzTODS1WWYcVYbmUUdiSOGdpmMT+kRU4VT8IQ2lfAp2VsNRokpd6+MsdPxoseWOWEPpJuhNF45bxvRl64nbP5PEdaDoiAVCX5GLukkc/36zN5lAHt6gJjnQ8yWjLSiz+MSxeShXhsYyRLSFJ/HAk0+/Zfa3u3OqiKdp/WVOUV4zbaKajiK1gFfEk1AV5+iKKxzT12p22jF+qO1SROcqGNB9nNXskSz+BZrKsLIpWfPrFtkfA==
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DuZ4M4gHTq91mRmAjaPbTMmhEJh5TIjlu09hJWHueSA=; b=IdLujCt1qO87yvH125j1vNAbc9KyzSJqBvSdM+szGb55WJpYB4XdB44+90k9e3Uhv7BP9B73JU88tYhRf2D1dVcEVaGKPzQHT6YP6C6028u5nIzGDtY9VA2NY/dXIAKgWFdIiepZ8SgwRhU8aF946Enqkvi2HF4FOCyb2QzjO3QjwGHRXGEOwRoAyb3n7dNPif1dpYSbjYTWUgWV+PNvCQYBflDs05aO+IMBpBscE7zL/R2NPJPj1ML693ruBpBwfignCXmHfRWxMw4JbguVcKZvOICFvf83doX1oIs5Ejb+qt9WTqSfNPCycMWCty1TMgd4v7NHv4vQOiQtfNGPOQ==
  • Arc-seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=Sn2jFNUogXpPLaeNHTr1z8mODOk7KeGz0y5x8t/bZ3gabGFvXjQYu8sSVVZHyRLKJcsEbFWsJP1hKVFpFzjmY/mM+2Rqyhgnw1kJtMeuOMbQJTJWiXEMmtCWN7bfPI3ncTXt0NiiljGyPZnZK8x+BproW6WJB7aLrOIILNT1TRhYDEuvh6K/XzTyfokbIGuyUjW9DsIq5G8jet8pGh6I/JziF3g2mdx6ae49YmEjFHVeb10Us/VHDQ7gR5bzyUCbZUJkBuYMX9q9Dgbn5HZC5tLo/oMBmiPCsGrqr14lnr/YOj+Ud8e7FcTcWtsJZKPszmH1RgIrypheD0HN5xxCxA==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=H0CPn37bhR6uVKromApbS4n90VEcc4r8ubdGsrU95SvLfLtC2D3PcjRTW5+u7VxYUTsnpYa/iTFK8x4AbiSeAY5Tzm0ZUn+PY+yqcNV5gwe1ojqASUKaZITSmwXTYN+3AZ4OdkLUrpJa9b7CS9s5kl5FfRhx2ZeoKoKcWdKk7ny4xg/WLrOmh7zRO2wpy3nSmRS+Zx/4BSLJMvCh5g1tEElZ6LNX40KUjmTqxCDJ2yKfBZfFCxnUlYBNVxsHCWUW8BOKq0EQuqCe8M+V/eCTeyyVOxolQXdZxWFzuSoa3S/L8pV8b8rTb++hfP419GVyXBhwGMi4T4GlhyNSnA9T/Q==
  • Authentication-results-original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
  • Cc: xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Julien Grall <jgrall@xxxxxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>
  • Delivery-date: Thu, 21 Jul 2022 08:36:31 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Nodisclaimer: true
  • Original-authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
  • Thread-index: AQHYnGjbgtq6TqhSCkikJaFgpVJiDq2IgQiA
  • Thread-topic: [PATCH v2 2/5] xen/arm32: mm: Consolidate the domheap mappings initialization

Hi Julien,

> On 20 Jul 2022, at 19:44, Julien Grall <julien@xxxxxxx> wrote:
> 
> From: Julien Grall <jgrall@xxxxxxxxxx>
> 
> At the moment, the domheap mappings initialization is done separately for
> the boot CPU and secondary CPUs. The main difference is for the former
> the pages are part of Xen binary whilst for the latter they are
> dynamically allocated.
> 
> It would be good to have a single helper so it is easier to rework
> on the domheap is initialized.
> 
> For CPU0, we still need to use pre-allocated pages because the
> allocators may use domain_map_page(), so we need to have the domheap
> area ready first. But we can still delay the initialization to setup_mm().
> 
> Introduce a new helper init_domheap_mappings() that will be called
> from setup_mm() for the boot CPU and from init_secondary_pagetables()
> for secondary CPUs.
> 
> Signed-off-by: Julien Grall <jgrall@xxxxxxxxxx>

With a small typo mentioned after which can be fixed on commit:

Reviewed-by: Bertrand Marquis <bertrand.marquis@xxxxxxx>

> 
> ----
>    Changes in v2:
>        - Fix function name in the commit message
>        - Remove duplicated 'been' in the comment
> ---
> xen/arch/arm/include/asm/arm32/mm.h |  2 +
> xen/arch/arm/mm.c                   | 92 +++++++++++++++++++----------
> xen/arch/arm/setup.c                |  8 +++
> 3 files changed, 71 insertions(+), 31 deletions(-)
> 
> diff --git a/xen/arch/arm/include/asm/arm32/mm.h 
> b/xen/arch/arm/include/asm/arm32/mm.h
> index 6b039d9ceaa2..575373aeb985 100644
> --- a/xen/arch/arm/include/asm/arm32/mm.h
> +++ b/xen/arch/arm/include/asm/arm32/mm.h
> @@ -10,6 +10,8 @@ static inline bool arch_mfns_in_directmap(unsigned long 
> mfn, unsigned long nr)
>     return false;
> }
> 
> +bool init_domheap_mappings(unsigned int cpu);
> +
> #endif /* __ARM_ARM32_MM_H__ */
> 
> /*
> diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
> index 0177bc6b34d2..9311f3530066 100644
> --- a/xen/arch/arm/mm.c
> +++ b/xen/arch/arm/mm.c
> @@ -372,6 +372,58 @@ void clear_fixmap(unsigned int map)
> }
> 
> #ifdef CONFIG_DOMAIN_PAGE
> +/*
> + * Prepare the area that will be used to map domheap pages. They are
> + * mapped in 2MB chunks, so we need to allocate the page-tables up to
> + * the 2nd level.
> + *
> + * The caller should make sure the root page-table for @cpu has been
> + * allocated.
> + */
> +bool init_domheap_mappings(unsigned int cpu)
> +{
> +    unsigned int order = get_order_from_pages(DOMHEAP_SECOND_PAGES);
> +    lpae_t *root = per_cpu(xen_pgtable, cpu);
> +    unsigned int i, first_idx;
> +    lpae_t *domheap;
> +    mfn_t mfn;
> +
> +    ASSERT(root);
> +    ASSERT(!per_cpu(xen_dommap, cpu));
> +
> +    /*
> +     * The domheap for cpu0 is before the heap is initialized. So we

There is a “initialized” missing.

Cheers
Bertrand

> +     * need to use pre-allocated pages.
> +     */
> +    if ( !cpu )
> +        domheap = cpu0_dommap;
> +    else
> +        domheap = alloc_xenheap_pages(order, 0);
> +
> +    if ( !domheap )
> +        return false;
> +
> +    /* Ensure the domheap has no stray mappings */
> +    memset(domheap, 0, DOMHEAP_SECOND_PAGES * PAGE_SIZE);
> +
> +    /*
> +     * Update the first level mapping to reference the local CPUs
> +     * domheap mapping pages.
> +     */
> +    mfn = virt_to_mfn(domheap);
> +    first_idx = first_table_offset(DOMHEAP_VIRT_START);
> +    for ( i = 0; i < DOMHEAP_SECOND_PAGES; i++ )
> +    {
> +        lpae_t pte = mfn_to_xen_entry(mfn_add(mfn, i), MT_NORMAL);
> +        pte.pt.table = 1;
> +        write_pte(&root[first_idx + i], pte);
> +    }
> +
> +    per_cpu(xen_dommap, cpu) = domheap;
> +
> +    return true;
> +}
> +
> void *map_domain_page_global(mfn_t mfn)
> {
>     return vmap(&mfn, 1);
> @@ -633,16 +685,6 @@ void __init setup_pagetables(unsigned long 
> boot_phys_offset)
>         p[i].pt.xn = 0;
>     }
> 
> -#ifdef CONFIG_ARM_32
> -    for ( i = 0; i < DOMHEAP_SECOND_PAGES; i++ )
> -    {
> -        p[first_table_offset(DOMHEAP_VIRT_START+i*FIRST_SIZE)]
> -            = pte_of_xenaddr((uintptr_t)(cpu0_dommap +
> -                                         i * XEN_PT_LPAE_ENTRIES));
> -        p[first_table_offset(DOMHEAP_VIRT_START+i*FIRST_SIZE)].pt.table = 1;
> -    }
> -#endif
> -
>     /* Break up the Xen mapping into 4k pages and protect them separately. */
>     for ( i = 0; i < XEN_PT_LPAE_ENTRIES; i++ )
>     {
> @@ -686,7 +728,6 @@ void __init setup_pagetables(unsigned long 
> boot_phys_offset)
> 
> #ifdef CONFIG_ARM_32
>     per_cpu(xen_pgtable, 0) = cpu0_pgtable;
> -    per_cpu(xen_dommap, 0) = cpu0_dommap;
> #endif
> }
> 
> @@ -719,39 +760,28 @@ int init_secondary_pagetables(int cpu)
> #else
> int init_secondary_pagetables(int cpu)
> {
> -    lpae_t *first, *domheap, pte;
> -    int i;
> +    lpae_t *first;
> 
>     first = alloc_xenheap_page(); /* root == first level on 32-bit 3-level 
> trie */
> -    domheap = 
> alloc_xenheap_pages(get_order_from_pages(DOMHEAP_SECOND_PAGES), 0);
> 
> -    if ( domheap == NULL || first == NULL )
> +    if ( !first )
>     {
> -        printk("Not enough free memory for secondary CPU%d pagetables\n", 
> cpu);
> -        free_xenheap_pages(domheap, 
> get_order_from_pages(DOMHEAP_SECOND_PAGES));
> -        free_xenheap_page(first);
> +        printk("CPU%u: Unable to allocate the first page-table\n", cpu);
>         return -ENOMEM;
>     }
> 
>     /* Initialise root pagetable from root of boot tables */
>     memcpy(first, cpu0_pgtable, PAGE_SIZE);
> +    per_cpu(xen_pgtable, cpu) = first;
> 
> -    /* Ensure the domheap has no stray mappings */
> -    memset(domheap, 0, DOMHEAP_SECOND_PAGES*PAGE_SIZE);
> -
> -    /* Update the first level mapping to reference the local CPUs
> -     * domheap mapping pages. */
> -    for ( i = 0; i < DOMHEAP_SECOND_PAGES; i++ )
> +    if ( !init_domheap_mappings(cpu) )
>     {
> -        pte = mfn_to_xen_entry(virt_to_mfn(domheap + i * 
> XEN_PT_LPAE_ENTRIES),
> -                               MT_NORMAL);
> -        pte.pt.table = 1;
> -        
> write_pte(&first[first_table_offset(DOMHEAP_VIRT_START+i*FIRST_SIZE)], pte);
> +        printk("CPU%u: Unable to prepare the domheap page-tables\n", cpu);
> +        per_cpu(xen_pgtable, cpu) = NULL;
> +        free_xenheap_page(first);
> +        return -ENOMEM;
>     }
> 
> -    per_cpu(xen_pgtable, cpu) = first;
> -    per_cpu(xen_dommap, cpu) = domheap;
> -
>     clear_boot_pagetables();
> 
>     /* Set init_ttbr for this CPU coming up */
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 85ff956ec2e3..068e84b10335 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -783,6 +783,14 @@ static void __init setup_mm(void)
>     setup_frametable_mappings(ram_start, ram_end);
>     max_page = PFN_DOWN(ram_end);
> 
> +    /*
> +     * The allocators may need to use map_domain_page() (such as for
> +     * scrubbing pages). So we need to prepare the domheap area first.
> +     */
> +    if ( !init_domheap_mappings(smp_processor_id()) )
> +        panic("CPU%u: Unable to prepare the domheap page-tables\n",
> +              smp_processor_id());
> +
>     /* Add xenheap memory that was not already added to the boot allocator. */
>     init_xenheap_pages(mfn_to_maddr(xenheap_mfn_start),
>                        mfn_to_maddr(xenheap_mfn_end));
> -- 
> 2.32.0
> 


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.