Re: [Xen-devel] [PATCH V2 6/6] iommu/arm: Add Renesas IPMMU-VMSA support

[Oleksandr <olekstysh@xxxxxxxxx>]

Hi, Shimoda-san.

Thank you for the review.


> > From: Oleksandr Tyshchenko, Sent: Saturday, August 3, 2019 1:40 AM
> >
> > From: Oleksandr Tyshchenko <oleksandr_tyshchenko@xxxxxxxx>
> >
> > The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU)
> > which provides address translation and access protection functionalities
> > to processing units and interconnect networks.
> >
> > Please note, current driver is supposed to work only with newest
> > Gen3 SoCs revisions which IPMMU hardware supports stage 2 translation
> This should be "R-Car Gen3 SoCs", instead of "Gen3 SoCs".
Will update.


> > table format and is able to use CPU's P2M table as is if one is
> > 3-level page table (up to 40 bit IPA).
> >
> > The major differences compare to the Linux driver are:
> >
> > 1. Stage 1/Stage 2 translation. Linux driver supports Stage 1
> > translation only (with Stage 1 translation table format). It manages
> > page table by itself. But Xen driver supports Stage 2 translation
> > (with Stage 2 translation table format) to be able to share the P2M
> > with the CPU. Stage 1 translation is always bypassed in Xen driver.
> >
> > So, Xen driver is supposed to be used with newest Gen3 SoC revisions only
> Same here.
ok


> > (H3 ES3.0, M3 ES3.0, etc.) which IPMMU H/W supports stage 2 translation
> According to the latest manual, M3 ES3.0 is named as "M3-W+".
Will update.


> > diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c 
> > b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
> > new file mode 100644
> > index 0000000..a34a8f8
> > --- /dev/null
> > +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
> > @@ -0,0 +1,1342 @@
> > +/*
> > + * xen/drivers/passthrough/arm/ipmmu-vmsa.c
> > + *
> > + * Driver for the Renesas IPMMU-VMSA found in R-Car Gen3 SoCs.
> > + *
> > + * The IPMMU-VMSA is VMSA-compatible I/O Memory Management Unit (IOMMU)
> > + * which provides address translation and access protection functionalities
> > + * to processing units and interconnect networks.
> > + *
> > + * Please note, current driver is supposed to work only with newest Gen3 SoCs
> > + * revisions which IPMMU hardware supports stage 2 translation table format and
> > + * is able to use CPU's P2M table as is.
> > + *
> > + * Based on Linux's IPMMU-VMSA driver from Renesas BSP:
> > + *    drivers/iommu/ipmmu-vmsa.c
> So, I think the Linux's Copyrights should be described here.
Yes, will add.


> > + * you can found at:
> > + *    url: git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git
> > + *    branch: v4.14.75-ltsi/rcar-3.9.6
> > + *    commit: e206eb5b81a60e64c35fbc3a999b1a0db2b98044
> > + * and Xen's SMMU driver:
> > + *    xen/drivers/passthrough/arm/smmu.c
> > + *
> > + * Copyright (C) 2016-2019 EPAM Systems Inc.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms and conditions of the GNU General Public
> > + * License, version 2, as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > + * General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU General Public
> > + * License along with this program; If not, see <http://www.gnu.org/licenses/>.
> I don't know that Xen license description rule, but since a few source files 
> have
> SPDX-License-Identifier, can we also use it on the driver?
I am afraid, I don't know a correct answer for this question. I would 
leave this to maintainers.
I just followed sample copyright notice for GPL v2 License according to 
the document:
http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=CONTRIBUTING


> > + */
> > +
> > +#include <xen/delay.h>
> > +#include <xen/err.h>
> > +#include <xen/irq.h>
> > +#include <xen/lib.h>
> > +#include <xen/list.h>
> I don't know that Xen passthrough driver rule though, doesn't here need
> #include <xen/iommu.h>? (The xen/sched.h seems to have it so that
> no compile error happens though.)
Probably, yes, I should have included that header.


> > +
> > +#define IMTTBCR                        0x0008
> > +#define IMTTBCR_EAE                    (1 << 31)
> > +#define IMTTBCR_PMB                    (1 << 30)
> > +#define IMTTBCR_SH1_NON_SHAREABLE      (0 << 28)
> > +#define IMTTBCR_SH1_OUTER_SHAREABLE    (2 << 28)
> > +#define IMTTBCR_SH1_INNER_SHAREABLE    (3 << 28)
> > +#define IMTTBCR_SH1_MASK               (3 << 28)
> > +#define IMTTBCR_ORGN1_NC               (0 << 26)
> > +#define IMTTBCR_ORGN1_WB_WA            (1 << 26)
> > +#define IMTTBCR_ORGN1_WT               (2 << 26)
> > +#define IMTTBCR_ORGN1_WB               (3 << 26)
> > +#define IMTTBCR_ORGN1_MASK             (3 << 26)
> > +#define IMTTBCR_IRGN1_NC               (0 << 24)
> > +#define IMTTBCR_IRGN1_WB_WA            (1 << 24)
> > +#define IMTTBCR_IRGN1_WT               (2 << 24)
> > +#define IMTTBCR_IRGN1_WB               (3 << 24)
> > +#define IMTTBCR_IRGN1_MASK             (3 << 24)
> > +#define IMTTBCR_TSZ1_MASK              (1f << 16)
> At the moment, no one uses it though, this should be (0x1f << 16).
Will correct.


> <snip>
> +/* Xen IOMMU ops */
> > +static int __must_check ipmmu_iotlb_flush_all(struct domain *d)
> > +{
> > +    struct ipmmu_vmsa_xen_domain *xen_domain = dom_iommu(d)->arch.priv;
> > +
> > +    if ( !xen_domain || !xen_domain->root_domain )
> > +        return 0;
> > +
> > +    spin_lock(&xen_domain->lock);
> Is local irq is already disabled here?
> If no, you should use spin_lock_irqsave() because the ipmmu_irq() also
> gets the lock.

No, it is not disabled. But, ipmmu_irq() uses another mmu->lock. So, I 
think, there won't be a deadlock.
Or I really missed something?

If we worry about ipmmu_tlb_invalidate() which is called here (to 
perform a flush by request from P2M code, which manages a page table) 
and from the irq handler (to perform a flush to resume address 
translation), I could use a tasklet to schedule ipmmu_tlb_invalidate() 
from the irq handler then. This way we would get this serialized. What 
do you think?

> # To be honest, in normal case, any irq on the current implementation
> # should not happen though.
Agree here.


> > +    /*
> > +     * Destroy Root IPMMU domain which context is mapped to this Xen domain
> > +     * if exits.
> > +     */
> > +    if ( xen_domain->root_domain )
> > +        ipmmu_free_root_domain(xen_domain->root_domain);
> > +
> > +    spin_unlock(&xen_domain->lock);
> > +
> > +    /*
> > +     * We assume that all master devices have already been detached from
> > +     * this Xen domain and there must be no associated Cache IPMMU domains
> > +     * in use.
> > +     */
> > +    ASSERT(list_empty(&xen_domain->cache_domains));
> I think this should be in the spin lock held by &xen_domain->lock.
OK. Will put spin_unlock after it.


> > +    xfree(xen_domain);
> > +    dom_iommu(d)->arch.priv = NULL;
> > +}
> > +
> > +static const struct iommu_ops ipmmu_iommu_ops =
> > +{
> > +    .init            = ipmmu_iommu_domain_init,
> > +    .hwdom_init      = ipmmu_iommu_hwdom_init,
> > +    .teardown        = ipmmu_iommu_domain_teardown,
> > +    .iotlb_flush     = ipmmu_iotlb_flush,
> > +    .iotlb_flush_all = ipmmu_iotlb_flush_all,
> > +    .assign_device   = ipmmu_assign_device,
> > +    .reassign_device = ipmmu_reassign_device,
> > +    .map_page        = arm_iommu_map_page,
> > +    .unmap_page      = arm_iommu_unmap_page,
> > +    .add_device      = ipmmu_add_device,
> > +};
> > +
> > +/* RCAR GEN3 product and cut information. */
> "R-Car Gen3 SoCs" is better than "RCAR GEN3".
Will update.


> > +#define RCAR_PRODUCT_MASK    0x00007F00
> > +#define RCAR_PRODUCT_H3      0x00004F00
> > +#define RCAR_PRODUCT_M3      0x00005200
> At least, I think we should be M3W, instead of M3.
> # FYI, M3-W and M3-W+ are the same value.
Will update.


--
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel