[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH] xen/arm: smmuv1: remove iommu group when deassign a device


  • To: Mykyta Poturai <mykyta.poturai@xxxxxxxxx>
  • From: Rahul Singh <Rahul.Singh@xxxxxxx>
  • Date: Thu, 16 Jun 2022 10:55:56 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com])
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
  • Arc-message-signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OnT+k9wLLW9bNIoLd/BEczpkgmZl1ZO94X5qi6ahdWc=; b=mZjtEI+i2YlPFEPAF9e/IRLi4uEJWd0YvnCrunbHNGIrMNpPQ06vzJxGtLQh/vl8HNM2b/CcUKiSGxBaKrQlq4Likz7sz0nX0dYTX72wl9rUqW2H7EvjNBSjs+G4qN3SUzEjrSKqmn+5OUyug8pDQv8zkx6Y7PjGD5XYPwTIa0ekaqxmHpySMOS6fZlyulPERwyieVHJGxgIcSYMeqlLYQr5Bxx6RkOllfviI83j1J5UnJrp2yFRUgQJYKsgg55N/JsD+7dHZ+LCL8muwUGKwpkIpKGVj0myw2gE8e/kH0K5ihpIXYvZyi+uNivgdoCciDzK0mJTbgDf8SVIp0BH3A==
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OnT+k9wLLW9bNIoLd/BEczpkgmZl1ZO94X5qi6ahdWc=; b=gG/yH49kMXZQo2md0VDODGayhYfheVwmhODOEqk0z0DYe2H7b6WYZU1b8vAt45sDO7R9CMcpJpD8IPYhrs3MV6b9EZLIC8ypywETAOeCCQAObb7j0fazECJzB2EXoC4lVU7NUm62AUyXcCxM6Av5TsDqeTvjAgYRKsXq2SkqaHNz5ku/ShbJ01wcJBSo2nhkcAsqd5ymuMINSmQ9UkXfkwNywZLvFLUTwPurvoa7scCcUCjEB+ChM14fe4i8YTILWirbAgOGE/CVxACV+TZhpZjHLGjv+fB3zg4hbYkcAlVC7As/ceoPH3xH9z2nMklpeu47icwWqUaea0jU8I0Hcw==
  • Arc-seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=lo+hmxwralDP4lF54ah1tH9VAQvhmZX4+dCbG77nGzF9DZklckMk+UWWBKDVFW4rR5YML1EaLrJfTkZJ7m2+6WfGBMALPzfvjnyGNYjOKfWWEWWYlzzUtmdhYWWvKoXnBrJyVEZYqmajGxdXl8mHagaAeavc3rumI26Fcq3tjU1lVq64jPIkieDDuozIPyu/uvtL7ac8hlboUotQZ7P8PE04kb8gtKI9skesypg1uiUgYbPva81dl5IAa2esEwzXC3ZYSopybV237ieivRhJ/pW0Ag/HSNgN+uZ6hdFl8ORlCjSgxNN54nPWTfkfUsYrJfd8/WKKa+g1znpQar8lzQ==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VQc3rp0BLtq03yp3NUP6qk9grbNoV2jz015rv8yR3jIVjj5VEG17IqEavrEa0+FPyZFi1amsWYkcZpkUm4vx1XdX2323brlbosiucKshlutL3QwgIQiNgZAN6EDoso7HjYknUehZRF4kEGeIhv8yh5UcrlDbfCqkGdx9wAT5KLl3gL29G64xCfkkIHT21UQV53dBOgB1POjBM6+mOZRC01SgqRbXKoYvIdFpS1y20CFbIcZrmXQ2y0CsxqWjH+YG001Xy6WGfqsYn8cZRd1l4GypFKZY7Irx8yW8TRwjTeVHzV6sdGUp8wXmcZZwjtX4/gldETYjkjcT29Fce0L1tg==
  • Authentication-results-original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
  • Cc: Julien Grall <julien@xxxxxxx>, Bertrand Marquis <Bertrand.Marquis@xxxxxxx>, "Volodymyr_Babchuk@xxxxxxxx" <Volodymyr_Babchuk@xxxxxxxx>, "sstabellini@xxxxxxxxxx" <sstabellini@xxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Thu, 16 Jun 2022 10:56:50 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Nodisclaimer: true
  • Original-authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
  • Thread-index: AQHYWlIekEFYa4Z+ZkOKg6prrRxnv60EB7eAgALvxoCABl1EgIBEoW2AgAA0fAA=
  • Thread-topic: [PATCH] xen/arm: smmuv1: remove iommu group when deassign a device

Hi Mykyta,

> On 16 Jun 2022, at 8:48 am, Mykyta Poturai <mykyta.poturai@xxxxxxxxx> wrote:
> 
> Hi Julien, Rahul
> I've encountered a similar problem with IMX8 GPU recently. It wasn't probing
> properly after the domain reboot.  After some digging, I came to the same
> solution as Rahul and found this thread. I also encountered the occasional
> "Unexpected global fault, this could be serious" error message when destroying
> a domain with an actively-working GPU.
> 
>> Hmmmm.... Looking at the code, arm_smmu_alloc_smes() doesn't seem to use
>> the domain information. So why would it need to be done every time it is 
>> assigned?
> Indeed after removing the arm_smmu_master_free_smes() call, both reboot and 
> global
> fault issues are gone. If I understand correctly, device removing is not yet
> supported, so I can't find a proper place for the arm_smmu_master_free_smes() 
> call.
> Should we remove the function completely or just left it commented for later 
> or
> something else?
> 
> Rahul, are you still working on this or could I send my patch?

Yes, I have this on my to-do list but I was busy with other work and it got 
delayed. 

I created another solution for this issue, in which we don’t need to call 
arm_smmu_master_free_smes() 
in arm_smmu_detach_dev() but we can configure the s2cr value to type fault in 
detach function.

Will call new function arm_smmu_domain_remove_master() in detach function that 
will revert the changes done 
by arm_smmu_domain_add_master()  in attach function.

I don’t have any board to test the patch. If it is okay, Could you please test 
the patch and let me know the result.

diff --git a/xen/drivers/passthrough/arm/smmu.c 
b/xen/drivers/passthrough/arm/smmu.c
index 69511683b4..da3adf8e7f 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -1598,21 +1598,6 @@ out_err:
        return ret;
 }
 
-static void arm_smmu_master_free_smes(struct arm_smmu_master_cfg *cfg)
-{
-    struct arm_smmu_device *smmu = cfg->smmu;
-       int i, idx;
-       struct iommu_fwspec *fwspec = arm_smmu_get_fwspec(cfg);
-
-       spin_lock(&smmu->stream_map_lock);
-       for_each_cfg_sme(cfg, i, idx, fwspec->num_ids) {
-               if (arm_smmu_free_sme(smmu, idx))
-                       arm_smmu_write_sme(smmu, idx);
-               cfg->smendx[i] = INVALID_SMENDX;
-       }
-       spin_unlock(&smmu->stream_map_lock);
-}
-
 static int arm_smmu_domain_add_master(struct arm_smmu_domain *smmu_domain,
                                      struct arm_smmu_master_cfg *cfg)
 {
@@ -1635,6 +1620,20 @@ static int arm_smmu_domain_add_master(struct 
arm_smmu_domain *smmu_domain,
        return 0;
 }
 
+static void arm_smmu_domain_remove_master(struct arm_smmu_domain *smmu_domain,
+                                     struct arm_smmu_master_cfg *cfg)
+{
+       struct arm_smmu_device *smmu = smmu_domain->smmu;
+       struct arm_smmu_s2cr *s2cr = smmu->s2crs;
+       struct iommu_fwspec *fwspec = arm_smmu_get_fwspec(cfg);
+       int i, idx;
+
+       for_each_cfg_sme(cfg, i, idx, fwspec->num_ids) {
+               s2cr[idx] = s2cr_init_val;
+               arm_smmu_write_s2cr(smmu, idx);
+       }
+}
+
 static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev)
 {
        int ret;
@@ -1684,10 +1683,11 @@ static int arm_smmu_attach_dev(struct iommu_domain 
*domain, struct device *dev)
 
 static void arm_smmu_detach_dev(struct iommu_domain *domain, struct device 
*dev)
 {
+       struct arm_smmu_domain *smmu_domain = domain->priv;
        struct arm_smmu_master_cfg *cfg = find_smmu_master_cfg(dev);
 
        if (cfg)
-               arm_smmu_master_free_smes(cfg);
+               return arm_smmu_domain_remove_master(smmu_domain, cfg);
 
 }

Regards,
Rahul

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.