[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [PATCH] xen/arm: Enhance IPMMU-VMSA driver robustness and debug output


  • To: Julien Grall <julien@xxxxxxx>, "Orzel, Michal" <michal.orzel@xxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • From: Jahan Murudi <jahan.murudi.zg@xxxxxxxxxxx>
  • Date: Mon, 30 Jun 2025 06:37:59 +0000
  • Accept-language: en-IN, kn-IN, en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=renesas.com; dmarc=pass action=none header.from=renesas.com; dkim=pass header.d=renesas.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RKTwIAMPZQYCMfAm2UpJV0s1Q9W/Ub2YeOqTbzmLcXM=; b=wHKAn/7yFbqGCjM9u4yNqoZ24kMoaqjmA+MiJ9DXHzBqhYaLEAYTlkWRdr0vCQlPKHrz+d8rkDlREpwNMB0mXSH8sotVYIbzHfiSzuKY1f4uoc5yH2YFDMfDFfy6nKR8n97PmnSWtBz7gqSCu3vHTCtWuLQRAgBHpXv5Lmsv5B2bPCOveUt34jQV0jV+iu3y3OeexPBt+ahBdpum9nMagQW7lWn0OcAA0mlWYnVjU3LkxmzLLhTnmNd4QQMQB171x5LGSpy6aVNZiR0FyoJ0uult5mzLi0JaI17Rc9yocSk6k0V+taI/oLaCpmRbNn+0gMrr0QzrpnS4AVbE1XCXJA==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=QK6lIra9gFwOa8qsAtDW3onJrp1U6RVsY4WIbPhUIj0Re6FCMJ0mXDesLXTQlr1BEXVf9N8p+tlj5fH7FVmLD3hv1svyKIiRse2S+Cbjbb7wsNgC4qxr8dnxmDp5yRL+HKJb44Uv+dYl9hyoIrE7PccBSS50oQejdfr9ASkS0DVmWZOZsIZSvfQ76cQMPCLNUjIT81l3rU+JWISFUO5oDOrjurK/0BROU8wjqyEL4AK4vCGQB45ObWC/WSnlnlMcBZk6Ir3hFIZZPNyAQJuSz7S067BtwFMccbyDCgaDAxeVfzCtLbuPevK710ddc8Q6d6LZfjJYfDh638tr/e00kQ==
  • Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=renesas.com;
  • Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>, Bertrand Marquis <bertrand.marquis@xxxxxxx>, Volodymyr Babchuk <volodymyr_babchuk@xxxxxxxx>
  • Delivery-date: Mon, 30 Jun 2025 06:38:14 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Thread-index: AQHb4c6ALl3IHvqkBEOgbLXzyjOhU7QQXHcAgANRIGCAABW5gIAHh3bA
  • Thread-topic: [PATCH] xen/arm: Enhance IPMMU-VMSA driver robustness and debug output

Hi Julien,

On 25/06/2025 16:53, Julien Grall wrote:

>Hi Jahan,

>>>> +    dsb(sy);
>>> Any clue why Linux (mainline) does not do that?

> One process remark, we typically comment inline rather than pasting a quote 
> and replying at the top of the e-mail.

 Thanks for the style note - I'll follow the inline commenting convention 
moving forward.

>> The implementation writel() which contains an implicit dsb(st) which likely 
>> sufficient for Linux for its Stage-1 IOMMU usage where CPU and IOMMU 
>> interactions are coherent.
>> However, Xen uses the IPMMU as a Stage-2 IOMMU for non-coherent DMA 
>> operations (such as PCIe passthrough), requiring the stronger dsb(sy) to 
>> ensure writes fully propagate to the IPMMU >>hardware before continuing.

> I don't follow. Are you saying the IPMMU driver in Linux doesn't non-coherent 
> DMA operations?

Let me clarify my understanding:  In native Linux, the IOMMU works at stage-1 
(VA -> PA) and typically assumes coherency between CPU and IOMMU. The implicit 
dsb(st) in writel() is enough there. But in Xen, we use this as stage-2 (GPA -> 
HPA) for cases like PCI passthrough where devices might be non-coherent. We 
might need stronger barrier dsb(sy) in xen because: 1) We can't assume the TLB 
walker is coherent for stage -2 and we must also prevent(minimise) any DMA 
operations during TLB invalidation( observed some IPMMU hardware limitations in 
the documentation) .

> But even if that's the case, I still don't see why non-coherent DMA would 
> matter. From my understanding, here we want to make sure the TLB walker sees 
> the change before the flush.
> So if the TLB walker is coherent with the rest of the system. Then it would 
> be similar to the CPU TLBs where we only need a "dsb st" (well we use "nshst" 
> because the TLB is in non-shareable domain).
> If the walker is not coherent, then that's a different topic.

You're correct that dsb(st) would suffice in an ideal coherent system. However, 
with PCI passthrough we must handle non-coherent devices. While dsb(st) ensures 
writes complete, dsb(sy) provides the stronger system-wide visibility we need - 
guaranteeing all components (including non-coherent devices) see the changes 
before proceeding.

> Anyway, I am not against using "dsb(sy)". It is stronger than necessary but 
> also probably not a massive deal in the TLB flush path.

Thank you. I agree the performance impact is negligible in the flush path, and 
it's better to be safe when dealing with passthrough devices in xen.

Regards,
Jahan Murudi

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.