[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [XEN v4] xen/arm: vGICv3: Emulate properly 32-bit access on GICR_PENDBASER


  • To: Andre Przywara <andre.przywara@xxxxxxx>
  • From: Ayan Kumar Halder <ayankuma@xxxxxxx>
  • Date: Thu, 27 Oct 2022 12:35:13 +0100
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oVfX0Aryh8o2SWZb+BKO3OPNPiM3peaM0ZlSDjIa5NU=; b=DAtA8dkxsCbK5n+bI2iANAW29wu5lMEocTy3ZC0KknqVMnft49BWHX1CBMhldFwCVH7L39sQNB7dUyTjgvx5H5CO25Sv2W2Gg712BLnxTeRBHDNfZolrEZZ0Dh12W5oYiCEoj+NWalHkUiyZb4KtCCVgKI9ATRtOiPKgQvrZyHSlVkRLqyfejb+GvcN1F7+WsDKwHn3Tp8N1794J0Va7hQ9lwvDdRjP8F/pWC+br06ScOZ0vw7b8grY84bdKEaIiaAGR62D1ARPqoLNr9Qus3+6a0j1YqxUk9gjGAlX0zmLUkXhrC0KyEY1aejbXDktx9/AlxQZjgf2+OkYKZN5rDQ==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SEZVoj2DKxs3jtMymDEUvUW3bDsMCIQbuU/pfuN+sTSREwlsYJ4pdvH25efXcoKcn4r8lThZTDCbaRTOu+mKN6M5q3uKdPzYSi3/74VVnmUrEVPAJAt8qcrj8mK1KQRoYp9exR9RqfpX1A5z0OoE6K4b40M1PbvZbmvzXa/6lmfnB1euv2+WSmA1J0g3SX9RsOLgrMf8tapJqeC2BCJtGln/GwvokiotpRPFnWRRL0uKwErgfsMaShzOFgWPcDAPnFVC4mkxHFWFoRVZoZaasKg6I+T6ffyPMUcLDbpKgkYinGAWAOdBKKXyr9X6oRtqpMjqCXmj0xKZOp9uTO6A/Q==
  • Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com;
  • Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx, sstabellini@xxxxxxxxxx, stefanos@xxxxxxxxxx, julien@xxxxxxx, Volodymyr_Babchuk@xxxxxxxx, bertrand.marquis@xxxxxxx, Henry.Wang@xxxxxxx
  • Delivery-date: Thu, 27 Oct 2022 11:35:43 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>


On 27/10/2022 10:44, Andre Przywara wrote:
On Wed, 26 Oct 2022 19:30:04 +0100
Ayan Kumar Halder <ayankuma@xxxxxxx> wrote:

Hi,

Hi Andre,

Thanks for the feedback. Just one clarification.


If a guest is running in 32 bit mode and it tries to access
"GICR_PENDBASER + 4" mmio reg, it will be trapped to Xen. vreg_reg64_extract()
will return the value stored "v->arch.vgic.rdist_pendbase + 4".
This will be stored in a 64bit cpu register.
So now we have the top 32 bits of GICR_PENDBASER (a 64 bit MMIO register) stored
in the lower 32 bits of the 64bit cpu register.

This 64bit cpu register is then modified bitwise with a mask (ie
GICR_PENDBASER_PTZ, it clears the 62nd bit). But the PTZ (which is bit 30 in the
64 bit cpu register) is not cleared as expected by the specification.

The correct thing to do here is to store the value of
"v->arch.vgic.rdist_pendbase" in a temporary 64 bit variable. This variable is
then modified bitwise with GICR_PENDBASER_PTZ mask. It is then passed to
vreg_reg64_extract() which will extract 32 bits from the given offset.

Also, we have removed spin_lock_irqsave()/spin_unlock_irqrestore() to protect
v->arch.vgic.rdist_pendbase. The reason being v->arch.vgic.rdist_pendbase is
now being read/written in an atomic manner (using read_atomic()/write_atomic()).

Signed-off-by: Ayan Kumar Halder <ayankuma@xxxxxxx>
Release-acked-by: Henry Wang <Henry.Wang@xxxxxxx>
---

Changes from:-

v1 - 1. Extracted this fix from "[RFC PATCH v1 05/12] Arm: GICv3: Emulate
GICR_PENDBASER and GICR_PROPBASER on AArch32" into a separate patch with an
appropriate commit message.

v2 - 1. Removed spin_lock_irqsave(). Used read_atomic() to read
v->arch.vgic.rdist_pendbase in an atomic context.
2. Rectified the commit message to state that the cpu register is 64 bit.
(because currently, GICv3 is supported on Arm64 only). Reworded to make it
clear.

v3 - 1. Added read_atomic()/write_atomic() for access to 
v->arch.vgic.rdist_pendbase
in __vgic_v3_rdistr_rd_mmio_write().

  xen/arch/arm/vgic-v3.c | 19 ++++++-------------
  1 file changed, 6 insertions(+), 13 deletions(-)

diff --git a/xen/arch/arm/vgic-v3.c b/xen/arch/arm/vgic-v3.c
index 0c23f6df9d..1adbdc0e54 100644
--- a/xen/arch/arm/vgic-v3.c
+++ b/xen/arch/arm/vgic-v3.c
@@ -249,16 +249,15 @@ static int __vgic_v3_rdistr_rd_mmio_read(struct vcpu *v, 
mmio_info_t *info,
case VREG64(GICR_PENDBASER):
      {
-        unsigned long flags;
+        uint64_t val;
if ( !v->domain->arch.vgic.has_its )
              goto read_as_zero_64;
          if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
- spin_lock_irqsave(&v->arch.vgic.lock, flags);
-        *r = vreg_reg64_extract(v->arch.vgic.rdist_pendbase, info);
-        *r &= ~GICR_PENDBASER_PTZ;       /* WO, reads as 0 */
-        spin_unlock_irqrestore(&v->arch.vgic.lock, flags);
+        val = read_atomic(&v->arch.vgic.rdist_pendbase);
+        val &= ~GICR_PENDBASER_PTZ;      /* WO, reads as 0 */
+        *r = vreg_reg64_extract(val, info);
That part looks fine now.

          return 1;
      }
@@ -566,25 +565,19 @@ static int __vgic_v3_rdistr_rd_mmio_write(struct vcpu *v, mmio_info_t *info, case VREG64(GICR_PENDBASER):
      {
-        unsigned long flags;
-
          if ( !v->domain->arch.vgic.has_its )
              goto write_ignore_64;
          if ( !vgic_reg64_check_access(dabt) ) goto bad_width;
- spin_lock_irqsave(&v->arch.vgic.lock, flags);
-
I don't think you can drop the lock here easily. If it would be just for
the LPIs enabled check, that'd be fine, because you can never turn LPIs off
again (but that would deserve an explicit comment then).

/* LPIs once enabled (ie via vgic_vcpu_enable_lpis()) are never disabled by Xen */

Let me know if this comment makes sense.


But down below you do a read-modify-write operation of rdist_pendbase, and
need to make sure no one else is attempting that at the same time.

AFAIU, Xen is non preemptable. So there cannot be a context switch between read-modify-write.

But I think you have a valid point.

If two callers are trying to read-modify-write on rdist_pendbase at the same time, it is possible that caller2 reads a stale value of rdist_pendbase (just before caller1 writes). So caller2 might program rdist_pendbase in such a way that caller1's update is gone.

I will fix this in v5.

- Ayan


Cheers,
Andre

          /* Writing PENDBASER with LPIs enabled is UNPREDICTABLE. */
          if ( !(v->arch.vgic.flags & VGIC_V3_LPIS_ENABLED) )
          {
-            reg = v->arch.vgic.rdist_pendbase;
+            reg = read_atomic(&v->arch.vgic.rdist_pendbase);
              vreg_reg64_update(&reg, r, info);
              reg = sanitize_pendbaser(reg);
-            v->arch.vgic.rdist_pendbase = reg;
+            write_atomic(&v->arch.vgic.rdist_pendbase, reg);
          }
- spin_unlock_irqrestore(&v->arch.vgic.lock, false);
-
          return 1;
      }



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.