[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Need guidance to support reading GICR_TYPER (64 bit register) on Aarch32_v8r


  • To: Julien Grall <julien@xxxxxxx>
  • From: Bertrand Marquis <Bertrand.Marquis@xxxxxxx>
  • Date: Mon, 17 Oct 2022 09:17:42 +0000
  • Accept-language: en-GB, en-US
  • Arc-authentication-results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com])
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
  • Arc-message-signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2Rqj6bNA0Hyqnzx2dS4iINL/TzqaVpGrU1oBbeVg54g=; b=YMPCwFevDAk6XNQcWjUyqe0pdUbX2b5OlxKEssKhgPU7W6SDTp5oMKGufzr93eYWcemtVuMLyjhK5FvEhxSvhIB56hu3xJqIkHdC5oK3n7sjvW6OA+4pP/4zZEU6e+RgXDLdlYBYMWZyHD9jy7IjsuE9daVQ/lFo65ekMzsas8LERNOW7F/QzNi4HMt54Bk5nfybVePQcC0o8f/5hpkTQ3If+BUX3wPziWezr/rmoeFMI5jxiUPJSr7DRrLcPLy/sWNUtxvs8yeD4bOhCXQO04C8DY0ek6geG78NwWj2i5vKLUT/6ve0zG8cMti3xpU33TIs5V/jFXKolohoVNh+lA==
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2Rqj6bNA0Hyqnzx2dS4iINL/TzqaVpGrU1oBbeVg54g=; b=Ac0phueeVwIoLs9wYTQbHkvT2/VFIsWPy5a0ML/2WXamyyHy5EbvYWv0ZqhnftXv/u/Mt7wWbHWqIxjHDUyAttk7G8gqrTAUIb/vqRSy6EGbLrusiUbXEIcHvD4T6YpTGUWVrRYngPqpAYvtpwzLtAgBpQGTGqbXoaSx2SQzxBq/thL59g7MeZuF0AItUXRdek86SK36Tae+/jfVkF4CpM6saFImvsQEQI7dHEZbGLjMU9z+JcCW1MO8qkqcKGawZAxZm95Yu9s8+8MmN0IHA/W++DgRfrEGIXFWPJLd56cOtoJzxD0UAwDbj8L8+vQIdQieilcalUHDiNdPbQr7EQ==
  • Arc-seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=N2XPB9x+ClvVxti1EbiC/YUciq6DIf1vbIg9vp7zFeZvaW0uMJdJ5Tgwx2SlFj2kami0sKbSAUOa4V9hn2y3PAjxQJioKoEWyEgvkM+Sv8goTPowew5ZC04rDCf81EvPh2BabeuAm1pSDoQ/T3d58UjCht0b1RAmSdTlov1qts8w/Tddf4C8dLjDfGBjs3heb/BWI7qTRBs/vYGoOs5ddi0inoR3yClwsY2epxfnu5B2vuhlQpbKKrkH74gxFCCHuQejdf3CicnuwA0tCIfhxmPfgmlLn+sZN8UMuhmCPUzdfz909oRCy/9E2K0s1pI4hwaSg6RRI5GMQApG+ZUxfg==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MZefKxlikl8kO00MI3PpJb2l0JebFbvVB4G5johF2WiIfyU9+UqV2fQDSqn96oAM1DbI4BB9eA1TKDI7yGG6+fnPJ7ct1hYc8Gpl+SvyWfueRRqYfvcK/MktHIAPP6dDZXR8xUKthsYA8w8PjgxUk+mWXzlxmSyV5RLp3oi1DcK0poWwqC/XDF/Iwwc1SE2vYUilKEZSiwL2+7GDndBsOCdlxT3fU4USH//QHxEMJZ/Orc1ISpnlmlDgGz3bJO3InS9HofJwyExCjpQDX6Cnob8mQ/qbvRZxwxw/nXEQ1SEF4UMcbQGWRXEa8ahxu0GfYpqf+KiZsmT/8pwY7CcAxg==
  • Authentication-results-original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
  • Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>, Ayan Kumar Halder <ayankuma@xxxxxxx>, "Stabellini, Stefano" <stefano.stabellini@xxxxxxx>, "Volodymyr_Babchuk@xxxxxxxx" <Volodymyr_Babchuk@xxxxxxxx>, Henry Wang <Henry.Wang@xxxxxxx>, Penny Zheng <Penny.Zheng@xxxxxxx>, Jaxson Han <Jaxson.Han@xxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Mon, 17 Oct 2022 09:17:58 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Nodisclaimer: true
  • Original-authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
  • Thread-index: AQHY3vmtdKYr9F2nyUuznJvSQYcvHq4OMlaAgAEAmYCAAyG8AA==
  • Thread-topic: Need guidance to support reading GICR_TYPER (64 bit register) on Aarch32_v8r

Hi,

> On 15 Oct 2022, at 10:28, Julien Grall <julien@xxxxxxx> wrote:
> 
> Hi Stefano,
> 
> On 14/10/2022 19:09, Stefano Stabellini wrote:
>> On Thu, 13 Oct 2022, Ayan Kumar Halder wrote:
>>> Hi Arm mantainers/Folks,
>>> 
>>> Please refer to the discussion
>>> https://github.com/zephyrproject-rtos/zephyr/pull/51163 .
>>> 
>>> We intend to run Zephyr as a domU guest on Xen on Aarch32_v8R fixed virtual
>>> platform.
>>> 
>>> Zephyr is trying to read GICR_TYPER which is a 64 bit register using ldrd
>>> instruction.
>>> 
>>> As GICR is emulated by Xen, so this instruction gets trapped with HSR =
>>> 0x9200000c.
>>> 
>>> As ISV is 0, so Xen cannot emulate this instruction.
>>> 
>>> The proposed solution is to use two sys_read32() on GICR_TYPER to return the
>>> lower and upper 32 bits.
>>> 
>>> With this, HSR = 0x9383 000c, ISV=1 so ISS is valid.
>> Hi all,
>> I wanted to take a step back on this issue before we jump into the
>> details.
>> Differently from other instructions we discussed in the past, strd and ldrd
>> are not deprecated and are not "unusual corner cases". There is no
>> statements such as "please don't use this" on the ARM ARM. If I were to
>> write an register read/write function in assembly for an RTOS, it would
>> be reasonable to use them.
> 
> Just to be clear it is fine to use the ldrd/strd for accessing non MMIO area. 
> The problem comes with MMIO access because they can be emulated by the 
> hypervisor and we don't have the syndrome. At the moment, this is only a 
> problem when accessing some of the GICv3 (including ITS) registers.
> 
>> So, I struggle to see how we'll be able to deal with all the possible
>> RTOSes out there that might have them in the code. We can fix Zephyr,
>> but what about FreeRTOS, ThreadX and the proprietary ones (VxWorks,
>> etc.)?
> 
> This is not an Xen issue but architecture issue. The RTOSes will face the 
> exact same issue on any hypervisors unless they decided to decode the 
> instruction.
> 
> As we discussed before decoding an instruction correctly is quite difficult 
> to do (what we have in Xen for pos-increment store/load is just a band-aid). 
> So I would expect the other hypervisors to have made the decision to not 
> implement it. AFAIK KVM doesn't suppor them,
> Note that looking at ID_ISAR2, it seems that ldrd/strd is technically 
> optional. Therefore, the RTOS would have to assume it is targeting a 
> processor that supports them.
> 
>> Unless we can get ARM to issue a clear guidance that strd and ldrd are
>> deprecated, 
> 
> Arm Arm cannot say that because ldrd/strd are necessary to modify the LPAE 
> page-tables atomically. What we need to know is which instructions can be 
> allowed on MMIO accesses.

Definitely this is something that arm arm cannot fully answer as it is also 
down to the full platform. MMIO accesses are going out of the CPU and hence 
wether or not 64bit MMIO accesses can be properly done might also depend on the 
bus or the IP on the other side (some peripherals might just refuse 64bit 
accesses or some bus might only be 32bit so the operations would need to be 
divided).

> 
> I think I already raised that when Ayan added decoding for post-increment 
> instructions. There are plenty of instructions (or combinations) that doesn't 
> provide a syndrome and yet the processor doesn't prevent anyone to use them 
> on MMIO.
> 
> I was worry we are going to have to continue to decode instructions in a 
> non-compliant way in Xen just to please a few RTOs that may not even run 
> anywhere else.
> 
> This would also reduce our leverage to request a change in the RTOes or the 
> Arm Arm (maybe there is already a statement I haven't spotted) because Xen 
> will already (badly) support the instruction.

Going back on the ID_ISAR2, if Xen is properly setting the value seen by the 
guests, there is not reason for us to actually emulate those instructions.
The emulation code inside Xen cost a lot in matter of lines of code and would 
need a lot of testing (which is missing at the moment).
So as we have a standard way to inform the guest that this is not supported, we 
should stick to that.

> 
>> I think it would be better to attempt to decode them rather
>> than just fail. I don't like to have this kind of code in Xen, but I
>> don't see a way to support R52s without it.
> That's not specific to R52. This is anyone using GICv3 on Arm32 core.

Agree.

> 
>> That said, of course if Zephyr was to use two 32-bit reads instead of
>> one 64-bit read, it would be better for Xen. And we have more important
>> things to deal with right now in terms of R52 support (it is not even
>> upstream yet). So it is totally fine to change Zephyr and move forward
>> for now.
>> But medium term it doesn't seem to me that we can get away without a
>> solution in Xen for this (or a change in the ARM ARM).
> 
> See above. This is an architecture problem and we should discuss with Arm 
> first before continuing to add more decoding in Xen.

I will discuss it internally to have an answer but I think that the answer 
cannot only come from Arm as there are for sure hardware implementations that 
cannot support this, as explain before.

Cheers
Bertrand

> 
> Cheers,
> 
> -- 
> Julien Grall




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.