[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: U-Boot on XEN - dcache off


  • To: Michal Simek <michal.simek@xxxxxxx>, "Stabellini, Stefano" <stefano.stabellini@xxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Julien Grall <julien@xxxxxxx>, Bertrand Marquis <Bertrand.Marquis@xxxxxxx>
  • From: Ayan Kumar Halder <ayankuma@xxxxxxx>
  • Date: Mon, 17 Jul 2023 16:31:21 +0100
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VLaD+0ThcHVK2/31hvO4n7RuN4Tigg93pMW/x7rwHpg=; b=mBh7ziIizl8hHtfYAm+n9nzP5M54G3CqUXBhqHZ3xMjky5qEkdGOecQB4GFdSwgzsI2td+U+GTbSLmeuShdSaeYlGLiuzh32TlPHlXX+5l6v8Md/AVj+LH/y36zY2R1aDT9Rv2Zu9t11Rfql4IxSk2CSwHmobx1srjaCzlGphgPVej57Qyp0cB7UK9kgOfa/JXQVnCOoVeO5VqtlQDLyrbgsZ34ZzxO107Qq0YdjXdE9ZlJeYoYJXOqQgstB5ZWRxLfYZ//B9jnIEOmtmlNWiD0rANJ9YONzhlbpQ4rhq8IgT9mv+YF5FbiNloFMAwvBZKT5PNlUav3EZFquS/w09g==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IXu0Ep1f3TjU6PkRNoSZmdK5MfSLQ2ObeiamdrE6uQo0WaRe6l58sQSMxF++TUwhFlko773orT05MQZ96vU5NULxqwOVu14GKqpTEnfZJA++J00/ImK5qIubL6Jag1dghW4np26o8Kun3Cc1ttRXlcWs3XT7nHMdgAY0JkhxJz3BKS6TH3+VdTJZ4gQZlvwBtpnq/PAgYPhDHZEh4/1z92SmLIOHJjmkeD1df0UpGM2RzRn5qIM4niydaXfI6C3PPmkAO4fKs2Ic6n+FjQL8WY53kXOdDb14f9NQ3IT1CDb4DNiZpGKDciVWqFyV1VzCPguB5JMMKn+u0pVfs0fF6g==
  • Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com;
  • Cc: Michal Orzel <michal.orzel@xxxxxxx>, Stewart Hildebrand <stewart.hildebrand@xxxxxxx>, "Garhwal, Vikram" <vikram.garhwal@xxxxxxx>, Xenia.Ragiadakou@xxxxxxx
  • Delivery-date: Mon, 17 Jul 2023 15:31:44 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

+ xen-devel (Since the question is related to Xen on Arm in general).

Hi Michal,

Julien/Stefano/Bertrand - Please feel free to add in.

On 14/07/2023 15:14, Michal Simek wrote:
+xen

On 7/14/23 16:11, Michal Simek wrote:
Hi Stefano,

I am playing with running u-boot on the top of xen (xenguest_arm64_defconfig) based on ask from Kris to run QNX on the top of it.
In past we found that QNX requires to have DCACHE off before it's start.
Chris found in past that dcache off is not working and he passed this to us to take look at it more.

That's why I enabled cache commands to be able to run dcache off and see the hang as Kris. I found one interesting commit created by Marc for Android which were taken back to u-boot upstream. It is talking about KVM and I barely understand what he is talking about.
I of course enabled this option to try it and behavior is the same.
That's why my question is what Xen is doing in connection to dcache operations.

WRT cache operations, Refer https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=53b705d02cec03861044e673536586bd1b2443bd

When the data abort is caused due to cache maintenance for an address,
there are three scenarios:-

1. Address belonging to a non emulated region - For this, Xen should
set the corresponding bit in the translation table entry to valid and
return to the guest to retry the instruction. This can happen sometimes
as Xen need to set the translation table entry to invalid. (for eg
'Break-Before-Make' sequence). Xen returns to the guest to retry the
instruction.

2. Address belongs to an emulated region - Xen should ignore the
instruction (ie increment the PC) and return to the guest.

3. Address is invalid - Xen should forward the data abort to the guest.

Do you know what by-VA cache maintenance operations are?

These are cache operations by virtual address. Refer https://developer.arm.com/documentation/den0024/a/Caches/Cache-maintenance

The cache operations (invalidate, clean, zero) can be performed in any of the three ways :-

1. all - The entire cache is affected

2. VA - Cache line that contains a certain virtual address is affected.

3. Set/way :- Cache line selected by its position in the cache structure.



commit 46dc54287031759c03c68902283d92076938305c
Author:     Marc Zyngier <maz@xxxxxxxxxx>
AuthorDate: Thu Feb 9 04:54:27 2023 +0800
Commit:     Tom Rini <trini@xxxxxxxxxxxx>
CommitDate: Mon Mar 6 17:03:55 2023 -0500

     arm: cpu: Add optional CMOs by VA

     Exposing set/way cache maintenance to a virtual machine is unsafe, not
     least because the instructions are not permission-checked but also
     because they are not broadcast between CPUs. Consequently, KVM traps and      emulates such maintenance in the host kernel using by-VA operations

I am not sure if Xen behave in same way (ie traps and emulates *all* cache maintenance instructions).

@Julien, Stefano, Bertrand - Do you know more about this ?

and
     looping over the stage-2 page-tables. However, when running under
     protected KVM, these instructions are not able to be emulated and will
     instead result in an exception being delivered to the guest.

     Introduce CONFIG_CMO_BY_VA_ONLY so that virtual platforms can select      this option and perform by-VA cache maintenance instead of using the
     set/way instructions.

     Signed-off-by: Marc Zyngier <maz@xxxxxxxxxx>
     Signed-off-by: Will Deacon <willdeacon@xxxxxxxxxx>
     Signed-off-by: Pierre-Clément Tosi <ptosi@xxxxxxxxxx>
     [ Paul: pick from the Android tree. Fixup Pierre's commit. And fix some
       checkpatch warnings. Rebased to upstream. ]
     Signed-off-by: Ying-Chun Liu (PaulLiu) <paul.liu@xxxxxxxxxx>
     Cc: Tom Rini <trini@xxxxxxxxxxxx>
     Link: https://android.googlesource.com/platform/external/u-boot/+/db5507f47f4f57f766d52f753ff2cc761afc213b      Link: https://android.googlesource.com/platform/external/u-boot/+/2baf54e743380a1e4a6bc2dbdde020a2e783ff67


Code ends when it tries to call
set_sctlr(sctlr & ~(CR_C|CR_M));

Here is implementation. I confirm EL level which is obviously EL1.

180 static inline void set_sctlr(unsigned long val)
181 {
182         unsigned int el;
183
184         el = current_el();
185         if (el == 1)
186                 asm volatile("msr sctlr_el1, %0" : : "r" (val) : "cc");
187         else if (el == 2)
188                 asm volatile("msr sctlr_el2, %0" : : "r" (val) : "cc");
189         else
190                 asm volatile("msr sctlr_el3, %0" : : "r" (val) : "cc");
191
192         asm volatile("isb");
193 }

Is there any reason why it is not possible to clear to sctlr_el1 CR_C/CR_M - cache disabling, MMU disabling?

Not sure IIUC,  sctlr_el1 CR_C/CR_M are being cleared to disable D cache/MMU. Did I miss something ?

- Ayan

I checked already that u-boot starts with cache off/mmu off and when tables are setup bits are setup.

Thanks,
Michal



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.