RE: [EXT] Re: xen arm64 low power sleep support

[Dan Waqar <danwaqar@xxxxxxxxxx>]

-----Original Message-----
From: Stefano Stabellini <sstabellini@xxxxxxxxxx>
Sent: Tuesday, August 29, 2023 4:13 PM
To: Anthony Chan <anthonychan@xxxxxxxxxx>
Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx; bertrand.marquis@xxxxxxx; julien@xxxxxxx; 
Volodymyr_Babchuk@xxxxxxxx; michal.orzel@xxxxxxx; Dan Waqar 
<danwaqar@xxxxxxxxxx>; sstabellini@xxxxxxxxxx
Subject: [EXT] Re: xen arm64 low power sleep support

On Tue, 29 Aug 2023, Anthony Chan wrote:
> Hi all,
>
> My name is Tony and I've been researching/developing using Xen for potential 
> upcoming uses in our embedded systems.  I started with Xen using Xilinx tools 
> about a year ago and still have lots to learn about what it can to do in the 
> embedded space.  So far, I've managed to integrate Xen and Linux into an 
> existing product that exclusively runs bare-metal code on a ZynqMP SoC and 
> migrate some of the functionality into custom Linux driver/userspace.
>
> I'm now looking at low power support, for now at least between Xen (4.16) and 
> Linux (5.15) dom0.  I've tried a few different Linux kernel configs around 
> power management and each time I try to suspend from linux dom0 (via sysfs or 
> systemctl), Xen will watchdog on dom0 guest.  AFAIK, Xen should trap on a 
> 'WFI' from guests, but from what I can tell debugging through the linux 
> suspend process is it's spinning in a 'suspend-to-idle' loop before it can 
> get to issuing a 'WFI' or using PSCI interface to notify Xen.  I'm beginning 
> to suspect that 'low power' support for embedded arm64 just isn't quite there 
> yet, or am I missing something in the configs?
>
> I realize this could very well be a Linux 'issue' but checking here first.  I 
> know Xen presents a flattened device tree to Linux without CPU idle-state 
> nodes and maybe this is causing the linux guest to only do the 
> suspend-to-idle mode?  I should mention that I'm booting up using dom0less 
> feature if that matters.


Hi Anthony,

Assuming you are using the default Xen command line parameters for Xilinx 
boards: sched=null vwfi=native, then if the guest uses WFI, the CPU will 
execute WFI directly and go into low power mode.

Given the issue you are describing, I am suspecting the guest is not issuing 
WFI: that is simple and known to work. Instead, I suspect that Linux might be 
trying to use PSCI_suspend in a way that is not supported or well-implemented 
by Xen.

Can you check? You can add a printk in Linux 
drivers/firmware/psci/psci.c:__psci_cpu_suspend or in Xen 
xen/arch/arm/vpsci.c:do_psci_0_2_cpu_suspend

If that is the case, the attached patch might work because it disables 
PSCI_suspend support in Xen and Linux should fall back to WFI. (There is no 
power saving in using PSCI_suspend versus WFI.)

Cheers,

Stefano

CONFIDENTIALITY NOTICE: This e-mail, including any attachments, may contain 
information that is confidential and privileged. Any unauthorized disclosure, 
reproduction or use of this e-mail is prohibited. If you are not the intended 
recipient, please notify the sender by reply e-mail or telephone and 
permanently delete this e-mail and any reproductions immediately.