RE: [EXT] Re: xen arm64 low power sleep support

[Anthony Chan <anthonychan@xxxxxxxxxx>]

On Tue, 29 Aug 2023, Stefano Stabellini wrote:
> On Tue, 29 Aug 2023, Anthony Chan wrote:
> > Hi all,
> >
> > My name is Tony and I've been researching/developing using Xen for
> potential upcoming uses in our embedded systems.  I started with Xen
> using Xilinx tools about a year ago and still have lots to learn about what it
> can to do in the embedded space.  So far, I've managed to integrate Xen
> and Linux into an existing product that exclusively runs bare-metal code on
> a ZynqMP SoC and migrate some of the functionality into custom Linux
> driver/userspace.
> >
> > I'm now looking at low power support, for now at least between Xen
> (4.16) and Linux (5.15) dom0.  I've tried a few different Linux kernel
> configs around power management and each time I try to suspend from
> linux dom0 (via sysfs or systemctl), Xen will watchdog on dom0 guest.
> AFAIK, Xen should trap on a 'WFI' from guests, but from what I can tell
> debugging through the linux suspend process is it's spinning in a 'suspend-
> to-idle' loop before it can get to issuing a 'WFI' or using PSCI interface to
> notify Xen.  I'm beginning to suspect that 'low power' support for
> embedded arm64 just isn't quite there yet, or am I missing something in
> the configs?
> >
> > I realize this could very well be a Linux 'issue' but checking here first.  
> > I
> know Xen presents a flattened device tree to Linux without CPU idle-state
> nodes and maybe this is causing the linux guest to only do the suspend-
> to-idle mode?  I should mention that I'm booting up using dom0less
> feature if that matters.
>
>
> Hi Anthony,
>
> Assuming you are using the default Xen command line parameters for
> Xilinx boards: sched=null vwfi=native, then if the guest uses WFI, the CPU
> will execute WFI directly and go into low power mode.
Yes, using these command line params.

> Given the issue you are describing, I am suspecting the guest is not issuing
> WFI: that is simple and known to work. Instead, I suspect that Linux might
> be trying to use PSCI_suspend in a way that is not supported or well-
> implemented by Xen.
>
> Can you check? You can add a printk in Linux
> drivers/firmware/psci/psci.c:__psci_cpu_suspend or in Xen
> xen/arch/arm/vpsci.c:do_psci_0_2_cpu_suspend
Instrumented both places it doesn't appear to reach there.  In 
kernel/power/suspend.c, there's a call to s2idle_loop that it's currently 
'stuck' in and I think it doesn't get to the psci suspend your referring till 
afterwards, when suspend_ops->enter is called.  Unfortunately, without any 
idle-states nodes in the FDT, the only suspend state Linux is defaults to is 
'suspend to idle'.

Sorry about the boilerplate confidentiality footer below, I am not allowed to 
disable it...



CONFIDENTIALITY NOTICE: This e-mail, including any attachments, may contain 
information that is confidential and privileged. Any unauthorized disclosure, 
reproduction or use of this e-mail is prohibited. If you are not the intended 
recipient, please notify the sender by reply e-mail or telephone and 
permanently delete this e-mail and any reproductions immediately.