Re: arm (qemu -M virt) 64 bit xen running 32 bit guest problem

[Julien Grall <julien@xxxxxxx>]

Hi,

On 26/08/2020 18:20, Stefano Stabellini wrote:
> On Wed, 26 Aug 2020, Charles Chiou wrote:
> > > > > Looking at the code, this seems like an issue when trying to
> > > > > translate a guest virtual address to a machine address.
> > > > >
> > > > > A few questions:
> > > > >      - Which QEMU version are you using?
> > > > >      - What's your Linux configuration? Are you using LPAE or short page
> > > tables?
> > > > I am using default ubuntu package on bionic:
> > > >
> > > > $ qemu-system-aarch64 --version
> > > > QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.29) Copyright
> > > > (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
> > > >
> > > > I just tried the stable-4.14 branch and this "Invalid MFN 0x..." doesn't
> > > happen. It was on the master branch where I encountered this problem.
> > > > However, I don't seem to get anything to call guest_printk() after making
> > > hypervisor_console_io call even for 64-bit guest in this version (worked on
> > > master). This is a separate question: could that be a XEN compile option
> > > problem as I see "debug=n" from xen register dump:
> > > > (XEN) ----[ Xen-4.14.1-pre  arm64  debug=n   Not tainted ]----
> > > The call chain is:
> > >
> > > xen/drivers/char/console.c:do_console_io
> > > xen/drivers/char/console.c:guest_console_write
> > > xen/drivers/char/console.c:guest_printk
> > >
> > > You can enable debug through kconfig by doing "make menuconfig" under
> > > xen/
> > Thank you for pointing this out! After I enabled the verbose debug messages, 
> > the hvc #0xea1 call is now taking effect.
> Great!
>
>
> > I'm seeing the previous problem of "invalid MFN" after that. It seems that the 
> > pointer 0x40000058 had been mapped to MFN 0x2be08. How do I debug or where locate the 
> > problem? I'm very new to XEN so it's no obvious yet to me from reading the source code 
> > how after guest_printk(), xen translate the addresses.
>   
> The baremetal application code and the disassembly look correct. I
> wonder if the issue is that the baremetal application is passing a guest
> physical address when Xen expects a guest virtual address.
Bear in mind that issuing an hypercall with MMU disabled is usually 
risky because of memory attribute mismatch between Xen and the Guest.
Although, you would only see data corruption and not "invalid MFN".

> The string gets copied by:
>
> xen/drivers/char/console.c:guest_console_write
>
> calling:
>
> copy_from_guest -> copy_from_guest_offset -> raw_copy_from_guest
>
> eventually it goes to:
>
> xen/arch/arm/guestcopy.c:copy_guest
> xen/arch/arm/guestcopy.c:translate_get_page
>
>
> Looking at the code, translate_get_page is called with linear=true
> write=false. linear=true causes translate_get_page to call
> get_page_from_gva. I wonder if it is possible that get_page_from_gva is
> not doing the right thing here.
When the MMU is turned off, VA == PA. The AT instruction is able to deal 
with such situation. However...
>>>> QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.29) Copyright
>>>> (c) 2003-2017 Fabrice Bellard and the QEMU Project developers


... this is fairly an old version of QEMU. I remember some issues with 
the implementation of the AT instruction in QEMU. Looking at the commit 
logs, it seems there are a few fixes that is not part of 2.11.1.
I would highly recomend to use a more recent version of QEMU in order to 
narrow down the problem.
Cheers,

--
Julien Grall