Re: arm (qemu -M virt) 64 bit xen running 32 bit guest problem

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Thu, 27 Aug 2020, Charles Chiou wrote:
> > >>>>> Looking at the code, this seems like an issue when trying to
> > >>>>> translate a guest virtual address to a machine address.
> > >>>>>
> > >>>>> A few questions:
> > >>>>>      - Which QEMU version are you using?
> > >>>>>      - What's your Linux configuration? Are you using LPAE or
> > >>>>> short page
> > >>> tables?
> > >>>>
> > >>>> I am using default ubuntu package on bionic:
> > >>>>
> > >>>> $ qemu-system-aarch64 --version
> > >>>> QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.29)
> > >>>> Copyright
> > >>>> (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
> > >>>>
> > >>>> I just tried the stable-4.14 branch and this "Invalid MFN 0x..."
> > >>>> doesn't
> > >>> happen. It was on the master branch where I encountered this problem.
> > >>>> However, I don't seem to get anything to call guest_printk() after
> > >>>> making
> > >>> hypervisor_console_io call even for 64-bit guest in this version
> > >>> (worked on master). This is a separate question: could that be a XEN
> > >>> compile option problem as I see "debug=n" from xen register dump:
> > >>>> (XEN) ----[ Xen-4.14.1-pre  arm64  debug=n   Not tainted ]----
> > >>>
> > >>> The call chain is:
> > >>>
> > >>> xen/drivers/char/console.c:do_console_io
> > >>> xen/drivers/char/console.c:guest_console_write
> > >>> xen/drivers/char/console.c:guest_printk
> > >>>
> > >>> You can enable debug through kconfig by doing "make menuconfig"
> > >>> under xen/
> > >>
> > >> Thank you for pointing this out! After I enabled the verbose debug
> > messages, the hvc #0xea1 call is now taking effect.
> > >
> > > Great!
> > >
> > >
> > >> I'm seeing the previous problem of "invalid MFN" after that. It seems 
> > >> that
> > the pointer 0x40000058 had been mapped to MFN 0x2be08. How do I debug
> > or where locate the problem? I'm very new to XEN so it's no obvious yet to
> > me from reading the source code how after guest_printk(), xen translate the
> > addresses.
> > >
> > > The baremetal application code and the disassembly look correct. I
> > > wonder if the issue is that the baremetal application is passing a
> > > guest physical address when Xen expects a guest virtual address.
> >
> > Bear in mind that issuing an hypercall with MMU disabled is usually risky
> > because of memory attribute mismatch between Xen and the Guest.
> >
> > Although, you would only see data corruption and not "invalid MFN".
> >
> > >
> > > The string gets copied by:
> > >
> > > xen/drivers/char/console.c:guest_console_write
> > >
> > > calling:
> > >
> > > copy_from_guest -> copy_from_guest_offset -> raw_copy_from_guest
> > >
> > > eventually it goes to:
> > >
> > > xen/arch/arm/guestcopy.c:copy_guest
> > > xen/arch/arm/guestcopy.c:translate_get_page
> > >
> > >
> > > Looking at the code, translate_get_page is called with linear=true
> > > write=false. linear=true causes translate_get_page to call
> > > get_page_from_gva. I wonder if it is possible that get_page_from_gva
> > > is not doing the right thing here.
> >
> > When the MMU is turned off, VA == PA. The AT instruction is able to deal
> > with such situation. However...
> 
> Hi Stefano, thank you for the pointers. It was very helpful to get started in 
> tracing the internals of xen.

You are welcome :-)


> >  >>>> QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.29)
> > Copyright  >>>> (c) 2003-2017 Fabrice Bellard and the QEMU Project
> > developers
> >
> >
> > ... this is fairly an old version of QEMU. I remember some issues with the
> > implementation of the AT instruction in QEMU. Looking at the commit logs, it
> > seems there are a few fixes that is not part of 2.11.1.
> >
> > I would highly recomend to use a more recent version of QEMU in order to
> > narrow down the problem.
> 
> Hi Julien, switching qemu to v3.1.1.1 solved the problem! (Tried 5.x but ran 
> into other problems, but those are something else). The address translation 
> now seems ok now and gvirt_to_maddr() is returning the right values after 
> executing "at s12e1w, %0", and guest console output is working. Thank you!
 
Great to hear!