Re: arm (qemu -M virt) 64 bit xen running 32 bit guest problem

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Wed, 26 Aug 2020, Charles Chiou wrote:
> > > > Looking at the code, this seems like an issue when trying to
> > > > translate a guest virtual address to a machine address.
> > > >
> > > > A few questions:
> > > >     - Which QEMU version are you using?
> > > >     - What's your Linux configuration? Are you using LPAE or short page
> > tables?
> > >
> > > I am using default ubuntu package on bionic:
> > >
> > > $ qemu-system-aarch64 --version
> > > QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.29) Copyright
> > > (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
> > >
> > > I just tried the stable-4.14 branch and this "Invalid MFN 0x..." doesn't
> > happen. It was on the master branch where I encountered this problem.
> > > However, I don't seem to get anything to call guest_printk() after making
> > hypervisor_console_io call even for 64-bit guest in this version (worked on
> > master). This is a separate question: could that be a XEN compile option
> > problem as I see "debug=n" from xen register dump:
> > > (XEN) ----[ Xen-4.14.1-pre  arm64  debug=n   Not tainted ]----
> >
> > The call chain is:
> >
> > xen/drivers/char/console.c:do_console_io
> > xen/drivers/char/console.c:guest_console_write
> > xen/drivers/char/console.c:guest_printk
> >
> > You can enable debug through kconfig by doing "make menuconfig" under
> > xen/
> 
> Thank you for pointing this out! After I enabled the verbose debug messages, 
> the hvc #0xea1 call is now taking effect.

Great!


>I'm seeing the previous problem of "invalid MFN" after that. It seems that the 
>pointer 0x40000058 had been mapped to MFN 0x2be08. How do I debug or where 
>locate the problem? I'm very new to XEN so it's no obvious yet to me from 
>reading the source code how after guest_printk(), xen translate the addresses.
 
The baremetal application code and the disassembly look correct. I
wonder if the issue is that the baremetal application is passing a guest
physical address when Xen expects a guest virtual address.

The string gets copied by:

xen/drivers/char/console.c:guest_console_write

calling:

copy_from_guest -> copy_from_guest_offset -> raw_copy_from_guest

eventually it goes to:

xen/arch/arm/guestcopy.c:copy_guest
xen/arch/arm/guestcopy.c:translate_get_page


Looking at the code, translate_get_page is called with linear=true
write=false. linear=true causes translate_get_page to call
get_page_from_gva. I wonder if it is possible that get_page_from_gva is
not doing the right thing here. 

As a test, maybe you could hack guest_console_write to call a modified
version of raw_copy_from_guest that uses guest physical addresses
instead.



diff --git a/xen/arch/arm/guestcopy.c b/xen/arch/arm/guestcopy.c
index 7a0f3e9d5f..106a95e33f 100644
--- a/xen/arch/arm/guestcopy.c
+++ b/xen/arch/arm/guestcopy.c
@@ -130,6 +130,11 @@ unsigned long raw_copy_from_guest(void *to, const void 
__user *from, unsigned le
                       COPY_from_guest | COPY_linear);
 }
 
+unsigned long raw_copy_from_guest_special(void *to, const void __user *from, 
unsigned len)
+{
+    return copy_guest(to, (uint64_t)from, len, GPA_INFO(current->domain),
+                      COPY_from_guest);
+}
 unsigned long copy_to_guest_phys_flush_dcache(struct domain *d,
                                               paddr_t gpa,
                                               void *buf,
diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index 861ad53a8f..2dd6187aa5 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -590,6 +590,7 @@ static inline void xen_console_write_debug_port(const char 
*buf, size_t len)
 }
 #endif
 
+extern unsigned long raw_copy_from_guest_special(void *to, const void __user 
*from, unsigned len);
 static long guest_console_write(XEN_GUEST_HANDLE_PARAM(char) buffer,
                                 unsigned int count)
 {
@@ -604,9 +605,18 @@ static long 
guest_console_write(XEN_GUEST_HANDLE_PARAM(char) buffer,
                 __HYPERVISOR_console_io, "iih",
                 CONSOLEIO_write, count, buffer);
 
-        kcount = min((size_t)count, sizeof(kbuf) - 1);
-        if ( copy_from_guest(kbuf, buffer, kcount) )
-            return -EFAULT;
+        if ( current->domain->domain_id > 0 )
+        {
+            kcount = min((size_t)count, sizeof(kbuf) - 1);
+            if ( raw_copy_from_guest_special(kbuf, buffer.p, kcount) )
+                return -EFAULT;
+        }
+        else
+        {
+            kcount = min((size_t)count, sizeof(kbuf) - 1);
+            if ( copy_from_guest(kbuf, buffer, kcount) )
+                return -EFAULT;
+        }
 
         if ( is_hardware_domain(cd) )
         {