|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: Question about xen and Rasp 4B
On Sat, Jan 30, 2021 at 5:53 AM Jukka Kaartinen <jukka.kaartinen@xxxxxxxxxx> wrote: > > On 27/01/2021 11:47, Jukka Kaartinen wrote: > >> > >> > >> On Tue, Jan 26, 2021 at 10:22 PM Stefano Stabellini > >> <sstabellini@xxxxxxxxxx <mailto:sstabellini@xxxxxxxxxx>> wrote: > >> > >> On Tue, 26 Jan 2021, Jukka Kaartinen wrote: > >> > On Tue, Jan 26, 2021 at 2:54 AM Stefano Stabellini > >> <sstabellini@xxxxxxxxxx <mailto:sstabellini@xxxxxxxxxx>> wrote: > >> > On Sat, 23 Jan 2021, Jukka Kaartinen wrote: > >> > > Thanks for the response! > >> > > > >> > > On Sat, Jan 23, 2021 at 2:27 AM Stefano Stabellini > >> <sstabellini@xxxxxxxxxx <mailto:sstabellini@xxxxxxxxxx>> wrote: > >> > > + xen-devel, Roman, > >> > > > >> > > > >> > > On Fri, 22 Jan 2021, Jukka Kaartinen wrote: > >> > > > Hi Stefano, > >> > > > I'm Jukka Kaartinen a SW developer working on > >> enabling hypervisors on mobile platforms. One of our HW that we > >> use on > >> > > development is > >> > > > Raspberry Pi 4B. I wonder if you could help me a > >> bit :). > >> > > > > >> > > > I'm trying to enable the GPU with Xen + Raspberry > >> Pi for > >> > > dom0. > >> > >> https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=232323#p1797605 > >> > >> <https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=232323#p1797605> > >> > > > > >> > > > I got so far that GPU drivers are loaded (v3d & > >> vc4) without errors. But now Xen returns error when X is starting: > >> > > > (XEN) traps.c:1986:d0v1 HSR=0x93880045 > >> pc=0x00007f97b14e70 gva=0x7f7f817000 gpa=0x0000401315d000 > >> > > > I tried to debug what causes this and looks > >> like find_mmio_handler cannot find handler. > >> > > > (See more here: > >> > >> https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=232323&start=25#p1801691 > >> > >> > >> <https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=232323&start=25#p1801691> > >> > >> ) > >> > > > > >> > > > Any ideas why the handler is not found? > >> > > > >> > > > >> > > Hi Jukka, > >> > > > >> > > I am glad to hear that you are interested in Xen on > >> RaspberryPi :-) I > >> > > haven't tried the GPU yet, I have been using the > >> serial only. > >> > > Roman, did you ever get the GPU working? > >> > > > >> > > > >> > > The error is a data abort error: Linux is trying to > >> access an address > >> > > which is not mapped to dom0. The address seems to > >> be 0x401315d000. It is > >> > > a pretty high address; I looked in device tree but > >> couldn't spot it. > >> > > > >> > > >From the HSR (the syndrom register) it looks like > >> it is a translation > >> > > fault at EL1 on stage1. As if the Linux address > >> mapping was wrong. > >> > > Anyone has any ideas how this could happen? Maybe a > >> reserved-memory > >> > > misconfiguration? > >> > > > >> > > I had issues with loading the driver in the first place. > >> Apparently swiotlb is used, maybe it can cause this. I also tried to > >> > enable CMA. > >> > > config.txt: > >> > > dtoverlay=vc4-fkms-v3d,cma=320M@0x0-0x40000000 > >> > > gpu_mem=128 > >> > > >> > Also looking at your other reply and the implementation of > >> > vc4_bo_create, it looks like this is a CMA problem. > >> > > >> > It would be good to run a test with the swiotlb-xen > >> disabled: > >> > > >> > diff --git a/arch/arm/xen/mm.c b/arch/arm/xen/mm.c > >> > index 467fa225c3d0..2bdd12785d14 100644 > >> > --- a/arch/arm/xen/mm.c > >> > +++ b/arch/arm/xen/mm.c > >> > @@ -138,8 +138,7 @@ void > >> xen_destroy_contiguous_region(phys_addr_t pstart, unsigned int order) > >> > static int __init xen_mm_init(void) > >> > { > >> > struct gnttab_cache_flush cflush; > >> > - if (!xen_initial_domain()) > >> > - return 0; > >> > + return 0; > >> > xen_swiotlb_init(1, false); > >> > > >> > cflush.op = 0; > >> > > >> > With this change the kernel is not booting up. (btw. I'm using > >> USB SSD for my OS.) > >> > [ 0.071081] bcm2835-dma fe007000.dma: Unable to set DMA mask > >> > [ 0.076277] bcm2835-dma fe007b00.dma: Unable to set DMA mask > >> > (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=25: not implemented > >> > (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=15: not implemented > >> > [ 0.592695] pci 0000:00:00.0: Failed to add - passthrough or > >> MSI/MSI-X might fail! > >> > (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=15: not implemented > >> > [ 0.606819] pci 0000:01:00.0: Failed to add - passthrough or > >> MSI/MSI-X might fail! > >> > [ 1.212820] usb 1-1: device descriptor read/64, error 18 > >> > [ 1.452815] usb 1-1: device descriptor read/64, error 18 > >> > [ 1.820813] usb 1-1: device descriptor read/64, error 18 > >> > [ 2.060815] usb 1-1: device descriptor read/64, error 18 > >> > [ 2.845548] usb 1-1: device descriptor read/8, error -61 > >> > [ 2.977603] usb 1-1: device descriptor read/8, error -61 > >> > [ 3.237530] usb 1-1: device descriptor read/8, error -61 > >> > [ 3.369585] usb 1-1: device descriptor read/8, error -61 > >> > [ 3.480765] usb usb1-port1: unable to enumerate USB device > >> > > >> > Traces stop here. I could try with a memory card. Maybe it makes > >> a difference. > >> > >> This is very surprising. Disabling swiotlb-xen should make things > >> better > >> not worse. The only reason I can think of why it could make things > >> worse > >> is if Linux runs out of low memory. Julien's patch > >> 437b0aa06a014ce174e24c0d3530b3e9ab19b18b for Xen should have > >> addressed > >> that issue though. Julien, any ideas? > > > > I think, Stefano's small patch is not enough to disable the swiotlb as > > we will still override the DMA ops. You also likely want: > > > > diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c > > index 8a8949174b1c..aa43e249ecdd 100644 > > --- a/arch/arm/mm/dma-mapping.c > > +++ b/arch/arm/mm/dma-mapping.c > > @@ -2279,7 +2279,7 @@ void arch_setup_dma_ops(struct device *dev, u64 > > dma_base, u64 size, > > set_dma_ops(dev, dma_ops); > > > > #ifdef CONFIG_XEN > > - if (xen_initial_domain()) > > + if (0 || xen_initial_domain()) > > dev->dma_ops = &xen_swiotlb_dma_ops; > > #endif > > dev->archdata.dma_ops_setup = true; > > > > Otherwise, you would still use the swiotlb DMA ops that would not be > > functional as we disabled the swiotlb. > > > > This would explain the following error because it will check whether the > > mask is valid using the callback dma_supported(): > > > > [ 0.071081] bcm2835-dma fe007000.dma: Unable to set DMA mask > > > Good catch. > GPU works now and I can start X! Thanks! I was also able to create domU > that runs Raspian OS. This is very interesting that you were able to achieve that - congrats! Now, sorry to be a bit dense -- but since this thread went into all sorts of interesting directions all at once -- I just have a very particular question: what is exact combination of versions of Xen, Linux kernel and a set of patches that went on top that allowed you to do that? I'd love to obviously see it productized in Xen upstream, but for now -- I'd love to make available to Project EVE/Xen community since there seems to be a few folks interested in EVE/Xen combo being able to do that. Thanks, Roman.
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |