Re: [Xen-devel] Xen ARM Dom0less passthrough without IOMMU

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Tue, 17 Dec 2019, Andrei Cherechesu wrote:
> > On Mon, 16 Dec 2019, Julien Grall wrote:
> > > On 16/12/2019 23:05, Stefano Stabellini wrote:
> > > > On Mon, 16 Dec 2019, Julien Grall wrote:
> > > > > On 16/12/2019 18:02, Andrei Cherechesu wrote:
> > > > > But even with this patch, RAM in DomU is not direct mapped (i.e Guest
> > > > > Physical
> > > > > Address == Host Physical Address). This means that DMA-capable device
> > > > > would
> > > > > not work properly in DomU.
> > > > >
> > > > > We could theoritically map DomU direct mapped, but this would break 
> > > > > the
> > > > > isolation provided by the hypervisor.
> > > >
> > > > Yes, being able to map the DomU memory 1:1 can be pretty useful for some
> > > > very embedded dom0less configurations, in fact I was surprised that a
> > > > couple of Xilinx users asked me for that recently. Typically, the users
> > > > are aware of the consequences but they still find them better than the
> > > > alternative (i.e. the lack of isolation is bad but is tolerable in their
> > > > configuration.)
> > > This does not make much sense... The whole point of a hypervisor is to 
> > > isolate
> > > guest between each other... So if you are happy with the lack of 
> > > isolation,
> > > then why are you using an hypervisor at the first place?
> >
> > There are a number of reasons, although they are all variation of the
> > same theme. In all these cases the IOMMU cannot be used for one reason
> > or the other (a device is not behind the IOMMU, or due to an errata,
> > etc.)
> >
> > - multiple baremetal apps
> > The user wants to run two or more baremetal (unikernel-like)
> > applications. The user owns both applications and she is not much
> > concerned about isolation (although it is always desirable when
> > possible.)
> >
> > - multiple OSes
> > This is similar to the one before, however, instead of multiple
> > baremetal apps, we are talking about multiple full OSes. For instance,
> > Linux and Android or Linux and VxWorks. Again, they are both maintained
> > by the same user (no multi-tenancy) so isolation is desirable but it is
> > not the top concern.
> >
> > - real-time / no real-time
> > The user wants to run a real-time OS or real-time baremetal app and a
> > non real-time OS. For instance a tiny baremetal app controlling one
> > specific device and Linux. Again, the user is responsible for both
> > systems so isolation is not a concern.
> >
> > In all these cases the users has to run multiple OSes or baremetal apps
> > so she needs a hypervisor. However, it is tolerable that the apps are
> > not actually fully isolated from each others because they are both
> > developed and deployed together by the same "owner".
> >
> 
> Basically, since we do not have an IOMMU, we would be able
> to ensure memory isolation via a NXP IP named xRDC (Extended
> Resource Domain Controller) that our boards have, which supervises
> the access to memory buses.
> 
> But before we get to think about isolation, we need to enable
> basic passthrough functionality (via 1:1 mapping, since no IOMMU).
> 
> Firstly, a good step forward would be to get any non-DMA-capable
> device passed-through and working. 
> I rebased onto upstream/staging branch and applied the hack
> that skips the setting of XEN_DOMCTL_CDF_iommu flag,
> that Julien specified.
> 
> Then I tried to passthrough the eMMC, but I got the following
> error:
> (XEN) DOM1: [    0.879151] sdhci-esdhc-imx 4005d000.usdhc: can't request 
> region for resource [mem 0x4005d000-0x4005dfff]
> (XEN) DOM1: [    0.891137] sdhci-esdhc-imx 4005d000.usdhc: sdhci_pltfm_init 
> failed -16
> (XEN) DOM1: [    0.900249] sdhci-esdhc-imx: probe of 4005d000.usdhc failed 
> with error -16
> 
> Where 0x4005d000 is the physical address of the uSDHC(eMMC) node in the DT.
> It seems that the DomU1 kernel does not have access to that memory zone.

It looks like drivers/mmc/host/sdhci-pltfm.c:sdhci_pltfm_init failed,
but I cannot see a simple reason why it would. As Julien mentioned the
device tree snippet would be useful. Also the domU config and the full
device tree would be useful. i.e. did you add "xen,passthrough;" under
the related uSDHC node on the host device tree?


> I'm trying to passthrough the eMMC in order to mount DomU1's root
> on a SDCard partition, because I couldn't get to DomU1's Linux prompt
> when I tried to boot with a ramdisk module. I always get this error:
> (XEN) DOM1: [    1.544199] RAMDISK: Couldn't find valid RAM disk image 
> starting at 0.
> 
> Could this be because the ramdisk is too big? The smallest I've tried with
> Is approximately 60MB in size. What size are the ramdisks that you
> are using in your dom0less booting demos?

I don't think so, I could boot with ramdisk 120MB in size or even
larger. It is probably an address calculation error: it is easy to make
a small mistake in the addresses so that they end up overlapping.
Sometimes it is even U-Boot that causes the overlaps.

I would suggest to use ImageBuilder to create the U-Boot boot script to
load all the binaries and boot the system. Have a look at
uboot-script-gen in particular:

https://gitlab.com/ViryaOS/imagebuilder/blob/master/scripts/uboot-script-gen

See this wikipage on the subject:

https://wiki.xenproject.org/wiki/ImageBuilder

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel