|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] Xen ARM Dom0less passthrough without IOMMU
Hi Andrei, On 17/12/2019 17:20, Andrei Cherechesu wrote: On Mon, 16 Dec 2019, Julien Grall wrote:On 16/12/2019 23:05, Stefano Stabellini wrote:On Mon, 16 Dec 2019, Julien Grall wrote:On 16/12/2019 18:02, Andrei Cherechesu wrote: But even with this patch, RAM in DomU is not direct mapped (i.e Guest Physical Address == Host Physical Address). This means that DMA-capable device would not work properly in DomU. We could theoritically map DomU direct mapped, but this would break the isolation provided by the hypervisor.Yes, being able to map the DomU memory 1:1 can be pretty useful for some very embedded dom0less configurations, in fact I was surprised that a couple of Xilinx users asked me for that recently. Typically, the users are aware of the consequences but they still find them better than the alternative (i.e. the lack of isolation is bad but is tolerable in their configuration.)This does not make much sense... The whole point of a hypervisor is to isolate guest between each other... So if you are happy with the lack of isolation, then why are you using an hypervisor at the first place?There are a number of reasons, although they are all variation of the same theme. In all these cases the IOMMU cannot be used for one reason or the other (a device is not behind the IOMMU, or due to an errata, etc.) - multiple baremetal apps The user wants to run two or more baremetal (unikernel-like) applications. The user owns both applications and she is not much concerned about isolation (although it is always desirable when possible.) - multiple OSes This is similar to the one before, however, instead of multiple baremetal apps, we are talking about multiple full OSes. For instance, Linux and Android or Linux and VxWorks. Again, they are both maintained by the same user (no multi-tenancy) so isolation is desirable but it is not the top concern. - real-time / no real-time The user wants to run a real-time OS or real-time baremetal app and a non real-time OS. For instance a tiny baremetal app controlling one specific device and Linux. Again, the user is responsible for both systems so isolation is not a concern. In all these cases the users has to run multiple OSes or baremetal apps so she needs a hypervisor. However, it is tolerable that the apps are not actually fully isolated from each others because they are both developed and deployed together by the same "owner".Basically, since we do not have an IOMMU, we would be able to ensure memory isolation via a NXP IP named xRDC (Extended Resource Domain Controller) that our boards have, which supervises the access to memory buses. Ok, so you have some sort of MPU. I assume this will be between the devices and the memory, am I right? But before we get to think about isolation, we need to enable basic passthrough functionality (via 1:1 mapping, since no IOMMU). So you are in better place than what Stefano described. Your use case is probably the only place where a 1:1 mapping would be warrant as isolation is still provided by the HW. Firstly, a good step forward would be to get any non-DMA-capable device passed-through and working. I rebased onto upstream/staging branch and applied the hack that skips the setting of XEN_DOMCTL_CDF_iommu flag, that Julien specified. Then I tried to passthrough the eMMC, but I got the following error: (XEN) DOM1: [ 0.879151] sdhci-esdhc-imx 4005d000.usdhc: can't request region for resource [mem 0x4005d000-0x4005dfff] (XEN) DOM1: [ 0.891137] sdhci-esdhc-imx 4005d000.usdhc: sdhci_pltfm_init failed -16 (XEN) DOM1: [ 0.900249] sdhci-esdhc-imx: probe of 4005d000.usdhc failed with error -16 Where 0x4005d000 is the physical address of the uSDHC(eMMC) node in the DT. It seems that the DomU1 kernel does not have access to that memory zone. Could you paste your partial Device-Tree and domain node? I'm trying to passthrough the eMMC in order to mount DomU1's root on a SDCard partition, because I couldn't get to DomU1's Linux prompt when I tried to boot with a ramdisk module. I always get this error: (XEN) DOM1: [ 1.544199] RAMDISK: Couldn't find valid RAM disk image starting at 0. How did you pass the ramdisk to dom1? Could this be because the ramdisk is too big? The smallest I've tried with Is approximately 60MB in size. What size are the ramdisks that you are using in your dom0less booting demos? How much memory did you give to your guest? [...] I'll gladly write the patch if you give me some basic instructions regarding it, because I'm not that familiar with all the Xen internal mechanisms, and I wouldn't know where to look in order to ensure everything is properly done. I am going to suggest a quick and dirty way but it should get you to the point where 1:1 mapping will work in basic use case: 1) Update the guest memory map in xen/include/public/arch-arm.h (see GUEST_*) so all the regions don't overlap your RAM. The best way would be to re-use the same address for the GIC and UART as the host. 2) Allocate the memory bank 1:1. The patch below should do the job (not compiled nor tested):
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index dd9c3b73ba..9155728640 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -374,6 +374,7 @@ static void __init allocate_memory_11(struct domain *d,
}
}
+#if 0
static bool __init allocate_bank_memory(struct domain *d,
struct kernel_info *kinfo,
gfn_t sgfn,
@@ -472,6 +473,7 @@ fail:
" %ldKB unallocated. Fix the VMs configurations.\n",
(unsigned long)kinfo->unassigned_mem >> 10);
}
+#endif
static int __init write_properties(struct domain *d, struct
kernel_info *kinfo,
const struct dt_device_node *node)
@@ -2434,7 +2436,7 @@ static int __init construct_domU(struct domain *d,
/* type must be set before allocate memory */
d->arch.type = kinfo.type;
#endif
- allocate_memory(d, &kinfo);
+ allocate_memory_11(d, &kinfo);
rc = prepare_dtb_domU(d, &kinfo);
if ( rc < 0 )
If you want to use Xen hypercalls such as for ballooning, you can look
at the macro is_domain_direct_mapped(). This will most likely require
change in your guest OS because any hypercall modifying memory will not
keep the 1:1 assumption. Therefore the OS has to keep track of it. This
is a bit tedious but already exists in Linux as we use it for Dom0.
For a more generic approach, you would need to replace the static version suggested 1) with a more dynamic layout that could be based on the host memory layout. Best regards, -- Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxxx https://lists.xenproject.org/mailman/listinfo/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |