Re: exynos-mixer 14450000.mixer: [drm:exynos_drm_register_dma] *ERROR* Device 14450000.mixer lacks support for IOMMU

[Chuck Zmudzinski <brchuckz@xxxxxxxxxxxx>]

On 11/1/2023 4:27 AM, Julien Grall wrote:
> Hi,
> 
> @Stefano, as you pointed out, there is already a thread on xen-users for 
> this discussion. Could we use this thread for any discussion? This would 
> make easier to follow.
> 
> Some high level comment below.

I agree to keep the discussion here and not at other places.

I just want to add that the best results for Xen dom0 so far are
by implementing Marek's suggestion to disable these two settings
in the 6.1.59 kernel config, but leaving everything else the same,
including keeping the EXYNOS_IOMMU support enabled:

# CONFIG_DRM_EXYNOS_MIXER is not set

Disabling the mixer also makes this unavailable:

# CONFIG_HDMI

With this change, the GPU is working well enough to allow the display
manager and an X11 session to run normally on the built-in display of the
Chromebook. The Wifi also works well.

The patch from Linux 6.2 and above that fixes the exynos IOMMU initialization
did not help at all, and the same error is reported that the mixer lacks
support for IOMMU.


> 
> On 01/11/2023 02:50, Chuck Zmudzinski wrote:
>> On 10/31/2023 7:45 PM, Stefano Stabellini wrote:
>>> Unfortunately there is no easy solution.
>>>
>>> Do you know the version of the SMMU available on the platform?
>> 
>> I am trying to discern, but I doubt we have v3 because we are
>> working on a very old chromebook from 2012, and I am finding
>> patches for smmv3 in Linux not starting until 2015. It is good to
>> know about this option, though, for future work we might do on newer
>> devices.
> 
> The chromebook is using the Exynos Sysmmu. So there is no SMMU.
> 
>> 
>>> If it is a SMMUv3 you can try to use the nested SMMU patch series to
>>> enable a virtual SMMU in Dom0: 
>>> https://marc.info/?l=xen-devel&m=166991020831005
>>> That way, Xen can use the SMMU to protect VMs, and Dom0 can also use the
>>> SMMU for its own purposes at the same time.
>>>
>>> Alternatively, you can dig into the details of the exynos-drm driver to
>>> see what exactly is the dependency on the IOMMU framework in Linux and
>>> remove the dependency. Unfortunately none of us in this thread are
>>> expert on exynos-drm so it would be difficult to advise on how to do
>>> that. For example, I don't know how you could debug the x11 problem you
>>> described because I don't typically work with x11 or with the exynos. If
>>> there is an open source mailing list for exynos-drm development they
>>> might be able to advise on how to remove the IOMMU dependency there.
>> 
>> We have received this message from Marek Szyprowski of Samsung:
>> 
>> https://lore.kernel.org/lkml/7a71e348-f892-4fd4-8857-b72f35ab5134@xxxxxxxxxxx/
>> 
>> Marek recommends this patch to possibly help with this issue:
>> 
>> https://github.com/torvalds/linux/commit/bbc4d205d93f52ee18dfa7858d51489c0506547f
>> 
>> and also these kernel config settings:
>> 
>> On 10/31/2023 8:08 AM, Marek Szyprowski wrote:
>>> If not, then as a temporary workaround please disable
>>> CONFIG_DRM_EXYNOS_MIXER and CONFIG_DRM_EXYNOS_HDMI in your kernel config
>>> and check what will happen (You will lose the HDMI output, but maybe
>>> this won't a big issue).
>> 
>> Mario and I have preliminary evidence that applying both of Marek's
>> recommendations to the 6.1.59 kernel have improved the situation to
>> the point where now the Chromebook can run X.org on Xen. We are doing
>> further tests to see how Marek's patch and/or the kernel config settings
>> to disable the mixer and the HDMI affect the behavior of the GPU in dom0
>> on Xen.
>> 
>>>
>>> The final option, which is a gross hack, would be to let Dom0 use the
>>> IOMMU for its own gain. Xen doesn't use the IOMMU. If you do that you
>>> lose freedom from interference between the VMs and you cannot run driver
>>> domains or directly assign devices to DomUs. But if you are running a
>>> research project you might be OK with that. To get it to work, you need
>>> to hack Xen so that it remaps the IOMMU to Dom0 to let Dom0 program it
>>> directly. The attached patch (untested) would be a place to start. You
>>> also need to pass iommu=false to the Xen command line to prevent Xen
>>> from using the iommu itself.
> 
> This is actually one of the reason why I am suggesting to do all the 
> investigation in one thread. There, we already discovered that the IOMMU 
> was assigned to dom0 because Xen doesn't have a driver and we don't hide 
> them by default.
> 
>> 
>> I am interested to investigate if only the mixer and the HDMI is causing
>> the trouble. Based on what you are telling me about Xen not exposing the
>> IOMMU to dom0, I don't fully understand the original log messages I was
>> getting when I followed Julien's suggestion to find the symbols associated
>> with each address in the original message, and those seemed to indicate that
>> the exynos_drm device was using the IOMMU in dom0, but the mixer was not,
>> and the fact that they both were not using the IOMMU is what caused the
>> test to fail and Linux refuse to initialize the GPU on dom0, whereas on
>> bare metal, the logs indicated both the exynos mixer, which I think is a
>> sub device of the exynos_drm, and the exynos_drm, use the IOMMU on bare
>> metal.
>> 
>> I also found this patch which suggests if we can get the devices to work
>> we will be compromising the security and isolation between guests:
> 
> If you don't assign any device to the guests, then you will not break 
> any isolation between guests because dom0 will own all of them.
> 
> But for passthrough, you would want to the IOMMU owned by Xen rather 
> than dom0. Unless the Exynos sysmmu support 2-stages page-tables, then 
> dom0 will not be able to use the IOMMU.
> 
> Cheers,
>