Re: [Xen-devel] Xen-unstable PVHdom0: Assertion 'IS_ALIGNED(dfn_x(dfn), (1ul << page_order))' failed at iommu.c:324

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

On 18/01/2019 18:56, Roger Pau Monné wrote:
> On Fri, Jan 18, 2019 at 03:17:57PM +0100, Sander Eikelenboom wrote:
>> On 18/01/2019 13:50, Roger Pau Monné wrote:
>>> On Fri, Jan 18, 2019 at 01:03:04PM +0100, Sander Eikelenboom wrote:
>>>> Hi Roger,
>>>>
>>>> I gave PVH dom0 a spin, see how far I would get.
>>>
>>> Thanks!
>>>
>>>> With current xen-unstable unfortunately not that far, i got the splat 
>>>> below.
>>>
>>> Yes, this was already reported:
>>>
>>> https://lists.xenproject.org/archives/html/xen-devel/2019-01/msg01030.html
>>>> If you need more info, would like me to test a patch (or some other git 
>>>> tree/branch), 
>>>> I will be happy to give it a spin !
>>>
>>> Paul is working on a fix, but in the meantime just removing the
>>> assertions should be fine:
>>>
>>> ---8<---
>>> diff --git a/xen/drivers/passthrough/iommu.c 
>>> b/xen/drivers/passthrough/iommu.c
>>> index bd1af35a13..98e6fc35e2 100644
>>> --- a/xen/drivers/passthrough/iommu.c
>>> +++ b/xen/drivers/passthrough/iommu.c
>>> @@ -321,9 +321,6 @@ int iommu_map(struct domain *d, dfn_t dfn, mfn_t mfn,
>>>      if ( !iommu_enabled || !hd->platform_ops )
>>>          return 0;
>>>  
>>> -    ASSERT(IS_ALIGNED(dfn_x(dfn), (1ul << page_order)));
>>> -    ASSERT(IS_ALIGNED(mfn_x(mfn), (1ul << page_order)));
>>> -
>>>      for ( i = 0; i < (1ul << page_order); i++ )
>>>      {
>>>          rc = hd->platform_ops->map_page(d, dfn_add(dfn, i), mfn_add(mfn, 
>>> i),
>>>
>>
>> I gave that a spin and i now get a seemingly endless stream of IO_PAGE_FAULTs
> 
> You shouldn't get those page faults since they are for addresses that
> belong to a reserved region, and that should be mapped into the p2m.
> I've just tested on my AMD box and I'm also seeing errors (albeit
> different ones), so I guess something broke since I last fixed PVH
> Dom0 to boot on AMD hardware.
> 
> I've also tested commit:
> 
> commit fad6ba64a8c98bebb9374f390cc255fac05237ab (HEAD)
> Author: Roger Pau Monné <roger.pau@xxxxxxxxxx>
> Date:   Fri Nov 30 12:10:00 2018 +0100
> amd/iommu: skip host bridge devices when updating IOMMU page tables
> 
> And it works on my AMD box and I'm able to boot as a PVH Dom0. Can you
> give this commit a spin?
> 
> Thanks, Roger.
> 

Hi Roger,

Tested that commit, but that didn't help.

I added some debug logging (to xen-unstable + Paul's patch) and found out the 
devices that are giving
the IO_PAGE_FAULTs are the onboard USB controllers (0x90 0x92 0x98 0xa5).

If I skip calling "amd_iommu_setup_domain_device" for these devices I can at 
least get it to boot a linux kernel 
(which than gives problems with the SATA controller, but that would be a next 
step). The patch I used is below.

I attached the output from lspci -vvvknn, perhaps you can spot something odd ?
When booting dom0 as PV instead of PVH everything boots and works fine.

--
Sander




diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c 
b/xen/drivers/passthrough/amd/pci_amd_iommu.c
index 33a3798f36..cc82c4b08d 100644
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ @@ static int amd_iommu_add_device(u8 devfn, struct pci_dev *pdev)
         return -ENODEV;
     }

+    if (PCI_SLOT(devfn) == 0x12 || PCI_SLOT(devfn) == 0x13 || PCI_SLOT(devfn) 
== 0x16 || ( PCI_SLOT(devfn) == 0x14 && PCI_FUNC(devfn) == 5) ){
+        AMD_IOMMU_DEBUG("%s ?!?!? SKIPPING %d/%d %04x:%02x:%02x.%u\n",
+                            __func__, pdev->domain->domain_id, 
is_hardware_domain(pdev->domain) ? 1 : 0, pdev->seg, pdev->bus, PCI_SLOT(devfn),
+                            PCI_FUNC(devfn));
+        
+       return 0;
+    }
+
     amd_iommu_setup_domain_device(pdev->domain, iommu, devfn, pdev);
     return 0;
 }

Attachment: lspci-vvvknn.txt
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel