Re: [Xen-devel] Xen-unstable PVHdom0: Assertion 'IS_ALIGNED(dfn_x(dfn), (1ul << page_order))' failed at iommu.c:324

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

On 25/01/2019 15:38, Roger Pau Monné wrote:
> On Thu, Jan 24, 2019 at 01:04:31PM +0100, Roger Pau Monné wrote:
>> On Thu, Jan 24, 2019 at 12:55:06PM +0100, Sander Eikelenboom wrote:
>>> On 24/01/2019 11:11, Roger Pau Monné wrote:
>>>> On Thu, Jan 24, 2019 at 10:25:33AM +0100, Sander Eikelenboom wrote:
>>>>> On 24/01/2019 08:50, Roger Pau Monné wrote:
>>>>>> On Wed, Jan 23, 2019 at 08:56:48PM +0100, Sander Eikelenboom wrote:
>>>>>>> On 23/01/2019 19:25, Roger Pau Monné wrote:
>>>>>>>> On Wed, Jan 23, 2019 at 12:39:21AM +0100, Sander Eikelenboom wrote:
>>>>>>>>> On 22/01/2019 17:14, Roger Pau Monné wrote:
>>>>>>>>>> On Sun, Jan 20, 2019 at 11:09:25PM +0100, Sander Eikelenboom wrote:
>>>>>>>>>>> On 18/01/2019 18:56, Roger Pau Monné wrote:
>>>>>>>>>>>> On Fri, Jan 18, 2019 at 03:17:57PM +0100, Sander Eikelenboom wrote:
>>>>>>>>>>>>> On 18/01/2019 13:50, Roger Pau Monné wrote:
>>>>>>>>>>>>>> On Fri, Jan 18, 2019 at 01:03:04PM +0100, Sander Eikelenboom 
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> Hi Roger,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I gave PVH dom0 a spin, see how far I would get.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> With current xen-unstable unfortunately not that far, i got the 
>>>>>>>>>>>>>>> splat below.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes, this was already reported:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://lists.xenproject.org/archives/html/xen-devel/2019-01/msg01030.html
>>>>>>>>>>>>>>> If you need more info, would like me to test a patch (or some 
>>>>>>>>>>>>>>> other git tree/branch), 
>>>>>>>>>>>>>>> I will be happy to give it a spin !
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Paul is working on a fix, but in the meantime just removing the
>>>>>>>>>>>>>> assertions should be fine:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ---8<---
>>>>>>>>>>>>>> diff --git a/xen/drivers/passthrough/iommu.c 
>>>>>>>>>>>>>> b/xen/drivers/passthrough/iommu.c
>>>>>>>>>>>>>> index bd1af35a13..98e6fc35e2 100644
>>>>>>>>>>>>>> --- a/xen/drivers/passthrough/iommu.c
>>>>>>>>>>>>>> +++ b/xen/drivers/passthrough/iommu.c
>>>>>>>>>>>>>> @@ -321,9 +321,6 @@ int iommu_map(struct domain *d, dfn_t dfn, 
>>>>>>>>>>>>>> mfn_t mfn,
>>>>>>>>>>>>>>      if ( !iommu_enabled || !hd->platform_ops )
>>>>>>>>>>>>>>          return 0;
>>>>>>>>>>>>>>  
>>>>>>>>>>>>>> -    ASSERT(IS_ALIGNED(dfn_x(dfn), (1ul << page_order)));
>>>>>>>>>>>>>> -    ASSERT(IS_ALIGNED(mfn_x(mfn), (1ul << page_order)));
>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>      for ( i = 0; i < (1ul << page_order); i++ )
>>>>>>>>>>>>>>      {
>>>>>>>>>>>>>>          rc = hd->platform_ops->map_page(d, dfn_add(dfn, i), 
>>>>>>>>>>>>>> mfn_add(mfn, i),
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I gave that a spin and i now get a seemingly endless stream of 
>>>>>>>>>>>>> IO_PAGE_FAULTs
>>>>>>>>>>>>
>>>>>>>>>>>> You shouldn't get those page faults since they are for addresses 
>>>>>>>>>>>> that
>>>>>>>>>>>> belong to a reserved region, and that should be mapped into the 
>>>>>>>>>>>> p2m.
>>>>>>>>>>>> I've just tested on my AMD box and I'm also seeing errors (albeit
>>>>>>>>>>>> different ones), so I guess something broke since I last fixed PVH
>>>>>>>>>>>> Dom0 to boot on AMD hardware.
>>>>>>>>>>>>
>>>>>>>>>>>> I've also tested commit:
>>>>>>>>>>>>
>>>>>>>>>>>> commit fad6ba64a8c98bebb9374f390cc255fac05237ab (HEAD)
>>>>>>>>>>>> Author: Roger Pau Monné <roger.pau@xxxxxxxxxx>
>>>>>>>>>>>> Date:   Fri Nov 30 12:10:00 2018 +0100
>>>>>>>>>>>> amd/iommu: skip host bridge devices when updating IOMMU page tables
>>>>>>>>>>>>
>>>>>>>>>>>> And it works on my AMD box and I'm able to boot as a PVH Dom0. Can 
>>>>>>>>>>>> you
>>>>>>>>>>>> give this commit a spin?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks, Roger.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi Roger,
>>>>>>>>>>>
>>>>>>>>>>> Tested that commit, but that didn't help.
>>>>>>>>>>
>>>>>>>>>> Thanks! Sorry for the delay, I got sidetracked with something else.
>>>>>>>>>
>>>>>>>>> No problem, it's not too urgent and probably a busy time with the 
>>>>>>>>> remaining 4.12 stuff.
>>>>>>>>>  
>>>>>>>>>> Can you please post the serial log when using the above commit?
>>>>>>>>>
>>>>>>>>> Sure, I attached a log of:
>>>>>>>>>  - fad6ba64a8c98bebb9374f390cc255fac05237ab  dom0 PVH unsuccesful boot
>>>>>>>>>  - fad6ba64a8c98bebb9374f390cc255fac05237ab  dom0 PV    succesful boot
>>>>>>>>
>>>>>>>> Thanks. So you get the same IO page faults.
>>>>>>>>
>>>>>>>> I don't seem to be able to reproduce this behaviour on my AMD box, but
>>>>>>>> that might be just luck. I've been finding some issues today related
>>>>>>>> to the IOMMU, could you give the following patch a spin and paste the
>>>>>>>> serial log that you get.
>>>>>>>
>>>>>>> Hi Roger,
>>>>>>>
>>>>>>> Sure, on top of what ?
>>>>>>> - fad6ba64a8c98bebb9374f390cc255fac05237ab ?
>>>>>>> - xen-unstable ?
>>>>>>> - xen-unstable + Paul's patch ?
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> Sorry for not proving the right context, let's try on top of
>>>>>> xen-unstable + Paul's patch.
>>>>>>
>>>>>> Thanks, Roger.
>>>>>>
>>>>>
>>>>> Seems to be giving the same result (stream of IO_PAGE_FAULTs), serial
>>>>> log attached.
>>>>
>>>> Thanks, I think I've figured out what's wrong. I've prepared a git
>>>> branch for you to test:
>>>>
>>>> git://xenbits.xen.org/people/royger/xen.git iommu-fixes
>>>>
>>>> Could you give this a try?
>>>>
>>>> Thanks, Roger.
>>>
>>> Hi Roger,
>>>
>>> The good news is, with that branch, dom0 boots on both PVH and PV !
>>> The bad news is, I can't boot up a PVH guest (but perhaps this warrants a 
>>> separate mail thread / change of subject):
>>>
>>> xc: error: panic: xc_dom_boot.c:159: xc_dom_boot_domU_map: failed to mmap 
>>> domU pages 0x1000+0x2426 [mmap, errno=22 (Invalid argument)]: Internal error
>>> libxl: error: libxl_dom.c:760:libxl__build_dom: xc_dom_build_image failed: 
>>> Invalid argument
>>> libxl: error: libxl_create.c:1286:domcreate_rebuild_done: Domain 1:cannot 
>>> (re-)build domain: -3
>>> libxl: error: libxl_domain.c:1038:libxl__destroy_domid: Domain 
>>> 1:Non-existant domain
>>> libxl: error: libxl_domain.c:993:domain_destroy_callback: Domain 1:Unable 
>>> to destroy guest
>>> libxl: error: libxl_domain.c:920:domain_destroy_cb: Domain 1:Destruction of 
>>> domain failed
>>>
>>> There is more in the attached serial.log
>>>
>>> Thanks so far !
>>
>> Thank you for the testing!
>>
>> Let me clean the branch a little bit and prepare a new patch.
> 
> Sorry, fixing that error took longer than expected, and requires
> modifying quite a lot of code, so I'm not sure whether it's something
> to consider for 4.12, I have to think about it.

I understand, especially since PVH dom0 is marked as experimental.

> In the meantime, can you please test the following branch:
> 
> git://xenbits.xen.org/people/royger/xen.git iommu-fixes-v2
> 
> I've been able to successfully create a PVH guest from a PVH dom0 on
> AMD hardware using this branch.

On the other hand, with a quick test I can confirm that booting a PVH guest 
from a PVH dom0 now works for me as well ! 
(and booting this build as PV dom0, with my normal PVH/HVM mix of guests still 
works, no regressions for me so far)


--
Sander

> Thanks, Roger.
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel