Re: [Xen-devel] Xen-unstable PVHdom0: Assertion 'IS_ALIGNED(dfn_x(dfn), (1ul << page_order))' failed at iommu.c:324

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Thu, Jan 24, 2019 at 01:04:31PM +0100, Roger Pau Monné wrote:
> On Thu, Jan 24, 2019 at 12:55:06PM +0100, Sander Eikelenboom wrote:
> > On 24/01/2019 11:11, Roger Pau Monné wrote:
> > > On Thu, Jan 24, 2019 at 10:25:33AM +0100, Sander Eikelenboom wrote:
> > >> On 24/01/2019 08:50, Roger Pau Monné wrote:
> > >>> On Wed, Jan 23, 2019 at 08:56:48PM +0100, Sander Eikelenboom wrote:
> > >>>> On 23/01/2019 19:25, Roger Pau Monné wrote:
> > >>>>> On Wed, Jan 23, 2019 at 12:39:21AM +0100, Sander Eikelenboom wrote:
> > >>>>>> On 22/01/2019 17:14, Roger Pau Monné wrote:
> > >>>>>>> On Sun, Jan 20, 2019 at 11:09:25PM +0100, Sander Eikelenboom wrote:
> > >>>>>>>> On 18/01/2019 18:56, Roger Pau Monné wrote:
> > >>>>>>>>> On Fri, Jan 18, 2019 at 03:17:57PM +0100, Sander Eikelenboom 
> > >>>>>>>>> wrote:
> > >>>>>>>>>> On 18/01/2019 13:50, Roger Pau Monné wrote:
> > >>>>>>>>>>> On Fri, Jan 18, 2019 at 01:03:04PM +0100, Sander Eikelenboom 
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>> Hi Roger,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I gave PVH dom0 a spin, see how far I would get.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks!
> > >>>>>>>>>>>
> > >>>>>>>>>>>> With current xen-unstable unfortunately not that far, i got 
> > >>>>>>>>>>>> the splat below.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Yes, this was already reported:
> > >>>>>>>>>>>
> > >>>>>>>>>>> https://lists.xenproject.org/archives/html/xen-devel/2019-01/msg01030.html
> > >>>>>>>>>>>> If you need more info, would like me to test a patch (or some 
> > >>>>>>>>>>>> other git tree/branch), 
> > >>>>>>>>>>>> I will be happy to give it a spin !
> > >>>>>>>>>>>
> > >>>>>>>>>>> Paul is working on a fix, but in the meantime just removing the
> > >>>>>>>>>>> assertions should be fine:
> > >>>>>>>>>>>
> > >>>>>>>>>>> ---8<---
> > >>>>>>>>>>> diff --git a/xen/drivers/passthrough/iommu.c 
> > >>>>>>>>>>> b/xen/drivers/passthrough/iommu.c
> > >>>>>>>>>>> index bd1af35a13..98e6fc35e2 100644
> > >>>>>>>>>>> --- a/xen/drivers/passthrough/iommu.c
> > >>>>>>>>>>> +++ b/xen/drivers/passthrough/iommu.c
> > >>>>>>>>>>> @@ -321,9 +321,6 @@ int iommu_map(struct domain *d, dfn_t dfn, 
> > >>>>>>>>>>> mfn_t mfn,
> > >>>>>>>>>>>      if ( !iommu_enabled || !hd->platform_ops )
> > >>>>>>>>>>>          return 0;
> > >>>>>>>>>>>  
> > >>>>>>>>>>> -    ASSERT(IS_ALIGNED(dfn_x(dfn), (1ul << page_order)));
> > >>>>>>>>>>> -    ASSERT(IS_ALIGNED(mfn_x(mfn), (1ul << page_order)));
> > >>>>>>>>>>> -
> > >>>>>>>>>>>      for ( i = 0; i < (1ul << page_order); i++ )
> > >>>>>>>>>>>      {
> > >>>>>>>>>>>          rc = hd->platform_ops->map_page(d, dfn_add(dfn, i), 
> > >>>>>>>>>>> mfn_add(mfn, i),
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> I gave that a spin and i now get a seemingly endless stream of 
> > >>>>>>>>>> IO_PAGE_FAULTs
> > >>>>>>>>>
> > >>>>>>>>> You shouldn't get those page faults since they are for addresses 
> > >>>>>>>>> that
> > >>>>>>>>> belong to a reserved region, and that should be mapped into the 
> > >>>>>>>>> p2m.
> > >>>>>>>>> I've just tested on my AMD box and I'm also seeing errors (albeit
> > >>>>>>>>> different ones), so I guess something broke since I last fixed PVH
> > >>>>>>>>> Dom0 to boot on AMD hardware.
> > >>>>>>>>>
> > >>>>>>>>> I've also tested commit:
> > >>>>>>>>>
> > >>>>>>>>> commit fad6ba64a8c98bebb9374f390cc255fac05237ab (HEAD)
> > >>>>>>>>> Author: Roger Pau Monné <roger.pau@xxxxxxxxxx>
> > >>>>>>>>> Date:   Fri Nov 30 12:10:00 2018 +0100
> > >>>>>>>>> amd/iommu: skip host bridge devices when updating IOMMU page 
> > >>>>>>>>> tables
> > >>>>>>>>>
> > >>>>>>>>> And it works on my AMD box and I'm able to boot as a PVH Dom0. 
> > >>>>>>>>> Can you
> > >>>>>>>>> give this commit a spin?
> > >>>>>>>>>
> > >>>>>>>>> Thanks, Roger.
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Hi Roger,
> > >>>>>>>>
> > >>>>>>>> Tested that commit, but that didn't help.
> > >>>>>>>
> > >>>>>>> Thanks! Sorry for the delay, I got sidetracked with something else.
> > >>>>>>
> > >>>>>> No problem, it's not too urgent and probably a busy time with the 
> > >>>>>> remaining 4.12 stuff.
> > >>>>>>  
> > >>>>>>> Can you please post the serial log when using the above commit?
> > >>>>>>
> > >>>>>> Sure, I attached a log of:
> > >>>>>>  - fad6ba64a8c98bebb9374f390cc255fac05237ab  dom0 PVH unsuccesful 
> > >>>>>> boot
> > >>>>>>  - fad6ba64a8c98bebb9374f390cc255fac05237ab  dom0 PV    succesful 
> > >>>>>> boot
> > >>>>>
> > >>>>> Thanks. So you get the same IO page faults.
> > >>>>>
> > >>>>> I don't seem to be able to reproduce this behaviour on my AMD box, but
> > >>>>> that might be just luck. I've been finding some issues today related
> > >>>>> to the IOMMU, could you give the following patch a spin and paste the
> > >>>>> serial log that you get.
> > >>>>
> > >>>> Hi Roger,
> > >>>>
> > >>>> Sure, on top of what ?
> > >>>> - fad6ba64a8c98bebb9374f390cc255fac05237ab ?
> > >>>> - xen-unstable ?
> > >>>> - xen-unstable + Paul's patch ?
> > >>>
> > >>> Hello,
> > >>>
> > >>> Sorry for not proving the right context, let's try on top of
> > >>> xen-unstable + Paul's patch.
> > >>>
> > >>> Thanks, Roger.
> > >>>
> > >>
> > >> Seems to be giving the same result (stream of IO_PAGE_FAULTs), serial
> > >> log attached.
> > > 
> > > Thanks, I think I've figured out what's wrong. I've prepared a git
> > > branch for you to test:
> > > 
> > > git://xenbits.xen.org/people/royger/xen.git iommu-fixes
> > > 
> > > Could you give this a try?
> > > 
> > > Thanks, Roger.
> > 
> > Hi Roger,
> > 
> > The good news is, with that branch, dom0 boots on both PVH and PV !
> > The bad news is, I can't boot up a PVH guest (but perhaps this warrants a 
> > separate mail thread / change of subject):
> > 
> > xc: error: panic: xc_dom_boot.c:159: xc_dom_boot_domU_map: failed to mmap 
> > domU pages 0x1000+0x2426 [mmap, errno=22 (Invalid argument)]: Internal error
> > libxl: error: libxl_dom.c:760:libxl__build_dom: xc_dom_build_image failed: 
> > Invalid argument
> > libxl: error: libxl_create.c:1286:domcreate_rebuild_done: Domain 1:cannot 
> > (re-)build domain: -3
> > libxl: error: libxl_domain.c:1038:libxl__destroy_domid: Domain 
> > 1:Non-existant domain
> > libxl: error: libxl_domain.c:993:domain_destroy_callback: Domain 1:Unable 
> > to destroy guest
> > libxl: error: libxl_domain.c:920:domain_destroy_cb: Domain 1:Destruction of 
> > domain failed
> > 
> > There is more in the attached serial.log
> > 
> > Thanks so far !
> 
> Thank you for the testing!
> 
> Let me clean the branch a little bit and prepare a new patch.

Sorry, fixing that error took longer than expected, and requires
modifying quite a lot of code, so I'm not sure whether it's something
to consider for 4.12, I have to think about it.

In the meantime, can you please test the following branch:

git://xenbits.xen.org/people/royger/xen.git iommu-fixes-v2

I've been able to successfully create a PVH guest from a PVH dom0 on
AMD hardware using this branch.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel