Re: [Xen-devel] [xen-unstable bisection] complete test-amd64-amd64-i386-pvgrub

[Julien Grall <julien.grall@xxxxxxxxxx>]

Hi Wei,

On 08/11/17 11:36, Wei Liu wrote:
> On Tue, Nov 07, 2017 at 05:24:32PM +0000, Julien Grall wrote:
> > Hi Wei,
> >
> > On 07/11/17 15:13, Wei Liu wrote:
> > > On Tue, Nov 07, 2017 at 03:09:07PM +0000, Julien Grall wrote:
> > > > Hi Wei,
> > > >
> > > > On 06/11/17 14:55, Wei Liu wrote:
> > > > > On Mon, Nov 06, 2017 at 01:47:56PM +0000, osstest service owner wrote:
> > > > > > branch xen-unstable
> > > > > > xenbranch xen-unstable
> > > > > > job test-amd64-amd64-i386-pvgrub
> > > > > > testid guest-start
> > > > > >
> > > > > > Tree: linux git://xenbits.xen.org/linux-pvops.git
> > > > > > Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
> > > > > > Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
> > > > > > Tree: qemuu git://xenbits.xen.org/qemu-xen.git
> > > > > > Tree: xen git://xenbits.xen.org/xen.git
> > > > > >
> > > > > > *** Found and reproduced problem changeset ***
> > > > > >
> > > > > >      Bug is in tree:  xen git://xenbits.xen.org/xen.git
> > > > > >      Bug introduced:  f48b5449dabc770acdde6d25cfbd265cfb71034d
> > > > > >      Bug not present: 86cf189a957129ea1ad6468fe9a0887b9e2819f3
> > > > > >      Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/115612/
> > > > > >
> > > > > >
> > > > > >      commit f48b5449dabc770acdde6d25cfbd265cfb71034d
> > > > > >      Author: Wei Liu <wei.liu2@xxxxxxxxxx>
> > > > > >      Date:   Thu Oct 12 20:19:07 2017 +0100
> > > > > >          tools/dombuilder: Switch to using gfn terminology for console and 
> > > > > > xenstore rings
> > > > > >          The sole use of xc_dom_translated() and xc_dom_p2m() outside of the 
> > > > > > domain
> > > > > >          builder is for libxl_dom() to translate the console and xenstore pfns 
> > > > > > back
> > > > > >          into useful values.  PV guest pfns are only interesting to the domain 
> > > > > > builder,
> > > > > >          and gfns are the address space used by all other hypercalls.
> > > > > >          Renaming the fields in xc_dom_image is deliberate, as it will cause
> > > > > >          out-of-tree users of the dombuilder to notice the different semantics.
> > > > > >          Correct the terminology throughout xc_dom_gnttab{_hvm,}_seed(), which 
> > > > > > are all
> > > > > >          using gfns despite the existing variable names.
> > > > > >          Signed-off-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
> > > > > >          Reviewed-by: Roger Pau Monn?? <roger.pau@xxxxxxxxxx>
> > > > > >          Acked-by: Wei Liu <wei.liu2@xxxxxxxxxx>
> > > > > >          Tested-by: Julien Grall <julien.grall@xxxxxxx>
> > > > > >          Release-acked-by: Julien Grall <julien.grall@xxxxxxxxxx>
> > > > > >          [ wei: fix stubdom build ]
> > > > > >          Signed-off-by: Wei Liu <wei.liu2@xxxxxxxxxx>
> > > > > This has broken pvgrub. The problem is more than just the name of the
> > > > > variables. I have reverted this and its successor patch.
> > > > It looks like osstest is still broken after the patches you reverted (see
> > > > [1] and [2]).
> > > >
> > > > AFAICT, the only series between the two flights is the dombuilder, there are
> > > > 2 patches not reverted.
> > > >
> > > > Do you have an idea of what's going on?
> > > >
> > > > Cheers,
> > > >
> > > > [1] http://logs.test-lab.xenproject.org/osstest/logs/115624/
> > > > [2]
> > > > https://lists.xenproject.org/archives/html/xen-devel/2017-11/msg00391.html
> > > test-amd64-amd64-xl-qemut-win7-amd64 16 guest-localmigrate/x10 fail REGR. vs.  
> > > 115526
Looking at the osstest result today, this one seem to be intermittent as 
it passed during the night but failed this morning.
> > > test-armhf-armhf-xl-vhd     15 guest-start/debian.repeat fail REGR. vs.  115526
> > The log for the xl-vhd contains ([1])
> >
> > libxl: error: libxl_bootloader.c:283:bootloader_local_detached_cb: Domain 
> > 11:unable to detach locally attached disk
> > libxl: error: libxl_create.c:1246:domcreate_rebuild_done: Domain 11:cannot 
> > (re-)build domain: -3
> > libxl: debug: libxl_domain.c:1138:devices_destroy_cb: Domain 11:Forked pid 5103 
> > for destroy of domain
> > libxl: debug: libxl_create.c:1683:do_domain_create: Domain 0:ao 0x5d6e8: 
> > inprogress: poller=0x56ad8, flags=i
> > libxl: debug: libxl_event.c:1869:libxl__ao_complete: ao 0x5d6e8: complete, rc=-3
> > libxl: debug: libxl_event.c:1838:libxl__ao__destroy: ao 0x5d6e8: destroy
> > libxl: debug: libxl_domain.c:868:libxl_domain_destroy: Domain 11:ao 0x5a170: 
> > create: how=(nil) callback=(nil) poller=0x56ad8
> > libxl: error: libxl_domain.c:1000:libxl__destroy_domid: Domain 11:Non-existant 
> > domain
> > libxl: error: libxl_domain.c:959:domain_destroy_callback: Domain 11:Unable to 
> > destroy guest
> > libxl: error: libxl_domain.c:886:domain_destroy_cb: Domain 11:Destruction of 
> > domain failed
> > libxl: debug: libxl_event.c:1869:libxl__ao_complete: ao 0x5a170: complete, 
> > rc=-21
> > libxl: debug: libxl_domain.c:877:libxl_domain_destroy: Domain 11:ao 0x5a170: 
> > inprogress: poller=0x56ad8, flags=ic
> > libxl: debug: libxl_event.c:1838:libxl__ao__destroy: ao 0x5a170: destroy
> >
> > It is in guest repeat and has succeed few times before.
> >
> > Looking at the success/failure ([2]), the same configuration passed on the 
> > Arndale
> > (see 115580) but fails reliably on the cubietruck.
> The same test failed on Arndale as well in 115314 and 115526, with the
> same error messages.
>
> http://logs.test-lab.xenproject.org/osstest/logs/115526/test-armhf-armhf-xl-vhd/16.ts-guest-start.log
>
> So the failure isn't related to Andrew's series.
>
> > My guess would be the disk is not detached by the previous guest in time.
> > Now the question is why? I am not familiar with this area, any ideas?
> I don't have immediate idea either. I've set up a repro flight so that
> we can have something to play with.
Thanks! Let me know when it is ready.

Cheers,

--
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel