Re: [Xen-devel] [libvirt test] 58119: regressions - FAIL

[Anthony PERARD <anthony.perard@xxxxxxxxxx>]

On Wed, Jun 24, 2015 at 02:40:02PM +0200, Dario Faggioli wrote:
> On Tue, 2015-06-23 at 14:38 +0100, Ian Campbell wrote:
> > On Tue, 2015-06-23 at 12:15 +0100, Anthony PERARD wrote:
> > > On Mon, Jun 08, 2015 at 10:22:28AM +0100, Ian Campbell wrote:
> > > > On Mon, 2015-06-08 at 04:37 +0000, osstest service user wrote:
> > > > > flight 58119 libvirt real [real]
> > > > > http://logs.test-lab.xenproject.org/osstest/logs/58119/
> > > > > 
> > > > > Regressions :-(
> > > > > 
> > > > > Tests which did not succeed and are blocking,
> > > > > including tests which could not be run:
> > > > 
> > > > This has been failing for a while now, sorry for not brining it to your
> > > > attention sooner.
> > > 
> > > > libxl: debug: libxl_event.c:638:libxl__ev_xswatch_deregister: watch 
> > > > w=0x7f805c25b248 wpath=/local/domain/0/device-model/1/state token=3/0: 
> > > > deregister slotnum=3
> > > > libxl: error: libxl_exec.c:393:spawn_watch_event: domain 1 device 
> > > > model: startup timed out
> > > > libxl: debug: libxl_event.c:652:libxl__ev_xswatch_deregister: watch 
> > > > w=0x7f805c25b248: deregister unregistered
> > > > libxl: debug: libxl_event.c:652:libxl__ev_xswatch_deregister: watch 
> > > > w=0x7f805c25b248: deregister unregistered
> > > > libxl: error: libxl_dm.c:1564:device_model_spawn_outcome: domain 1 
> > > > device model: spawn failed (rc=-3)
> > > > libxl: error: libxl_create.c:1373:domcreate_devmodel_started: device 
> > > > model did not start: -3
> > > 
> > > Hi,
> > > 
> > > I've tried to debug this "device model: startup time out" issue that I'm
> > > seeing on OpenStack. What I've done is strace every single QEMU. It appear
> > > that QEMU take more than 10s to load...
> > 
> > Looking through
> > http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-libvirt/ALL.html
> >  when it passes the collected var-log-libvirt-libxl-libxl-driver.log.gz 
> > seems to indicate that the device model is successfully spawned in 2-4s.
> > 
> > The same is true of the tests run on the Cambridge instance.
> > 
> So, can we take Anthony's code/instrumentation for stracing QEMU and do
> the same in the ad-hoc run on the test on merlot?

It's very easy to do. What I have done is replace the QEMU binary by a
script to start strace and QEMU.

qemu_path=/usr/bin/qemu-system-i386
mv $qemu_path{,.bak}
cat >> $qemu_path <<EOF
#!/bin/bash
strace -qqttT $0.bak "$@"
EOF

Done :).

> The goal would be to have something like what he attached to his email
> (the strace output) for our failing case on merlot.
> 
> That's assuming that what Anthony have done to get the traces could be
> put in a patch to libxl and/or libvirt, apply it to some branch, and
> make the ad-hoc test pick code for the proper components from such
> branch... which, I think, should all be doable, or am I talking
> nonsense?



-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel