Re: [Wg-openstack] XenProject OpenStack CI loop failure investigation

[Bob Ball <bob.ball@xxxxxxxxxx>]

The Citrix CI has similar issues with this test.  We currently exclude the test 
from our Neutron CI due to a timeout issue accessing via the floating IP:

http://git.openstack.org/cgit/openstack/xenapi-os-testing/tree/tempest_exclusion_list
# Need resolve the timeout issue when access from floating IP.
NEUTRON_NETWORK_TEMPEST_REGEX="$NEUTRON_NETWORK_TEMPEST_REGEX|.*tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern.*"

If the failures are isolated to this test, it may be you can exclude the test 
as a short-term fix in the same way as we have (although our short-term fix is 
a little longer-term than I like)

Bob

-----Original Message-----
From: Wg-openstack [mailto:wg-openstack-bounces@xxxxxxxxxxxxxxxxxxxx] On Behalf 
Of Anthony PERARD
Sent: 20 October 2016 17:44
To: wg-openstack@xxxxxxxxxxxxxxxxxxxx
Subject: [Wg-openstack] XenProject OpenStack CI loop failure investigation

Hi,

I've looked at recent job failure yesterday (5 jobs), and they all have the 
same test failed:
tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern

and the reason that tempest give is:
tempest.lib.exceptions.SSHTimeout: Connection to the 172.24.5.1 via SSH timed 
out

link to those logs:
http://logs.openstack.xenproject.org/98/282398/15/check/dsvm-tempest-xen/cbe660e/
http://logs.openstack.xenproject.org/15/386915/3/check/dsvm-tempest-xen/8dd1c6a/
http://logs.openstack.xenproject.org/70/367470/11/check/dsvm-tempest-xen/0029a96/
http://logs.openstack.xenproject.org/74/385074/4/check/dsvm-tempest-xen/c0a396e/
http://logs.openstack.xenproject.org/47/376547/5/check/dsvm-tempest-xen/056f21d/


This test boot from a volume, and this volume is via iSCSI.

Looking into logs/syslog.txt.gz, there are few lines that make me thing iSCSI 
is the issue:
kernel: [ 6526.368075]  connection23:0: ping timeout of 5 secs expired, recv 
timeout 5, last rx 4296521382, last ping 4296522634, now 429652
kernel: [ 6526.368191]  connection23:0: detected conn error (1022)
iscsid: Kernel reported iSCSI connection 23:0 error (1022 - Invalid or unknown 
error code) state (3)
iscsid: connection23:0 is operational after recovery (1 attempts)

but that appear to be for another tests, which have succeded.

So, I don't know why the test fail, and why the guest appear to be unresponsive.

--
Anthony PERARD

_______________________________________________
Wg-openstack mailing list
Wg-openstack@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-openstack
_______________________________________________
Wg-openstack mailing list
Wg-openstack@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/cgi-bin/mailman/listinfo/wg-openstack