Re: [Xen-API] XCP 1.6 BETA problem with pool-designate-new-master

[Black Bird <blackbird1758@xxxxxxxxx>]

I have made progress on this issue today.

On another thread
(http://lists.xen.org/archives/html/xen-api/2012-11/msg00060.html) I
have described having found references to old hosts, which I surmised
could be stopping the pool from considering the rolling pool upgrade
as completed.  So I have removed references to the old hosts (xe
host-forget) and rebooted all hosts in the pool.  This did not resolve
the issue faced in the thread, although it changed the error message
there.

I have also retried to execute the command in this thread (xe
pool-designate-new-master) to see if there was a difference, and I was
pleased to note that the command worked.  I was able to carry this out
successfully and successively on both slaves.

So my conclusion - there is no bug here, but simply the result of an
incomplete pool upgrade.

Thanks for reading.

On Tue, Nov 6, 2012 at 11:27 AM, Black Bird <blackbird1758@xxxxxxxxx> wrote:
> (discreet bump)
>
> I'd be grateful if anyone could point me in the right direction.
>
> On Wed, Oct 31, 2012 at 2:42 PM, Black Bird <blackbird1758@xxxxxxxxx> wrote:
>> sorry for yet another post today on yet another subject!
>>
>> Short recap on my environment: a 3 host pool which has been upgraded
>> from XCP1.1 to XCP 1.6 Beta 2.  The pool is functional in its basic
>> operations - vm creation, destroy, migration, vif creation etc.  One
>> pending (separate post) problem I have is with pool-dump-database.
>>
>> I have encountered a problem in promoting any of the 2 slaves to
>> become the new master, using the command 'xe
>> pool-designate-new-master'.  The command returns (on both slaves), the
>> following
>>
>> "The restore could not be performed because this backup has been
>> created by a different (incompatible) product version"
>>
>> The slave's xensource log contains:
>>
>>
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80||dummytaskhelper] task dispatch:pool.designate_new_master
>> D:8755726812be created by task R:ce8358aa6fcf
>> Oct 31 14:33:43 xen2 xapi: [ info|xen2|12729 INET
>> 0.0.0.0:80|dispatch:pool.designate_new_master
>> D:8755726812be|taskhelper] task pool.designate_new_master
>> R:ce8358aa6fcf forwarded (trackid=46da7bc98d2ecc468c5a4e4390a9f5b3)
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|mscgen] xapi=>xapi
>> [label="<methodCall><methodName>pool.sync_database</methodName><params><param><value>OpaqueRef:787d8a2b-6fd0-1606-ab35-93745e773e26</value></param></params></methodCall>"];
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|stunnel_cache]
>> Cache contents: [ 192.168.121.10:443 (id 13993 / idle 12.63 age 12.67)
>> ]
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|stunnel_cache]
>> Removing stunnel id 13993 (idle 12.63) from the cache
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|stunnel]
>> check_reusable: caught exception Unix.Unix_error(31, "write", "");
>> assuming not reusable
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|stunnel]
>> get_reusable_stunnel: Found non-reusable stunnel in the cache.
>> disconnecting from 192.168.121.10:443
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|stunnel_cache]
>> Cache contents: [ 192.168.121.10:443  ]
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|stunnel]
>> get_reusable_stunnel: stunnel cache is empty; creating a fresh
>> connection to 192.168.121.10:443
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|stunnel] Using
>> commandline: /usr/sbin/stunnel -fd
>> 94227978-53b4-2937-78a8-4b4573314ca5
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|stunnel] stunnel
>> has pidty: (FEFork (28,2652))
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|stunnel] stunnel
>> start: Log from stunnel: []
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|xmlrpc_client]
>> stunnel pid: 2652 (cached = true) connected to 192.168.121.10:443
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|xmlrpc_client]
>> with_recorded_stunnelpid task_opt=None s_pid=2652
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|stunnel] Using
>> commandline: /usr/sbin/stunnel -fd
>> 74e84376-4357-6632-0e5e-bfd8d18c4ad7
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|stunnel] stunnel has
>> pidty: (FEFork (31,2659))
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|stunnel] stunnel start
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|xmlrpc_client] stunnel
>> pid: 2659 (cached = false) connected to 192.168.121.10:443
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|xmlrpc_client]
>> with_recorded_stunnelpid task_opt=None s_pid=2659
>> Oct 31 14:33:43 xen2 xapi: [error|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|pool_db_sync] Pool
>> backup file was created with incompatable product version
>> Oct 31 14:33:43 xen2 xapi: [ warn|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|xmlrpc_client] stunnel
>> pid: 2659 caught
>> Api_errors.Server_error("RESTORE_INCOMPATIBLE_VERSION", 0)
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|xapi] Raised at
>> pool_db_backup.ml:48.8-78 -> pool_db_backup.ml:220.4-20 ->
>> xmlrpc_client.ml:258.2-63 -> xmlrpc_client.ml:237.7-10 ->
>> xmlrpc_client.ml:242.13-14 -> pervasiveext.ml:22.2-9
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|xapi] Raised at
>> pervasiveext.ml:26.22-25 -> pervasiveext.ml:22.2-9
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|backtrace] Raised at
>> pervasiveext.ml:26.22-25 -> pool_db_backup.ml:237.15-57 ->
>> rbac.ml:229.16-23
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|backtrace] Raised at
>> rbac.ml:238.10-15 -> server_helpers.ml:72.10-22
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|dispatcher]
>> Server_helpers.exec exception_handler: Got exception
>> RESTORE_INCOMPATIBLE_VERSION: [  ]
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|dispatcher] Raised at
>> string.ml:150.25-34 -> stringext.ml:108.13-29
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|backtrace] Raised at
>> string.ml:150.25-34 -> stringext.ml:108.13-29
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|xapi] Raised at
>> server_helpers.ml:94.14-15 -> pervasiveext.ml:22.2-9
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|host.request_backup D:84691f7ce90d|xapi] Raised at
>> pervasiveext.ml:26.22-25 -> pervasiveext.ml:22.2-9
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|dispatch:host.request_backup D:246162f2d9db|xapi] Raised at
>> pervasiveext.ml:26.22-25 -> pervasiveext.ml:22.2-9
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12730 INET
>> 0.0.0.0:80|dispatch:host.request_backup D:246162f2d9db|backtrace]
>> Raised at pervasiveext.ml:26.22-25 -> server_helpers.ml:140.10-106 ->
>> server.ml:15526.19-167 -> server_helpers.ml:119.4-7
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|stunnel_cache]
>> Adding stunnel id 13995 (idle 0.00) to the cache
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|stunnel_cache]
>> Cache contents: [ 192.168.121.10:443 (id 13995 / idle 0.00 age 0.27) ]
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|xmlrpc_client]
>> stunnel pid: 2652 (cached = true) returned stunnel to cache
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|xapi] Raised at
>> client.ml:6.37-75 -> client.ml:3415.12-65 -> pervasiveext.ml:22.2-9
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|backtrace] Raised
>> at pervasiveext.ml:26.22-25 -> xapi_pool.ml:948.2-111 ->
>> rbac.ml:229.16-23
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|backtrace] Raised
>> at rbac.ml:238.10-15 -> server_helpers.ml:72.10-22
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|dispatcher]
>> Server_helpers.exec exception_handler: Got exception
>> RESTORE_INCOMPATIBLE_VERSION: [  ]
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|dispatcher] Raised
>> at string.ml:150.25-34 -> stringext.ml:108.13-29
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|backtrace] Raised
>> at string.ml:150.25-34 -> stringext.ml:108.13-29
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|xapi] Raised at
>> server_helpers.ml:94.14-15 -> pervasiveext.ml:22.2-9
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|taskhelper]
>> forwarded task destroyed
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|pool.designate_new_master R:ce8358aa6fcf|xapi] Raised at
>> pervasiveext.ml:26.22-25 -> pervasiveext.ml:22.2-9
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|dispatch:pool.designate_new_master D:8755726812be|xapi]
>> Raised at pervasiveext.ml:26.22-25 -> pervasiveext.ml:22.2-9
>> Oct 31 14:33:43 xen2 xapi: [debug|xen2|12729 INET
>> 0.0.0.0:80|dispatch:pool.designate_new_master
>> D:8755726812be|backtrace] Raised at pervasiveext.ml:26.22-25 ->
>> server_helpers.ml:140.10-106 -> server.ml:3760.19-167 ->
>> server_helpers.ml:119.4-7
>>
>> This seems to be a bug.  Can anyone comment?

_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api