Re: [Xen-users] Xen 4.2.2 /etc/init.d/xendomains save and restore of domains does not work

[Ian Murray <murrayie@xxxxxxxxxxx>]

----- Original Message -----
> From: Andreas Greve <greve-ml@xxxxxxxxxx>
> To: Ian Murray <murrayie@xxxxxxxxxxx>
> Cc: greve-ml@xxxxxxxxxx; xen-users <xen-users@xxxxxxxxxxxxx>; 
> andreas.greve@xxxxxxxxxx
> Sent: Monday, 1 July 2013, 17:44
> Subject: Re: [Xen-users] Xen 4.2.2 /etc/init.d/xendomains save and restore
 of domains does not work
> 
> I have an idea.
> 
> assumptions:
> 
> SXP format:
> root@srv01:~# xl create --quiet --dryrun --defconfig 
> /etc/xen/auto/03_gnomedag | head -10
> (domain
>          (domid -1)
>          (create_info)
>          (hvm 0)
>          (hap <default>)
>          (oos <default>)
>          (ssidref 0)
>          (name gnomedag)
>          (uuid <unknown>)
>          (cpupool Pool-0)
> [...]
> )
> root@srv01:~#
> 
> JSON format:
> xl create --quiet --dryrun --defconfig /etc/xen/auto/03_gnomedag | head -10
> {
>      "domid": null,
>      "config": {
>          "c_info": {
>              "type": "pv",
>              "hap": "<default>",
>              "oos": "<default>",
>              "ssidref": 0,
>              "name": "gnomedag",
>              "uuid": "d9a9eba0-e7b8-4d16-ad6e-bb4cac05fd14",
> [...]
> }
> 
> Let use sed  pattern address ranges to distinguish between JSON and SXP.
> 
> 
> xl create --quiet --dryrun --defconfig /etc/xen/auto/03_gnomedag \
> |  sed -n -e '/^[{]$/,/^[}]$/ { s/^.*"name": 
> "\(.*\)",$/\1/p }
> /^[(]domain$/,/^[)]$/ { s/^.*(name \(.*\))$/\1/p }'
> 
> /^[{]$/,/^[}]$/  identify JSON
> /^[(]domain$/,/^[)]$/  identify SXP
> 
> the line break between ...\1/p} and /^[(]domain$... is syntactical 
> needed by sed.
> 
> 
> sed -n -e '/^[{]$/,$ { s/^.*"name": 
> "\(.*\)",$/\1/p }
> /^[(]domain$/,$ { s/^.*(name \(.*\))$/\1/p }' < json.out
> 
> in the example above  ,$ in the address range part  means until  end of file
> 
> For the JSON filter perhaps you can use somthing like this
> /^[ ]*"c_info": [{]$/,$
> as address range because `"c_info": {´ is comes close before 
> `"name:" 
> "gnomedag"´ but I
> know not enough about xl to estimate if that is always true.
> 
> but further I have to look at the stop() function at the block
> 
>      if test "$XENDOMAINS_AUTO_ONLY" = "true"; then
>         eval
>         [...]
>      fi
> 
> for bad side effects of the now working rdnames() -> rdname()

You've gone beyond my sed/regex skills so I can't really assist on the above. I 
think the best thing is for you to try to get it into a working patch and I'd 
be happy to test it, then you could submit it to the dev list for inclusion. 
Also beware that there is an bug in xl list -l in sxp format. The domain id 
comes out as -1 for a running domain. I agreed to adapt a commit as a backport 
when we discussed this on dev list.


> 
> 
> 
> On 07/01/13 16:51, Ian Murray wrote:
>> 
>> 
>> 
>>  ----- Original Message -----
>>>  From: Andreas Greve<greve-ml@xxxxxxxxxx>
>>>  To: Ian Murray<murrayie@xxxxxxxxxxx>
>>>  Cc: andreas.greve@xxxxxxxxxx; xen-users<xen-users@xxxxxxxxxxxxx>
>>>  Sent: Monday, 1 July 2013, 14:59
>>>  Subject: Re: [Xen-users] Xen 4.2.2 /etc/init.d/xendomains save and 
> restore
>>    of domains does not work
>>>  first sorry that I explained it bad but my English is not very well:
>>> 
>>>  second I realize why you could not reproduce my problems under xl JSON
>>>  format:
>>>  short: my symlink names in $XENDOMAINS_AUTO differ from the 
> "real"
>>>  domain name. Later more about that
>>> 
>>>  Problem description detail:
>>> 
>>>  File: /etc/init.d/xendomains (from 4.3 git
>>>  commit543a2657182dbb9237d1feeb1d3193096ab2cb2d )
>>> 
>>>  The sed "expression in function rdname()
>>> 
>>>  rdname()
>>>  {
>>>        NM=$($CMD create --quiet --dryrun --defconfig "$1" |
>>>             sed -n 's/^.*(name \(.*\))$/\1/p')
>>>  }
>>> 
>>>        called by
>>> 
>>>  is_running $dom
>>> 
>>>       called from
>>> 
>>>  start
>>> 
>>>       in the if block
>>> 
>>>  if contains_something "$XENDOMAINS_AUTO"
>>>  then
>>>  [....]
>>>  if [ $? -eq 0 ] || is_running $dom; then
>>> 
>>>        works only for the SXP format and not for the  JSON format. For
>>>  JSON fromat the funktion is_running $dom  will nearly always return 
> false
>>> 
>>> 
>>>  Example to show that:
>>> 
>>>  root@srv01:~# xl create --quiet --dryrun --defconfig
>>>  /etc/xen/auto/03_gnomedag | head -10
>>>  {
>>>        "domid": null,
>>>        "config": {
>>>            "c_info": {
>>>                "type": "pv",
>>>                "hap": "<default>",
>>>                "oos": "<default>",
>>>                "ssidref": 0,
>>>                "name": "gnomedag",
>>>                "uuid": 
> "2d33d9c8-7efe-4060-8e61-b47af2796a5c",
>>>  root@srv01:~#
>>> 
>>> 
>>>  root@srv01:~# xl create --quiet --dryrun --defconfig
>>>  /etc/xen/auto/03_gnomedag \
>>>  | sed -n 's/^.*(name \(.*\))$/\1/p'
>>>  root@srv01:~#
>>> 
>>> 
>>>  So $NM will always be empty and is_running() will nearly always return
>>>  false;
>>> 
>>>  is_running()
>>>  {
>>>        rdname $1
>>>        RC=1
>>>        name=;id=
>>>        while read LN; do
>>>            parseln "$LN" || continue
>>>            if test $id = 0; then continue; fi
>>>            case $name in
>>>                ($NM)
>>>                RC=0
>>>                ;;
>>>            esac
>>>        done<  <($CMD list -l | grep "$LIST_GREP")
>>>        return $RC
>>>  }
>>> 
>>> 
>>>  In contrast my sed "expression" will work for JSON format but 
> not for
>>>  SXP (Yes with that you are right. Sorry ! Up to now I did not know or
>>>  have forgotten that xl supports both formats).
>>  I went through the exact same the week before last.
>> 
>>>  root@srv01:~# xl create --quiet --dryrun --defconfig
>>>  /etc/xen/auto/03_gnomedag \
>>>  | sed -n 's/^.*"name": 
> "\(.*\)",$/\1/p'
>>>  gnomedag
>>>  root@srv01:~#
>>> 
>>>  That is what I expected for JSON format.
>>> 
>>> 
>>>  I found why our both test have different results for the JSON fromat:
>>> 
>>>  On my System (historical reason) the name of the symlinks in
>>>  $XENDOMAINS_AUTO/* differ from the "real" domain name 
> (example: domain
>>> 
>>>  name: gnomedag   symlink name: /etc/xen/auto/03_gnomedag ->
>>>  /etc/xen/gnomedag.cfg).
>>> 
>>>            shortdom=$(echo $dom | sed -n
>>>  's/^.*\/\(.*\)$/\1/p')
>>>            echo $saved_domains | grep -w $shortdom>  /dev/null
>>>            if [ $? -eq 0 ] || is_running $dom; then
>>>                  echo -n "(skip)"
>>>            else
>>> 
>>>  In your configuration the restored domain names  are filtered by the
>>>  echo ... | grep -w ... instruction. So
>>>  is_running $dom is never called for them. So you will never see the
>>>  semantical error produced  by rdname().
>>> 
>>>  In my system  the "real" domain name stored  in saved_domains 
> differ
>>>  from the value in $dom. So
>>>  is_running $dom is always called and always returns false. Which leads
>>>  to a "second start" of the already restored domain.
>>> 
>>  Right, I understand in principle. I will need to look at the script again 
> to understand in detail, but am happy to accept your word on it. :) What you 
> say 
> makes sense to me.
>> 
>> 
>> 
>>>  In the moment I have no good idea how to combine the two needed sed
>>>  expressions for the rdname() function
>>> 
>>>  sed -n 's/^.*(name \(.*\))$/\1/p'     #SXP
>>> 
>>>  sed -n 's/^.*"name": 
> "\(.*\)",$/\1/p'
>>>  #JSON
>>> 
>>  Could we not have somthing like (pseudo)
>>  sedresult = cmd... | sed... (name
>>  if sedresult = "" then
>>       sedresult = cmd ... | sed... "name":
>>  fi
>> 
>>  then either way sedresult should be blank for ones it should be and 
> populated for those where it should be?
>> 
>>  A discussion on the dev list with David Sutton and Ian Campbell concluded 
> that the whole thing should probably be re-written from scratch because the 
> SXP 
> vs JSON has made this script quite scrappy. There are still issues with the 
> handling of zombies, as $state isn't populated and can't be using xl 
> list -l
>> 
>> 
>>> 
>>>  On 07/01/13 02:06, Ian Murray wrote:
>>>>>     I download the file by from git and give him a chance. It 
> works
>>>>>    nearly perfect.
>>>>>    On xendomains stop there was all ok.
>>>>> 
>>>>>    On xendomains start it restores the saved domains but after 
> that it
>>>>>    tries to start them again and produce some error messages 
> like domain
>>>>>    is already running.
>>>>> 
>>>>    It appears to be working fine for me. I am running 4.3 rc 6 + 
> next
>>>>    commit. Rather than saying the output has changed for xl, I think 
> you
>>>>    may be referring to the fact that xl now can output JSON as well 
> as
>>>>    xm's SXP. Having said this, xendomains works both properly 
> for me both
>>>>    using JSON and SXP although I am not sure why the skipping of the
>>>>    autostart domains is working when I select JSON in xl.conf. 
> Certainly
>>>>    the xl ... sed line fails when I execute it manually when using 
> JSON.
>>>> 
>>>>    I think your solution is not the right approach because xl can 
> produce
>>>>    JSON and SXP format and that is defined in xl.conf. Your solution 
> will
>>>>    have a problem when setting xl to produce SXP, I think.
>>>> 
>>>>   root@xen6:/etc/xen/auto# service xendomains stop
>>>>    Shutting down Xen domains:
>>>>    ubuntu-email(save)................................
>>>>     vpn2(save)....
>>>>     *   [done]
>>>>   root@xen6:/etc/xen/auto# service xendomains start
>>>>    Restoring Xen domains: ubuntu-email vpn2
>>>>    Starting auto Xen domains: ubuntu-email(skip) vpn2(skip) *  
> [done]
>>>> 
>>>>    Not tested against xm as I have no means to do so.
>>>> 
>>>> 
>>>>>    The reason is  the sed script in rdname() does not work with 
> xl output.
>>>>>    I'll changed it in the way as you have done with HEADCOMP 
> (see the
>>>>>    diff below).
>>>>>    After that all was nice for me
>>>>> 
>>>>> 
>>>>>    ------------------------------------------------
>>>>>   root@srv01:/etc/init.d# diff -u .xendomains.4.3.original 
> xendomains
>>>>>    --- .xendomains.4.3.original    2013-06-30 20:54:14.000000000 
> +0200
>>>>>    +++ xendomains  2013-06-30 23:27:44.000000000 +0200
>>>>>    @@ -31,11 +31,13 @@
>>>>> 
>>>>>     CMD=${SBINDIR}/xm
>>>>>     HEADCOMP="LinuxGuestRecord"
>>>>>    +RDNAMESED='s/^.*(name \(.*\))$/\1/p'
>>>>>     $CMD list&>  /dev/null
>>>>>     if test $? -ne 0
>>>>>     then
>>>>>            CMD=${SBINDIR}/xl
>>>>>            HEADCOMP="Xen saved domain"
>>>>>    +        RDNAMESED='s/^.*"name":
>>>  "\(.*\)",$/\1/p'
>>>>>     fi
>>>>> 
>>>>>     $CMD list&>  /dev/null
>>>>>    @@ -185,8 +187,8 @@
>>>>>     # read name from xen config file
>>>>>     rdname()
>>>>>     {
>>>>>    -    NM=$($CMD create --quiet --dryrun --defconfig 
> "$1" |
>>>>>    -         sed -n 's/^.*(name 
> \(.*\))$/\1/p')
>>>>>    +    NM=$( $CMD create --quiet --dryrun --defconfig 
> "$1" |
>>>>>    +         sed -n "${RDNAMESED}" )
>>>>>     }
>>>>> 
>>>>>     rdnames()
>>>>>    ----------------------------------------------------
>>>>> 
>>>>>>    I am surprised you did not have issue with 4.2.1 because 
> the header
>>>>>>    issues have been present ever since xl became the 
> default/preferred
>>>>>>    toolstack, unless your Xen 4.2.1 came from a third-party.
>>>>>    Your are right I remember that there were some problems with 
> 4.2.1
>>>>>    too. As I switched from 4.1 to 4.2.1.
>>>>> 
>>>>>    Sorry at that time I had not much time. If I remember right, 
> I fixed
>>>>>    that in any way for me, but forget to report.
>>>>> 
>>>>>    Thanks for your help. I learned a lot about bash (bla) seams 
> to be
>>>>>    equal to bla) in case instructions. That was new for me.
>>>>> 
>>>>> 
>>>>>    Best wishes
>>>>> 
>>>>> 
>>>>>    Andreas
>>>>> 
>>>>>>    _______________________________________________
>>>>>>    Xen-users mailing list
>>>>>>   Xen-users@xxxxxxxxxxxxx
>>>>>>   http://lists.xen.org/xen-users
>>>>> 
> 

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users