Re: [Xen-devel] [linux-3.10 test] 21490: regressions - FAIL

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Wed, 2013-11-06 at 09:03 +0000, xen.org wrote:
> flight 21490 linux-3.10 real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/21490/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-amd64-i386-pv           18 leak-check/check          fail REGR. vs. 
> 21026
>  test-amd64-i386-qemut-rhel6hvm-intel 11 leak-check/check  fail REGR. vs. 
> 21026
>  test-amd64-amd64-pv          18 leak-check/check          fail REGR. vs. 
> 21026
>  test-amd64-i386-qemuu-rhel6hvm-intel 11 leak-check/check  fail REGR. vs. 
> 21026
>  test-amd64-amd64-xl          18 leak-check/check          fail REGR. vs. 
> 21026
>  test-amd64-i386-pair         20 leak-check/check/src_host fail REGR. vs. 
> 21026
>  test-amd64-i386-pair         21 leak-check/check/dst_host fail REGR. vs. 
> 21026
>  test-amd64-amd64-pair        21 leak-check/check/dst_host fail REGR. vs. 
> 21026
>  test-amd64-amd64-pair        20 leak-check/check/src_host fail REGR. vs. 
> 21026
>  test-amd64-i386-rhel6hvm-intel 11 leak-check/check        fail REGR. vs. 
> 21026

The bisector says, in <E1Ve0Tg-0008Kj-Il@xxxxxxxxxxxxxxxxxxxxxxx>:

*** Found and reproduced problem changeset ***

  Bug is in tree:  linux 
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
  Bug introduced:  f495ddc46f97dd0054ff2d5d3c7493d59f7511fb
  Bug not present: b9396c4c9e7f499b1dd8080e901c88705f2efa99


  commit f495ddc46f97dd0054ff2d5d3c7493d59f7511fb
  Author: Paul Durrant <paul.durrant@xxxxxxxxxx>
  Date:   Tue Oct 8 14:22:56 2013 +0100
  
      xen-netback: Don't destroy the netdev until the vif is shut down

http://www.chiark.greenend.org.uk/~xensrcts/logs/21490/test-amd64-amd64-xl/18.ts-leak-check.log
 shows:

        2013-11-06 04:23:59 Z executing ssh ... root@xxxxxxxxxxxx xl
        list
        2013-11-06 04:23:59 Z LEAKED [domain 1] domain: (null)                  
                     1     0     1     --pssd       4.6
        2013-11-06 04:23:59 Z LEAKED [domain 2] domain: (null)                  
                     2     0     1     --pssd       0.9
        2013-11-06 04:23:59 Z LEAKED [domain 3] domain: (null)                  
                     3     0     1     --pssd       0.5
        2013-11-06 04:23:59 Z LEAKED [domain 4] domain: (null)                  
                     4     0     1     --pssd       0.9
        2013-11-06 04:23:59 Z LEAKED [domain 5] domain: (null)                  
                     5     0     1     --pssd       1.1
        2013-11-06 04:23:59 Z LEAKED [domain 6] domain: (null)                  
                     6     0     1     --pssd       0.4
        2013-11-06 04:23:59 Z LEAKED [domain 7] domain: (null)                  
                     7     0     1     --pssd       0.4
        2013-11-06 04:23:59 Z LEAKED [domain 8] domain: (null)                  
                     8     0     1     --pssd       0.4
        2013-11-06 04:23:59 Z LEAKED [domain 9] domain: (null)                  
                     9     0     1     --pssd       0.4
        2013-11-06 04:23:59 Z LEAKED [domain 10] domain: (null)                 
                     10     0     1     --pssd       0.4
        2013-11-06 04:23:59 Z LEAKED [domain 11] domain: (null)                 
                     11     0     1     --pssd       0.4
        2013-11-06 04:23:59 Z LEAKED [domain 12] domain: (null)                 
                     12     0     1     --pssd       0.4
        2013-11-06 04:23:59 Z LEAKED [domain 13] domain: (null)                 
                     13     0     1     --pssd       0.4
        2013-11-06 04:23:59 Z LEAKED [domain 14] domain: (null)                 
                     14     0     1     --pssd       0.4
        2013-11-06 04:23:59 Z LEAKED [domain 16] domain: (null)                 
                     16     0     1     --p--d       4.7
        
I imagine this is because the vif isn't being properly torn down on
shutdown... ifconfig and brctl etc logs don't show the devices living on
in dom0, I expect it is failing to release the grant maps.

http://www.chiark.greenend.org.uk/~xensrcts/logs/21490/test-amd64-amd64-xl/serial-fire-frog.log
 shows each domain still owns two pages. Which I would bet are the vif rings... 
Also:

Nov  6 04:24:19.449065 (XEN) [ref] localdom mfn      pin          localdom gmfn 
    flags
Nov  6 04:24:19.449101 (XEN) grant-table for remote domain:    1 (v2)
Nov  6 04:24:19.461055 (XEN) [ 10]        0 0x415f36 0x00000001          0 
0x415f36 0x18
Nov  6 04:24:19.461091 (XEN) [ 11]        0 0x415f35 0x00000001          0 
0x415f35 0x18

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel