|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-users] Live-migrated domain crashes
Hi all,
I wonder if anyone has had any experience with the
live migration capabilites of Xen.
I've managed to install and start Xen 2.0.6 on two
Linux hosts, both running Mandrake 9.1 (I'm losing my
CDROM when I boot up with this 2.6.11.10 kernel
though).
I've also managed to create a domU on each using the
ttylinux as described in the user doc.
However, when I try live-migrating it from hostA to
hostB the operation appers to be successfull as the
new domU appers on hostB's 'xm list' output, but
crashes when I try 'xm console ExampleDomain'.
An if I try live-migrating it form hostB to hostA it
only briefly appers on the hostA's 'xm list' output
with status code 'p' before crashing.
This si the console output after I migrated
ExampleDomain from hostA to hostB and tried loging in.
I've also attached the log files from both hosts:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
[hostB]# xm list
Name Id Mem(MB) CPU State Time(s)
Console
Domain-0 0 827 0 r---- 2712.3
ExampleDomain 5 64 0 -b--- 0.0
9605
[hostB]# xm console ExampleDomain
************ REMOTE CONSOLE: CTRL-] TO QUIT ********
invalid operand: 0000 [#1]
PREEMPT
Modules linked in:
CPU: 0
EIP: 0061:[<c0220b62>] Not tainted VLI
EFLAGS: 00010292 (2.6.11.10-xenU)
EIP is at __make_request+0x52/0x510
eax: c0332dc4 ebx: 00000001 ecx: c10ac6f0 edx:
c00c5758
esi: c00c5758 edi: 00000000 ebp: c10aba60 esp:
c0023ca8
ds: 007b es: 007b ss: 0069
Process kjournald (pid: 573, threadinfo=c0022000
task=c0376570)
Stack: c00c5758 c0023ce0 00000003 00000000 0000201c
00000002 00000002 00000000
c00c5758 c0355a00 c012e3d0 c10aba60 c0221497
c00c5758 c10aba60 c10bb550
c0376570 00000000 c0376570 c012e3d0 c0023d18
c0023d18 c035c46c c035c474
Call Trace:
[<c012e3d0>] autoremove_wake_function+0x0/0x60
[<c0221497>] generic_make_request+0x157/0x1f0
[<c012e3d0>] autoremove_wake_function+0x0/0x60
[<c012e3d0>] autoremove_wake_function+0x0/0x60
[<c0138b43>] mempool_alloc+0x73/0x1b0
[<c012e3d0>] autoremove_wake_function+0x0/0x60
[<c0221592>] submit_bio+0x62/0x100
[<c015a437>] bio_alloc+0xe7/0x1e0
[<c0156840>] end_buffer_write_sync+0x0/0x80
[<c0159d65>] submit_bh+0xd5/0x120
[<c0159e1d>] ll_rw_block+0x6d/0x90
[<c01ce6c1>] journal_commit_transaction+0x1041/0x1260
[<c016e4f4>] d_rehash+0x54/0x80
[<c017785e>] simple_lookup+0x3e/0x50
[<c01057b8>] ctrl_if_send_message_noblock+0xc8/0x120
[<c01d0bff>] kjournald+0xdf/0x270
[<c012e3d0>] autoremove_wake_function+0x0/0x60
[<c012e3d0>] autoremove_wake_function+0x0/0x60
[<c0109202>] ret_from_fork+0x6/0x1c
[<c01d0b00>] commit_timeout+0x0/0x10
[<c01d0b20>] kjournald+0x0/0x270
[<c0107275>] kernel_thread_helper+0x5/0x10
Code: 16 8d 04 40 8b 44 81 04 c1 e8 09 89 44 24 14 8b
5a 10 8d 44 24 38 89 44 24 04 89 34 24 83 e3 01 e8 54
21 f2 ff 8b 86 ec 00 00 00 <0f> 0d 08 90 8b 44 24 38
8b 68 10 83 e5 04 74 0e f6 86 e9 00 00
<1>invalid operand: 0000 [#2]
PREEMPT
Modules linked in:
CPU: 0
EIP: 0061:[<c0220b62>] Not tainted VLI
EFLAGS: 00010292 (2.6.11.10-xenU)
EIP is at __make_request+0x52/0x510
eax: c0332dc4 ebx: 00000001 ecx: c10ac750 edx:
c00c5758
esi: c00c5758 edi: 00000000 ebp: c10ab1a0 esp:
c037dc14
ds: 007b es: 007b ss: 0069
Process pdflush (pid: 34, threadinfo=c037c000
task=c10bb550)
Stack: c00c5758 c037dc4c c0220dc6 c00c5758 0000201e
00000002 00000002 00004042
c00c5758 c0355a00 c012e3d0 c10ab1a0 c0221497
c00c5758 c10ab1a0 00000000
00000030 00000000 c10bb550 c012e3d0 c037dc84
c037dc84 c035c46c c035c474
Call Trace:
[<c0220dc6>] __make_request+0x2b6/0x510
[<c012e3d0>] autoremove_wake_function+0x0/0x60
[<c0221497>] generic_make_request+0x157/0x1f0
[<c012e3d0>] autoremove_wake_function+0x0/0x60
[<c012e3d0>] autoremove_wake_function+0x0/0x60
[<c0138b43>] mempool_alloc+0x73/0x1b0
[<c012e3d0>] autoremove_wake_function+0x0/0x60
[<c0221592>] submit_bio+0x62/0x100
[<c015a437>] bio_alloc+0xe7/0x1e0
[<c0157190>] end_buffer_async_write+0x0/0x150
[<c0159d65>] submit_bh+0xd5/0x120
[<c0158550>] __block_write_full_page+0x170/0x340
[<c013e3e9>] kmem_cache_alloc+0x69/0x70
[<c0159bd5>] block_write_full_page+0xd5/0x100
[<c01bbcb0>] ext3_get_block+0x0/0xa0
[<c01bc901>] ext3_ordered_writepage+0xd1/0x1b0
[<c01bbcb0>] ext3_get_block+0x0/0xa0
[<c01bc7f0>] bget_one+0x0/0x10
[<c017a4b3>] mpage_writepages+0x263/0x3b0
[<c015ce40>] blkdev_writepage+0x0/0x30
[<c01bc830>] ext3_ordered_writepage+0x0/0x1b0
[<c013b832>] do_writepages+0x42/0x50
[<c0178991>] __sync_single_inode+0x71/0x210
[<c0178b97>] __writeback_single_inode+0x67/0x160
[<c010797a>] __switch_to+0x2a/0x380
[<c0178e27>] sync_sb_inodes+0x197/0x2a0
[<c0179014>] writeback_inodes+0xe4/0x100
[<c013b5f6>] wb_kupdate+0x96/0x110
[<c013c188>] __pdflush+0xf8/0x220
[<c013c2b0>] pdflush+0x0/0x30
[<c013c2d6>] pdflush+0x26/0x30
[<c013b560>] wb_kupdate+0x0/0x110
[<c013c2b0>] pdflush+0x0/0x30
[<c012deaa>] kthread+0xaa/0xb0
[<c012de00>] kthread+0x0/0xb0
[<c0107275>] kernel_thread_helper+0x5/0x10
Code: 16 8d 04 40 8b 44 81 04 c1 e8 09 89 44 24 14 8b
5a 10 8d 44 24 38 89 44 24 04 89 34 24 83 e3 01 e8 54
21 f2 ff 8b 86 ec 00 00 00 <0f> 0d 08 90 8b 44 24 38
8b 68 10 83 e5 04 74 0e f6 86 e9 00 00
ttylinux 4.2
Linux 2.6.11.10-xenU on i686 arch
tiny.local login: root
<1>invalid operand: 0000 [#3]
PREEMPT
Modules linked in:
CPU: 0
EIP: 0061:[<c016f7af>] Not tainted VLI
************ REMOTE CONSOLE EXITED *****************
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
___________________________________________________________
How much free photo storage do you get? Store your holiday
snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.comnetwork start bridge=xen-br0 netdev=eth0 antispoof=no
VIRTUAL MEMORY ARRANGEMENT:
Loaded kernel: c0100000->c033bb84
Init. ramdisk: c033c000->c033c000
Phys-Mach map: c033c000->c034c000
Page tables: c034c000->c034e000
Start info: c034e000->c034f000
Boot stack: c034f000->c0350000
TOTAL: c0000000->c0400000
ENTRY ADDRESS: c0100000
/usr/lib/python/xen/xend/server/controller.py:183: DeprecationWarning:
Deferred.setTimeout is deprecated. Look for timeout support specific to the
API you are using instead.
deferred.setTimeout(self.timeout)
vif-bridge up vif=vif1.0 domain=ExampleDomain mac=aa:00:00:6f:68:cd
bridge=xen-br0
recv_fe_driver_status> {'status': 1}
recv_fe_driver_status>
recv_fe_interface_connect {'tx_shmem_frame': 130971, 'rx_shmem_frame': 130970,
'handle': 0}
/usr/lib/python/xen/xend/XendMigrate.py:261: DeprecationWarning:
Deferred.setTimeout is deprecated. Look for timeout support specific to the
API you are using instead.
d.setTimeout(self.timeout)
{}
VirqClient.virqReceived> 4
vif-bridge down vif=vif1.0 domain=ExampleDomain mac=aa:00:00:6f:68:cd
bridge=xen-br0
_op_migrate_cb> <xen.xend.XendMigrate.XendMigrateInfo instance at 0x850ed44>
<POST /xend/domain/ExampleDomain HTTP/1.1>
_op_migrate_cb> url= http://giotto.doc.ic.ac.uk:8002/xend/domain/5
[2005-05-27 11:42:54 xend] INFO (SrvDaemon:610) Xend Daemon started
[2005-05-27 11:42:56 xend] INFO (XendRoot:112) EVENT> xend.start 0
[2005-05-27 11:47:38 xend] DEBUG (XendDomainInfo:720) init_domain> Created
domain=1 name=ExampleDomain memory=64
[2005-05-27 11:47:38 xend] INFO (console:94) Created console id=11 domain=1
port=9601
[2005-05-27 11:47:39 xend] DEBUG (XendDomainInfo:1130) Creating vbd dom=1
uname=file:/data/vesso/ttylinux-xen
[2005-05-27 11:47:39 xend] DEBUG (blkif:155) Connecting blkif
<BlkifBackendInterface 1 0>
[2005-05-27 11:47:39 xend] DEBUG (XendDomainInfo:1107) Creating vif dom=1 vif=0
mac=aa:00:00:6f:68:cd
[2005-05-27 11:47:39 xend] INFO (XendRoot:112) EVENT> xend.console.create [11,
1, 9601]
[2005-05-27 11:47:39 xend] INFO (XendRoot:112) EVENT> xend.domain.create
['ExampleDomain', '1']
[2005-05-27 11:47:39 xend] INFO (XendRoot:112) EVENT> xend.domain.unpause
['ExampleDomain', '1']
[2005-05-27 11:47:40 xend] DEBUG (blkif:203) Connecting blkif to event channel
<BlkifBackendInterface 1 0> ports=12:3
[2005-05-27 11:48:09 xend] INFO (console:44) Console connected 11 127.0.0.1 1092
[2005-05-27 11:48:09 xend] INFO (XendRoot:112) EVENT> xend.console.connect [11,
'127.0.0.1', 1092]
[2005-05-27 11:48:38 xend] INFO (console:58) Console disconnected 11 127.0.0.1
1092
[2005-05-27 11:48:38 xend] INFO (XendRoot:112) EVENT> xend.console.disconnect
[11, '127.0.0.1', 1092]
[2005-05-27 11:52:02 xend] INFO (XendMigrate:323) Migrate BEGIN: ['migrate',
['id', '1'], ['state', 'begin'], ['live', 1], ['resource', 0], ['src', ['host',
'mir'], ['domain', '1']], ['dst', ['host', 'giotto.doc.ic.ac.uk']]]
[2005-05-27 11:52:02 xend] INFO (XendRoot:112) EVENT> xend.domain.migrate
['ExampleDomain', '1', 'begin', ['migrate', ['id', '1'], ['state', 'begin'],
['live', 1], ['resource', 0], ['src', ['host', 'mir'], ['domain', '1']],
['dst', ['host', 'giotto.doc.ic.ac.uk']]]]
[2005-05-27 11:52:08 xend] DEBUG (XendDomain:487) domain_restart_schedule> 1
suspend 1
[2005-05-27 11:52:08 xend] INFO (XendRoot:112) EVENT> xend.domain.shutdown
['ExampleDomain', '1', 'suspend']
[2005-05-27 11:52:08 xend] DEBUG (XendDomain:244) XendDomain>reap> domain died
name=ExampleDomain id=1
[2005-05-27 11:52:08 xend] DEBUG (XendDomain:247) XendDomain>reap> shutdown
id=1 reason=suspend
[2005-05-27 11:52:08 xend] INFO (XendRoot:112) EVENT> xend.virq 4
[2005-05-27 11:52:08 xend] INFO (XendRoot:112) EVENT> xend.domain.suspended
['ExampleDomain', '1']
[2005-05-27 11:52:08 xend] INFO (XendDomain:564) Destroying domain:
name=ExampleDomain
[2005-05-27 11:52:08 xend] DEBUG (XendDomainInfo:665) Destroying vifs for
domain 1
[2005-05-27 11:52:08 xend] DEBUG (netif:305) Destroying vif domain=1 vif=0
[2005-05-27 11:52:08 xend] DEBUG (XendDomainInfo:674) Destroying vbds for
domain 1
[2005-05-27 11:52:08 xend] DEBUG (blkif:552) Destroying blkif domain=1
[2005-05-27 11:52:08 xend] DEBUG (blkif:408) Destroying vbd domain=1 idx=0
[2005-05-27 11:52:08 xend] DEBUG (XendDomainInfo:634) Closing console, domain 1
[2005-05-27 11:52:08 xend] DEBUG (XendDomainInfo:622) Closing channel to domain
1
[2005-05-27 11:52:08 xend] INFO (XendRoot:112) EVENT> xend.domain.destroy
['ExampleDomain', '1']
[2005-05-27 11:52:08 xend] DEBUG (blkif:363) Unbinding vbd (type file) from
/dev/loop0
[2005-05-27 11:52:08 xend] INFO (XendDomain:564) Destroying domain:
name=ExampleDomain
[2005-05-27 11:52:08 xend] DEBUG (XendDomainInfo:634) Closing console, domain 1
[2005-05-27 11:52:08 xend] ERROR (XendDomainInfo:627) Domain destroy failed:
ExampleDomain
Traceback (most recent call last):
File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 625, in destroy_domain
return xc.domain_destroy(dom=self.dom)
error: (3, 'No such process')
[2005-05-27 11:52:08 xend] INFO (XendRoot:112) EVENT> xend.domain.destroy
['ExampleDomain', '1']
[2005-05-27 11:52:08 xend] INFO (XendMigrate:345) Migrate OK: ['migrate',
['id', '1'], ['state', 'ok'], ['live', 1], ['resource', 0], ['src', ['host',
'mir'], ['domain', '1']], ['dst', ['host', 'giotto.doc.ic.ac.uk'], ['domain',
5]]]
[2005-05-27 11:52:08 xend] INFO (XendRoot:112) EVENT> xend.domain.died
['ExampleDomain', '1']
[2005-05-27 11:52:08 xend] INFO (XendRoot:112) EVENT> xend.domain.migrate
['ExampleDomain', '1', 'ok', ['migrate', ['id', '1'], ['state', 'ok'], ['live',
1], ['resource', 0], ['src', ['host', 'mir'], ['domain', '1']], ['dst',
['host', 'giotto.doc.ic.ac.uk'], ['domain', 5]]]]
network start bridge=xen-br0 netdev=eth0 antispoof=no
/usr/lib/python/xen/xend/server/controller.py:183: DeprecationWarning:
Deferred.setTimeout is deprecated. Look for timeout support specific to the
API you are using instead.
deferred.setTimeout(self.timeout)
vif-bridge up vif=vif5.0 domain=ExampleDomain mac=aa:00:00:6f:68:cd
bridge=xen-br0
recv_fe_driver_status> {'status': 1}
recv_fe_interface_connect {'tx_shmem_frame': 4135, 'rx_shmem_frame': 4134,
'handle': 0}
VirqClient.virqReceived> 4
vif-bridge down vif=vif5.0 domain=ExampleDomain mac=aa:00:00:6f:68:cd
bridge=xen-br0
[2005-05-27 11:46:39 xend] INFO (SrvDaemon:610) Xend Daemon started [2005-05-27 11:46:39 xend] INFO (XendRoot:112) EVENT> xend.start 0 [2005-05-27 11:52:02 xend] INFO (XendRoot:112) EVENT> xend.domain.create ['Domain-5', '5'] [2005-05-27 11:52:02 xend] DEBUG (XendDomain:384) domain_configure> id=5 config=['vm', ['name', 'ExampleDomain'], ['memory', '64'], ['cpu', '-1'], ['image', ['linux', ['kernel', '/boot/vmlinuz-2.6-xenU'], ['ip', ':1.2.3.4::::eth0:dhcp'], ['root', '/dev/sda1 ro'], ['args', '4']]], ['device', ['vbd', ['uname', 'file:/data/vesso/ttylinux-xen'], ['dev', 'sda1'], ['mode', 'w']]], ['device', ['vif', ['mac', 'aa:00:00:6f:68:cd']]]] [2005-05-27 11:52:02 xend] INFO (console:94) Created console id=13 domain=5 port=9605 [2005-05-27 11:52:02 xend] DEBUG (XendDomainInfo:1130) Creating vbd dom=5 uname=file:/data/vesso/ttylinux-xen [2005-05-27 11:52:02 xend] DEBUG (blkif:155) Connecting blkif <BlkifBackendInterface 5 0> [2005-05-27 11:52:02 xend] DEBUG (XendDomainInfo:1107) Creating vif dom=5 vif=0 mac=aa:00:00:6f:68:cd [2005-05-27 11:52:02 xend] INFO (XendRoot:112) EVENT> xend.console.create [13, 5, 9605] [2005-05-27 11:52:02 xend] INFO (XendRoot:112) EVENT> xend.domain.create ['ExampleDomain', '5'] [2005-05-27 11:52:08 xend] DEBUG (blkif:203) Connecting blkif to event channel <BlkifBackendInterface 5 0> ports=15:4 [2005-05-27 11:53:41 xend] INFO (console:44) Console connected 13 127.0.0.1 37962 [2005-05-27 11:53:41 xend] INFO (XendRoot:112) EVENT> xend.console.connect [13, '127.0.0.1', 37962] [2005-05-27 11:53:53 xend] DEBUG (XendDomain:244) XendDomain>reap> domain died name=ExampleDomain id=5 [2005-05-27 11:53:53 xend] INFO (XendDomain:564) Destroying domain: name=ExampleDomain [2005-05-27 11:53:53 xend] DEBUG (XendDomainInfo:665) Destroying vifs for domain 5 [2005-05-27 11:53:53 xend] DEBUG (netif:305) Destroying vif domain=5 vif=0 [2005-05-27 11:53:53 xend] DEBUG (XendDomainInfo:674) Destroying vbds for domain 5 [2005-05-27 11:53:53 xend] DEBUG (blkif:552) Destroying blkif domain=5 [2005-05-27 11:53:53 xend] DEBUG (blkif:408) Destroying vbd domain=5 idx=0 [2005-05-27 11:53:53 xend] DEBUG (XendDomainInfo:634) Closing console, domain 5 [2005-05-27 11:53:53 xend] DEBUG (XendDomainInfo:622) Closing channel to domain 5 [2005-05-27 11:53:53 xend] INFO (XendRoot:112) EVENT> xend.virq 4 [2005-05-27 11:53:53 xend] DEBUG (blkif:363) Unbinding vbd (type file) from /dev/loop0 [2005-05-27 11:53:53 xend] INFO (console:58) Console disconnected 13 127.0.0.1 37962 [2005-05-27 11:53:53 xend] INFO (XendRoot:112) EVENT> xend.domain.exit ['ExampleDomain', '5', 'crash'] [2005-05-27 11:53:53 xend] INFO (XendRoot:112) EVENT> xend.domain.destroy ['ExampleDomain', '5'] [2005-05-27 11:53:53 xend] INFO (XendRoot:112) EVENT> xend.console.disconnect [13, '127.0.0.1', 37962] [2005-05-27 11:53:54 xend] INFO (XendRoot:112) EVENT> xend.domain.died ['ExampleDomain', '5'] _______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-users
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |