On Wed, 28 Jul 2010, Leo wrote:
> Hi,
>
> I don't see anyone having similar issues when running domU kernel in HVM. Hopefully it's just some very obvious stuff I
> missed.
>
> I am using the 2.6.34-pvhvm-v6 branch from git://
xenbits.xen.org/people/sstabellini/linux-pvhvm. I compiled the kernel
> with just about everything as modules, including platform-pci, xen-blkfront and xen-netfront.
>
> When I use the kernel parameter xen_emul_unplug=ignore, both ata_piix and 8139cp drivers attach to the emulated devices
> and work just fine.
>
> When I unplug either the nic or the ide-disk, the loading order of the 3 modules makes some difference. If I load
> platform-pci first before either of the frontend drivers, I get XENBUS Timeout. Note that loading the platform-pci module
> doesn't cause any error, it is loading the frontend drivers when I see this error: (This happens when I load the
> xen-netfront driver. platform-pci is loaded at 58s. I also trimmed the wait time from 300s to 30s)
>
> [ 102.570177] Initialising Xen virtual ethernet driver.
> [ 107.676025] XENBUS: Waiting for devices to initialise: 25s...20s...15s...10s...5s...0s...
> [ 132.677038] XENBUS: Timeout connecting to device: device/vif/0 (local state 1, remote state 2)
>
> So I have to load the frontend drivers before platform-pci, and none of this error like this occured. And because of that
> I cannot compile in all the three modules as they'll always be waiting to connect.
>
> However I get some other error. When I unplug ide-disks, I get this:
>
> <4>[ 29.842072] ------------[ cut here ]------------
> <4>[ 29.842785] WARNING: at fs/sysfs/dir.c:451 sysfs_add_one+0xac/0xc0()
> <4>[ 29.843434] Hardware name: HVM domU
> <4>[ 29.844070] sysfs: cannot create duplicate filename '/block/xvda'
> <4>[ 29.844713] Modules linked in: 8139cp mii platform_pci xen_netfront xen_blkfront
> <4>[ 29.847272] Pid: 20, comm: xenwatch Not tainted 2.6.34-pvhvm #10
> <4>[ 29.847945] Call Trace:
> <4>[ 29.856027] [<ffffffff81146ebc>] ? sysfs_add_one+0xac/0xc0
> <4>[ 29.856722] [<ffffffff810424fa>] warn_slowpath_common+0x77/0x8f
> <4>[ 29.857372] [<ffffffff81042587>] warn_slowpath_fmt+0x64/0x66
> <4>[ 29.858015] [<ffffffff8106e6c8>] ? do_raw_spin_unlock+0x17/0x1b
> <4>[ 29.858722] [<ffffffff81146e32>] ? sysfs_add_one+0x22/0xc0
> <4>[ 29.859358] [<ffffffff812b3b94>] ? __mutex_lock_common+0x229/0x243
> <4>[ 29.860034] [<ffffffff810e433d>] ? __raw_local_irq_save+0x22/0x28
> <4>[ 29.860815] [<ffffffff81146d7a>] ? sysfs_pathname+0x37/0x3f
> <4>[ 29.861461] [<ffffffff81146d7a>] ? sysfs_pathname+0x37/0x3f
> <4>[ 29.862126] [<ffffffff81146ebc>] sysfs_add_one+0xac/0xc0
> <4>[ 29.862768] [<ffffffff811475f1>] create_dir+0x58/0x87
> <4>[ 29.863393] [<ffffffff810e433d>] ? __raw_local_irq_save+0x22/0x28
> <4>[ 29.864092] [<ffffffff81147658>] sysfs_create_dir+0x38/0x4f
> <4>[ 29.864833] [<ffffffff812b49f8>] ? _raw_spin_unlock+0x2d/0x38
> <4>[ 29.865525] [<ffffffff8119aa6d>] kobject_add_internal+0xdb/0x19b
> <4>[ 29.866186] [<ffffffff8119ac05>] kobject_add_varg+0x41/0x4e
> <4>[ 29.866858] [<ffffffff8119b0d0>] kobject_add+0x89/0x8b
> <4>[ 29.867590] [<ffffffff812098b6>] ? kzalloc+0xf/0x11
> <4>[ 29.868280] [<ffffffff812098b6>] ? kzalloc+0xf/0x11
> <4>[ 29.868936] [<ffffffff8119405c>] ? exact_lock+0x0/0x14
> <4>[ 29.869574] [<ffffffff810e71d8>] ? __kmalloc+0x106/0x118
> <4>[ 29.870232] [<ffffffff812098b6>] ? kzalloc+0xf/0x11
> <4>[ 29.870866] [<ffffffff8119a98a>] ? kobject_get+0x1a/0x22
> <4>[ 29.871510] [<ffffffff81209ea0>] ? get_device+0x14/0x1a
> <4>[ 29.872209] [<ffffffff8120a451>] device_add+0xdc/0x60c
> <4>[ 29.872866] [<ffffffff8120ee8e>] ? kobj_map+0x68/0x12a
> <4>[ 29.873506] [<ffffffff8114082b>] register_disk+0x3c/0x11d
> <4>[ 29.874240] [<ffffffff81194db7>] add_disk+0xb8/0x119
> <4>[ 29.874922] [<ffffffffa000125b>] backend_changed+0x441/0x45b [xen_blkfront]
> <4>[ 29.875604] [<ffffffff811eb6d2>] ? xenwatch_thread+0x0/0x152
> <4>[ 29.876245] [<ffffffff811ec5ac>] otherend_changed+0x82/0x8b
> <4>[ 29.876864] [<ffffffff811eb7ea>] xenwatch_thread+0x118/0x152
> <4>[ 29.877481] [<ffffffff8105c560>] ? autoremove_wake_function+0x0/0x38
> <4>[ 29.878134] [<ffffffff811eb6d2>] ? xenwatch_thread+0x0/0x152
> <4>[ 29.878886] [<ffffffff8105c35e>] kthread+0x69/0x71
> <4>[ 29.879517] [<ffffffff8100aa04>] kernel_thread_helper+0x4/0x10
> <4>[ 29.880198] [<ffffffff8105c2f5>] ? kthread+0x0/0x71
> <4>[ 29.880923] [<ffffffff8100aa00>] ? kernel_thread_helper+0x0/0x10
> <4>[ 29.908017] ---[ end trace 3f99b54d0b8663be ]---
> <3>[ 29.909904] kobject_add_internal failed for xvda with -EEXIST, don't try to register things with the same name in
> the same directory.
> <4>[ 29.911103] Pid: 20, comm: xenwatch Tainted: G W 2.6.34-pvhvm #10
> <4>[ 29.916378] Call Trace:
> <4>[ 29.917001] [<ffffffff8119a861>] ? kobject_put+0x47/0x4b
> <4>[ 29.917682] [<ffffffff8119aaeb>] kobject_add_internal+0x159/0x19b
> <4>[ 29.918389] [<ffffffff8119ac05>] kobject_add_varg+0x41/0x4e
> <4>[ 29.919124] [<ffffffff8119b0d0>] kobject_add+0x89/0x8b
> <4>[ 29.919834] [<ffffffff812098b6>] ? kzalloc+0xf/0x11
> <4>[ 29.920474] [<ffffffff812098b6>] ? kzalloc+0xf/0x11
> <4>[ 29.921191] [<ffffffff8119405c>] ? exact_lock+0x0/0x14
> <4>[ 29.921955] [<ffffffff810e71d8>] ? __kmalloc+0x106/0x118
> <4>[ 29.922690] [<ffffffff812098b6>] ? kzalloc+0xf/0x11
> <4>[ 29.923578] [<ffffffff8119a98a>] ? kobject_get+0x1a/0x22
> <4>[ 29.924324] [<ffffffff81209ea0>] ? get_device+0x14/0x1a
> <4>[ 29.925178] [<ffffffff8120a451>] device_add+0xdc/0x60c
> <4>[ 29.925873] [<ffffffff8120ee8e>] ? kobj_map+0x68/0x12a
> <4>[ 29.926567] [<ffffffff8114082b>] register_disk+0x3c/0x11d
> <4>[ 29.927195] [<ffffffff81194db7>] add_disk+0xb8/0x119
> <4>[ 29.927821] [<ffffffffa000125b>] backend_changed+0x441/0x45b [xen_blkfront]
> <4>[ 29.928566] [<ffffffff811eb6d2>] ? xenwatch_thread+0x0/0x152
> <4>[ 29.929211] [<ffffffff811ec5ac>] otherend_changed+0x82/0x8b
> <4>[ 29.929855] [<ffffffff811eb7ea>] xenwatch_thread+0x118/0x152
> <4>[ 29.930490] [<ffffffff8105c560>] ? autoremove_wake_function+0x0/0x38
> <4>[ 29.931151] [<ffffffff811eb6d2>] ? xenwatch_thread+0x0/0x152
> <4>[ 29.931787] [<ffffffff8105c35e>] kthread+0x69/0x71
> <4>[ 29.932550] [<ffffffff8100aa04>] kernel_thread_helper+0x4/0x10
> <4>[ 29.933201] [<ffffffff8105c2f5>] ? kthread+0x0/0x71
> <4>[ 29.933834] [<ffffffff8100aa00>] ? kernel_thread_helper+0x0/0x10
> <5>[ 29.966562] SCSI subsystem initialized
> <0>[ 29.984037] ------------[ cut here ]------------
> <2>[ 29.984721] kernel BUG at fs/sysfs/group.c:65!
> <0>[ 29.985357] invalid opcode: 0000 [#1] PREEMPT SMP
> <0>[ 29.987151] last sysfs file: /sys/class/firmware/timeout
> <4>[ 29.987978] CPU 0
> <4>[ 29.988014] Modules linked in: sd_mod scsi_mod ext3 jbd mbcache 8139cp mii platform_pci xen_netfront xen_blkfront
> <4>[ 29.988014]
> <4>[ 29.988014] Pid: 20, comm: xenwatch Tainted: G W 2.6.34-pvhvm #10 /HVM domU
> <4>[ 29.988014] RIP: 0010:[<ffffffff8114883a>] [<ffffffff8114883a>] internal_create_group+0x2a/0x127
> <4>[ 29.988014] RSP: 0018:ffff88001f97fd00 EFLAGS: 00010246
> <4>[ 29.988014] RAX: 00000000ffffffef RBX: ffff88001e200870 RCX: ffffffff8120a927
> <4>[ 29.988014] RDX: ffffffff81640280 RSI: 0000000000000000 RDI: ffff88001e200870
> <4>[ 29.988014] RBP: ffff88001f97fd40 R08: ffff88001f97e000 R09: ffff880001a14c20
> <4>[ 29.988014] R10: ffff88001f97fca0 R11: 0000000000000000 R12: ffff88001e0e88a8
> <4>[ 29.988014] R13: 000000000000000f R14: ffff88001e200860 R15: ffffffff81640280
> <4>[ 29.988014] FS: 0000000000000000(0000) GS:ffff880001a00000(0000) knlGS:0000000000000000
> <4>[ 29.988014] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> <4>[ 29.988014] CR2: 00007f151c82200f CR3: 000000001e7bc000 CR4: 00000000000006b0
> <4>[ 29.988014] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> <4>[ 29.988014] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> <4>[ 29.988014] Process xenwatch (pid: 20, threadinfo ffff88001f97e000, task ffff88001f975b80)
> <0>[ 29.988014] Stack:
> <4>[ 29.988014] 0000000000000000 0000000089700e79 ffff88001f97fd70 ffff88001e200800
> <4>[ 29.988014] <0> ffff88001e0e88a8 000000000000000f ffff88001e200860 0000000000000000
> <4>[ 29.988014] <0> ffff88001f97fd50 ffffffff81148958 ffff88001f97fd60 ffffffff810a39fe
> <0>[ 29.988014] Call Trace:
> <4>[ 29.988014] [<ffffffff81148958>] sysfs_create_group+0xe/0x12
> <4>[ 29.988014] [<ffffffff810a39fe>] blk_trace_init_sysfs+0x14/0x16
> <4>[ 29.988014] [<ffffffff8118ff11>] blk_register_queue+0x42/0xc8
> <4>[ 29.988014] [<ffffffff81194dbf>] add_disk+0xc0/0x119
> <4>[ 29.988014] [<ffffffffa000125b>] backend_changed+0x441/0x45b [xen_blkfront]
> <4>[ 29.988014] [<ffffffff811eb6d2>] ? xenwatch_thread+0x0/0x152
> <4>[ 29.988014] [<ffffffff811ec5ac>] otherend_changed+0x82/0x8b
> <4>[ 29.988014] [<ffffffff811eb7ea>] xenwatch_thread+0x118/0x152
> <4>[ 29.988014] [<ffffffff8105c560>] ? autoremove_wake_function+0x0/0x38
> <4>[ 29.988014] [<ffffffff811eb6d2>] ? xenwatch_thread+0x0/0x152
> <4>[ 29.988014] [<ffffffff8105c35e>] kthread+0x69/0x71
> <4>[ 29.988014] [<ffffffff8100aa04>] kernel_thread_helper+0x4/0x10
> <4>[ 29.988014] [<ffffffff8105c2f5>] ? kthread+0x0/0x71
> <4>[ 29.988014] [<ffffffff8100aa00>] ? kernel_thread_helper+0x0/0x10
> <0>[ 29.988014] Code: c3 55 48 89 e5 41 57 49 89 d7 41 56 41 55 41 54 53 48 89 fb 48 83 ec 18 48 85 ff 89 75 c0 74 0b 85
> f6 75 11 48 83 7f 30 00 75 04 <0f> 0b eb fe 83 7d c0 00 74 11 48 83 7b 30 00 41 bd ea ff ff ff
> <1>[ 29.988014] RIP [<ffffffff8114883a>] internal_create_group+0x2a/0x127
> <4>[ 29.988014] RSP <ffff88001f97fd00>
> <4>[ 30.132020] ---[ end trace 3f99b54d0b8663bf ]---
>
> Then when I do an fdisk on block device 202:0, I get this:
>
> <4>[ 223.840262] ------------[ cut here ]------------
> <4>[ 223.841061] WARNING: at fs/fs-writeback.c:1105 __mark_inode_dirty+0xed/0x130()
> <4>[ 223.842008] Hardware name: HVM domU
> <4>[ 223.842585] Modules linked in: ata_piix libata sd_mod scsi_mod ext3 jbd mbcache 8139cp mii platform_pci xen_netfront
> xen_blkfront
> <4>[ 223.847533] Pid: 875, comm: busybox Tainted: G D W 2.6.34-pvhvm #10
> <4>[ 223.848259] Call Trace:
> <4>[ 223.848994] [<ffffffff8110e33a>] ? __mark_inode_dirty+0xed/0x130
> <4>[ 223.849761] [<ffffffff810424fa>] warn_slowpath_common+0x77/0x8f
> <4>[ 223.850651] [<ffffffff81042521>] warn_slowpath_null+0xf/0x11
> <4>[ 223.851509] [<ffffffff8110e33a>] __mark_inode_dirty+0xed/0x130
> <4>[ 223.852195] [<ffffffff810b55ac>] ? sync_page_killable+0x0/0x3e
> <4>[ 223.853449] [<ffffffff811041cd>] mark_inode_dirty_sync+0xe/0x10
> <4>[ 223.854260] [<ffffffff81105554>] touch_atime+0xfb/0x120
> <4>[ 223.855144] [<ffffffff810b41a6>] file_accessed+0x17/0x19
> <4>[ 223.855956] [<ffffffff810b5ac4>] generic_file_aio_read+0x4da/0x53e
> <4>[ 223.856686] [<ffffffff810f27c7>] do_sync_read+0xc2/0x106
> <4>[ 223.857307] [<ffffffff810e433d>] ? __raw_local_irq_save+0x22/0x28
> <4>[ 223.858023] [<ffffffff8115aca4>] ? security_file_permission+0x11/0x13
> <4>[ 223.858993] [<ffffffff810f2e9d>] vfs_read+0xa8/0x102
> <4>[ 223.859658] [<ffffffff810f313f>] sys_read+0x47/0x6d
> <4>[ 223.860379] [<ffffffff81009c02>] system_call_fastpath+0x16/0x1b
> <4>[ 223.861296] ---[ end trace 3f99b54d0b8663c0 ]---
> <3>[ 223.861934] bdi-block not registered
>
> Reading from the block device returns some random data: (here I am doing an fdisk on the block device 202:0), and it's
> returning the wrong disk size. The virtual disk is 15GiB.
>
> Disk /dev/xvda: 4393 MB, 4393723904 bytes
> 255 heads, 63 sectors/track, 534 cylinders, total 8581492 sectors
> Units = sectors of 1 * 512 = 512 bytes
>
> Disk /dev/xvda doesn't contain a valid partition table
>
>
> If I unplug the nics, there is no error,
>
> [ 29.738321] Initialising Xen virtual ethernet driver.
> [ 29.748844] alloc irq_desc for 28 on node -1
> [ 29.748847] alloc kstat_irqs on node -1
> [ 29.748866] xen-platform-pci 0000:00:03.0: PCI INT A -> GSI 28 (level, low) -> IRQ 28
> [ 29.750216] Grant table initialized
> [ 29.753307] vif vif-0: 2 parsing device/vif/0/mac
>
> 8130cp, as expected, is not attached to eth0. However, it seems xen-netfront cannot get the MAC address of eth0:
>
> eth0 Link encap:Ethernet HWaddr 00:00:00:00:00:00
> BROADCAST MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
>
> Is it something very obvious that I missed?
