Xen project Mailing List

But if I reconfigure my setup to use one LV partition per DRBD Physical Volume, the IO errors go away.

Hope this may help someone in future.

Jai.

On 25/06/2007, at 11:16 AM, jai lamerton wrote:

Hi,

Sorry for the need of the long winded email. Looking for some answers to the following.

I am setting up a xen PV domU on top of a LVM partitioned DRBD device. Everything was going just fine until I tried to test the filesystems in the domU.

Here is my setup;
Dom0 OS: CentOS release 5 (Final)
Kernel: 2.6.18-8.1.4.el5.centos.plusxen
Xen: xen-3.0.3-25.0.3.el5
DRBD: drbd-8.0.3-1.el5.centos

PV DomU OS: CentOS release 4.5 (Final)
Kernel: 2.6.9-55.ELxenU

my drbd.conf
global {
usage-count yes;
}
common {
startup {
wfc-timeout 0;
degr-wfc-timeout 60;
}
disk {
on-io-error detach;
}
syncer {
rate 100M;
al-extents 257;

}
net {
max-buffers   2048;
unplug-watermark   2048;
max-epoch-size 2048;
ko-count 4;
cram-hmac-alg "sha1";
shared-secret "asecretstring";
after-sb-0pri disconnect;
after-sb-1pri consensus;
after-sb-2pri disconnect;
rr-conflict disconnect;
}
}
resource r0 {
protocol C;
handlers {
pri-on-incon-degr "echo 0 > /proc/sysrq-trigger ; halt -f";
pri-lost-after-sb "echo o > /proc/sysrq-trigger ; halt -f";
local-io-error "echo 0 > /proc/sysrq-trigger ; halt -f";
outdate-peer "/usr/lib64/heartbeat/drbd-peer-outdater";
}
on xenhostA {
device /dev/drbd0;
disk /dev/sda3;
address 172.16.0.30:7788;
meta-disk internal;
}

on xenhostB {
device /dev/drbd0;
disk /dev/sda3;
address 172.16.0.31:7788;
meta-disk internal;
}
}

LVM
# pvscan
PV /dev/drbd0   VG VGvm1 lvm2 [15.54 GB / 3.05 GB free]
# lvscan
ACTIVE '/dev/VGvm1/LVroot' [10.00 GB] inherit
ACTIVE '/dev/VGvm1/LVtmp' [500.00 MB] inherit
ACTIVE '/dev/VGvm1/LVswap' [2.00 GB] inherit

lvm.conf has;
filter = [ "r|/dev/sda3|", "a|/dev/drbd0|" ]

My xen domU conf
import os, re
arch = os.uname()[4]
if re.search('64', arch):
arch_libdir = 'lib64'
else:
arch_libdir = 'lib'
name = "centos45domU"
memory = "512"
disk = [ 'phy:/dev/mapper/VGvm1-LVroot,xvda1,w',
   'phy:/dev/mapper/VGvm1-LVtmp,xvda2,w',
   'phy:/dev/mapper/VGvm1-LVswap,xvda3,w', ]
vif = [ 'mac=00:16:3e:d2:14:70, bridge=xenbr0', ]
vfb = ["type=vnc,vncunused=1"]
bootloader="/usr/bin/pygrub"
vcpus=2
on_reboot   = 'restart'
on_crash = 'restart'

DomU /etc/fstab
/dev/xvda1 /    ext3 defaults,usrquota 1 1
none /dev/pts devpts gid=5,mode=620 0 0
none /dev/shm tmpfs   defaults 0 0
none /proc    proc defaults 0 0
none /sys sysfs   defaults 0 0
/dev/xvda2 /tmp ext3 defaults 0 0
/dev/xvda3 swap swap defaults 0 0

** Problem **
While doing some tests writing files in domU I got;

# dd if=/dev/zero of=/deleteme bs=1M count=100
100+0 records in
100+0 records out

# dd if=/dev/zero of=/tmp/deleteme bs=1M count=100
end_request: I/O error, dev xvda2, sector 366778
end_request: I/O error, dev xvda2, sector 366844
end_request: I/O error, dev xvda2, sector 381116
printk: 8071 messages suppressed.
Buffer I/O error on device xvda2, logical block 190558
lost page write due to I/O error on xvda2
end_request: I/O error, dev xvda2, sector 381182
end_request: I/O error, dev xvda2, sector 415678
end_request: I/O error, dev xvda2, sector 450164
end_request: I/O error, dev xvda2, sector 1204
end_request: I/O error, dev xvda2, sector 1340
end_request: I/O error, dev xvda2, sector 1454
end_request: I/O error, dev xvda2, sector 1522
end_request: I/O error, dev xvda2, sector 1652
end_request: I/O error, dev xvda2, sector 1788
end_request: I/O error, dev xvda2, sector 1902
Aborting journal on device xvda2.
ext3_abort called.
EXT3-fs error (device xvda2): ext3_journal_start_sb: Detected aborted journal
Remounting filesystem read-only

DomU /var/log/messages contains
Jun 25 10:34:29 xenpv1 kernel: SELinux: initialized (dev xvda2, type ext3),uses xattr
Jun 25 10:34:34 xenpv1 kernel: end_request: I/O error, dev xvda2, sector 264182
Jun 25 10:34:34 xenpv1 kernel: printk: 32 messages suppressed.
Jun 25 10:34:34 xenpv1 kernel: Buffer I/O error on device xvda2, logical block 132091
Jun 25 10:34:34 xenpv1 kernel: lost page write due to I/O error on xvda2
Jun 25 10:34:34 xenpv1 kernel: Buffer I/O error on device xvda2, logical block 132092

Dom0 /var/log/messages contains
Jun 25 10:34:34 xenhosta kernel: drbd0: bio would need to, but cannot, be split: (vcnt=4,idx=0,size=4096,sector=10750334)
Jun 25 10:34:34 xenhosta kernel: drbd0: bio would need to, but cannot, be split: (vcnt=4,idx=0,size=4096,sector=10750458)
Jun 25 10:34:34 xenhosta kernel: drbd0: bio would need to, but cannot, be split: (vcnt=4,idx=0,size=4096,sector=10750524)
Jun 25 10:34:34 xenhosta kernel: drbd0: bio would need to, but cannot, be split: (vcnt=4,idx=0,size=4096,sector=10750590)

However running the same test on the device that is assigned to xvda2 mounted under dom0, there is no errors.

I will also ask the DRBD lists what "bio would need to, but cannot, be split:" means.

If anyone can shed some light on this, would be great.

Thank you kindly.
Jai.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

NTech Media

Media to the Nth Degree

PO Box 1183, Lismore, NSW 2480 Australia

Web: http://www.ntechmedia.com

ph (w): (02) 66 22 27 25

ph (m): (+61 ) 4 04 466 589

Fax: (02) 66 22 27 16

Re: [Xen-users] I/O errors in domU with LVM on DRBD