Re: [Xen-devel] [Qemu-devel] use O_DIRECT to open disk images for IDE failed under xen-4.1.2 and qemu upstream

["Gonglei (Arei)" <arei.gonglei@xxxxxxxxxx>]

> -----Original Message-----
> From: Stefan Hajnoczi [mailto:stefanha@xxxxxxxxx]
> Sent: Friday, May 24, 2013 8:20 PM
> To: Gonglei (Arei)
> Cc: Stefano Stabellini; Hanweidong; Luonengjun; qemu-devel@xxxxxxxxxx;
> Wangzhenguo; xen-devel@xxxxxxxxxxxxx; Huangweidong (Hardware)
> Subject: Re: [Qemu-devel] use O_DIRECT to open disk images for IDE failed
> under xen-4.1.2 and qemu upstream
> 
> On Fri, May 24, 2013 at 02:59:05AM +0000, Gonglei (Arei) wrote:
> > Hi,
> >
> > >
> > > On Thu, 23 May 2013, Gonglei (Arei) wrote:
> > > > Hi, all
> > > >
> > > > I use O_DIRECT to open disk images for IDE, but I'm failed. After 
> > > > debug, I
> get
> > > the below logs:
> > > > [2013-05-22 23:25:46] ide: CMD=c8
> > > > [2013-05-22 23:25:46] bmdma: readb 0x00 : 0x08
> > > > [2013-05-22 23:25:46] bmdma: writeb 0x00 : 0x09
> > > > [2013-05-22 23:25:46] bmdma_cmd_writeb: 0x00000009
> > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01
> > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01
> > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01
> > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01
> > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01
> > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01
> > > > [2013-05-22 23:26:39] bmdma: writeb 0x00 : 0x08
> > > > [2013-05-22 23:26:39] bmdma_cmd_writeb: 0x00000008
> > > > [2013-05-22 23:26:56] ====== offset:0 buf:0x7ff100f21c00 count:512
> > > aio_offset:0
> > > > [2013-05-22 23:31:30] ====== offset:0 buf:0x7ff100f21c00 count:512
> > > aio_offset:0
> > > > [2013-05-22 23:31:30] ====== handle_aiocb_rw_linear errno: -14
> > > > [2013-05-22 23:31:30] ====== paio_complete errno=14
> > > > [2013-05-22 23:31:30] ====== ide_dma_error!!!
> > > > [2013-05-22 23:31:30] ide: read status addr=0x3f6 val=41
> > > >
> > > > QEMU command line :
> > > > qemu-system-i386 -xen-domid 837 -chardev
> > > socket,id=libxl-cmd,path=/var/run/xen/qmp-libxl-837,server,nowait -mon
> > > chardev=libxl-cmd,mode=control -name suse11 -vnc 0.0.0.0:1 -serial pty
> -boot
> > > order=c -usb -usbdevice tablet -smp 2,maxcpus=2 -device
> > > rtl8139,id=nic0,netdev=net0,mac=00:16:3e:13:d3:72 -netdev
> > > type=tap,id=net0,ifname=tap837.0,downscript=no -M xenfv -m 2040 -drive
> > >
> file=/mnt/sdd/image/suse.image,if=ide,index=0,media=disk,format=raw,cache
> > > =none
> > > >
> > > > errno 14 shows Bad Address. And I find QEMU_AIO_MISALIGNED flag bit
> is
> > > not set through debug.
> > > >
> > > >     /*
> > > >      * If O_DIRECT is used the buffer needs to be aligned on a sector
> > > >      * boundary.  Check if this is the case or tell the low-level
> > > >      * driver that it needs to copy the buffer.
> > > >      */
> > > >     if ((bs->open_flags & BDRV_O_NOCACHE)) {
> > > >         if (!bdrv_qiov_is_aligned(bs, qiov)) {    //if the address is
> > > aligned-512, will no meet the conditions
> > > >             type |= QEMU_AIO_MISALIGNED;
> > > > #ifdef CONFIG_LINUX_AIO
> > > >         } else if (s->use_aio) {
> > > >             return laio_submit(bs, s->aio_ctx, s->fd, sector_num, qiov,
> > > >                                nb_sectors, cb, opaque, type);
> > > > #endif
> > > >
> > > > Next process:
> > > > static ssize_t handle_aiocb_rw(struct qemu_paiocb *aiocb)
> > > > {
> > > >     ssize_t nbytes;
> > > >     char *buf;
> > > >
> > > >     if (!(aiocb->aio_type & QEMU_AIO_MISALIGNED)) {
> > > >         /*
> > > >          * If there is just a single buffer, and it is properly aligned
> > > >          * we can just use plain pread/pwrite without any problems.
> > > >          */
> > > >         if (aiocb->aio_niov == 1)
> > > >              return handle_aiocb_rw_linear(aiocb,
> > > aiocb->aio_iov->iov_base); //this way, and reports errno 14 next
> > > >
> > > > Anyone have a good method to resolve this bug? Thanks!
> > >
> > > I know that this is not the answer you are looking for but why do you
> > > want to use O_DIRECT with IDE?
> > > It should be perfectly safe to use write-back.
> >
> > A few days ago, I asked a question about the IDE FLUSH time of guestï
> > http://lists.nongnu.org/archive/html/qemu-devel/2013-05/msg02642.html
> >
> > finally I found that because Qemu use write-back flag to open disk images by
> default.
> > so I hope to use O_DIRECT to avoid meeting that problem, but I'm failed
> under Xen platform with Qemu upstream.
> 
> cache=none does not eliminate the need for flush.  If you want to do
> that, try cache=writethrough or cache=directsync.
> 
I had tried it, but I got the same error.

> Regarding the EFAULT you are seeing, did you check if the I/O buffer
> address is valid?  Try breaking in gdb and inspecting /proc/<pid>/maps
> or just x <buffer-address> in gdb to see if the buffer is readable.
> 
> Stefan

Hi, Stefan

The follows are details in gdb:
handle_aiocb_rw_linear (aiocb=0x13a0f80, buf=0x7f0a637d7c00 <Address 
0x7f0a637d7c00 out of bounds>)
    at /mnt/sdd/gonglei/xen-4.1.2/tools/qemu-xen/posix-aio-compat.c:213
213     {
(gdb) bt
#0  handle_aiocb_rw_linear (aiocb=0x13a0f80, buf=0x7f0a637d7c00 <Address 
0x7f0a637d7c00 out of bounds>)
    at /mnt/sdd/gonglei/xen-4.1.2/tools/qemu-xen/posix-aio-compat.c:213
#1  0x000000000058ba4f in handle_aiocb_rw (aiocb=0x13a0f80) at 
/mnt/sdd/gonglei/xen-4.1.2/tools/qemu-xen/posix-aio-compat.c:254
#2  0x000000000058bde2 in aio_thread (unused=0x0) at 
/mnt/sdd/gonglei/xen-4.1.2/tools/qemu-xen/posix-aio-compat.c:351
#3  0x00007f0a6a1ec7b6 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f0a68f0e9cd in clone () from /lib64/libc.so.6
#5  0x0000000000000000 in ?? ()
(gdb) x 0x7f0a637d7c00
0x7f0a637d7c00: Cannot access memory at address 0x7f0a637d7c00
(gdb) n
214         ssize_t offset = 0;
(gdb) 
217         while (offset < aiocb->aio_nbytes) {
(gdb) n
218              if (aiocb->aio_type & QEMU_AIO_WRITE)
(gdb) 
224                  len = pread(aiocb->aio_fildes,
(gdb) 
229              if (len == -1 && errno == EINTR)
(gdb) 
231              else if (len == -1) {
(gdb) 
232                  offset = -errno;
(gdb) p errno
$1 = 14

linux-CGRmYS:~ # cat /proc/21837/maps 
00400000-00846000 r-xp 00000000 08:05 28601                              
/usr/lib/xen/bin/qemu-system-i386
00a45000-00a46000 r--p 00445000 08:05 28601                              
/usr/lib/xen/bin/qemu-system-i386
00a46000-00a92000 rw-p 00446000 08:05 28601                              
/usr/lib/xen/bin/qemu-system-i386
00a92000-01344000 rw-p 00000000 00:00 0                                  [heap]
01344000-01345000 rw-p 00000000 00:00 0                                  [heap]
01345000-01393000 rw-p 00000000 00:00 0                                  [heap]
01393000-01394000 rw-p 00000000 00:00 0                                  [heap]
01394000-013ef000 rw-p 00000000 00:00 0                                  [heap]
7f0a636d0000-7f0a637d0000 rw-s 00000000 00:03 4026532176                 
/proc/xen/privcmd
7f0a637d0000-7f0a638d0000 rw-s 00000000 00:03 4026532176                 
/proc/xen/privcmd
7f0a638d0000-7f0a639fd000 rw-p 00000000 00:00 0 

-Gonglei
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel