Re: [EXTERNAL] [Xen-devel] XEN Qdisk Ceph rbd support broken?

[Anthony PERARD <anthony.perard@xxxxxxxxxx>]

On Fri, Jul 17, 2020 at 08:48:01AM +0100, Paul Durrant wrote:
> > -----Original Message-----
> > From: Brian Marcotte <marcotte@xxxxxxxxx>
> > Sent: 16 July 2020 21:24
> > To: Paul Durrant <xadimgnik@xxxxxxxxx>
> > Cc: paul@xxxxxxx; 'Jules' <jules@xxxxxxxxx>; xen-devel@xxxxxxxxxxxxxxxxxxxx;
> > oleksandr_grytsov@xxxxxxxx; wl@xxxxxxx
> > Subject: Re: [EXTERNAL] [Xen-devel] XEN Qdisk Ceph rbd support broken?
> > 
> > > Your issue stems from the auto-creation code in xen-block:
> > >
> > > The "aio:rbd:rbd/machine.disk0" string is generated by libxl and does
> > > look a little odd and will fool the parser there, but the error you see
> > > after modifying the string appears to be because QEMU's QMP block
> > > device instantiation code is objecting to a missing parameter. Older
> > > QEMUs circumvented that code which is almost certainly why you don't
> > > see the issue with versions 2 or 3.
> > 
> > Xen 4.13 and 4.14 includes QEMU 4 and 5. They don't work with Ceph/RBD.
> > 
> > Are you saying that xl/libxl is doing the right thing and the problem
> > needs to be fixed in QEMU?
> 
> Unfortunately, from what you describe, it sounds like there is a problem in 
> both. To get something going, you could bring a domain
> up paused and then try manually adding your rbd device using the QMP shell.
> 
> It would be useful if a toolstack maintainer could take a look at this issue 
> in the near future.
> 

Hi,

I did start working on a solution some time ago and produced a patch for
QEMU (attached) which would allow QEMU to parse the aio:rdb:... string
from xenstore.

But I ran into an other issue when I tried with nbd (Network Block
Device), QEMU would connect twice to the NBD server and the server I had
didn't like it. Maybe Ceph would allow two connections to the same disk?

The two connections issue is less likely to happen on older QEMU because
it would delay the second connection until the guest connect to the PV
backend, so after the emulated disk has been unplugged (and thus the
first connection disconnected).

Anyway, it would be better to upgrade libxl to be able to create a QEMU
PV backend via QMP or qemu's command line rather than via xenstore, but
I don't think I have time to work on it just yet. But I feel like we are
going to have the same issue that QEMU will try to connect twice to the
Ceph server where this wasn't likely to happen before.

Jules, Brian, could you maybe try the attach QEMU patch and see if that
works?

Cheers,

-- 
Anthony PERARD

Attachment: 0001-xen-block-Fix-parsing-of-legacy-options.patch
Description: Text document