RE: [EXTERNAL] [Xen-devel] XEN Qdisk Ceph rbd support broken?

[Paul Durrant <xadimgnik@xxxxxxxxx>]

> -----Original Message-----
> From: Brian Marcotte <marcotte@xxxxxxxxx>
> Sent: 15 July 2020 20:17
> To: Durrant, Paul <pdurrant@xxxxxxxxxxxx>
> Cc: Jules <jules@xxxxxxxxx>; xen-devel@xxxxxxxxxxxxxxxxxxxx; 
> oleksandr_grytsov@xxxxxxxx; wl@xxxxxxx
> Subject: RE: [EXTERNAL] [Xen-devel] XEN Qdisk Ceph rbd support broken?
> 
> CAUTION: This email originated from outside of the organization. Do not click 
> links or open
> attachments unless you can confirm the sender and know the content is safe.
> 
> 
> 
> This issue with Xen 4.13 and Ceph/RBD was last discussed back in February.
> 
> > Remote network Ceph image works fine with Xen 4.12.x ...
> >
> > In Xen 4.13.0 which I have tested recently it blames with the error
> > message "no such file or directory" as it would try accessing the image
> > over filesystem instead of remote network image.
> > ---
> >
> > I doubt the issue is in xl/libxl; sounds more likely to be in QEMU. The
> > PV block backend infrastructure in QEMU was changed between the 4.12
> > and 4.13 releases. Have you tried using an older QEMU with 4.13?
> 
> I'm also encountering the problem:
> 
>     failed to create drive: Could not open 'rbd:rbd/machine.disk0': No such 
> file or directory
> 
> Xenstore has "params" like this:
> 
>     aio:rbd:rbd/machine.disk0
> 
> If I set it to "rbd:rbd/machine.disk0", I get a different message:
> 
>   failed to create drive: Parameter 'pool' is missing
> 
> Using upstream QEMU versions 2 or 3 works fine.
> 
> The interesting thing is that access by the virtual BIOS works fine. So,
> for a PVHVM domain, GRUB loads which loads a kernel, but the kernel can't
> access the disks.

Brian,

  That's not entirely surprising as the BIOS is likely to be using an emulated 
device rather than a PV interface. Your issue stems
from the auto-creation code in xen-block:

https://git.qemu.org/?p=qemu.git;a=blob;f=hw/block/xen-block.c;hb=HEAD#l723

  The "aio:rbd:rbd/machine.disk0" string is generated by libxl and does look a 
little odd and will fool the parser there, but the
error you see after modifying the string appears to be because QEMU's QMP block 
device instantiation code is objecting to a missing
parameter. Older QEMUs circumvented that code which is almost certainly why you 
don't see the issue with versions 2 or 3.

  Paul





> 
> --
> - Brian