Re: [Xen-devel] [RFC V6] Libxl Domain Snapshot API Design

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Thu, 2014-09-11 at 02:30 -0600, Chun Yan Liu wrote:

Sorry, I've just realised I lost track of this thread.
> 
> >>> On 9/10/2014 at 08:17 PM, in message
> <1410351477.8217.375.camel@xxxxxxxxxxxxxxxxxxxxxx>, Ian Campbell
> <Ian.Campbell@xxxxxxxxxx> wrote: 
> > On Tue, 2014-09-09 at 23:53 -0600, Chun Yan Liu wrote: 
> > > > So I don't think there is any need (or even desire) for libxl to  
> > > > replicate any of that functionality. It should concern itself with  
> > > > taking a snapshot when asked and then forget all about it.  
> > > >   
> > > > I think things work the same with e.g. qemu and other libvirt backends. 
> > > >  
> > > > A qemu process only exists to track an actual running domain, all of 
> > > > the  
> > > > other state including the configuration when the domain is not running  
> > > > etc is all handled at the libvirt layer. When you take a snapshot of a  
> > > > qemu backed domain then after that snapshot has happened the qemu  
> > > > process knows no more about it. So in this way of thinking qemu and  
> > > > libxl are functionally equivalent bits i.e. things which are used to  
> > > > instantiate actual running domains and perform operations on them.  
> > > >   
> > > > Does that make sense?  
> > >  
> > > Totally agree with you. I've mixed something in xl and libxl. It's 
> > > correct 
> > > that as libxl it should only do what application asks it to do and then 
> > > forget everything. Maintaining state and other information is not libxl's 
> > > work, it's the work of high level application. Libvirt does have 
> > > datamodel 
> > > to keep tracking snapshot info, it doesn't need libxl to do that. xl can 
> > > maintain snapshot info in application level. 
> >  
> > Great! I'm glad we are on the same page. 
> >  
> > > The only problem is snapshot-delete: 
> > > to delete disk snapshot, it has to call qmp command, libvirt libxl driver 
> > > cannot do that itself (except adding qmp related helper functions, quite 
> > > a duplicate with libxl_qmp.c I think). 
> >  
> > What is this qmp command, what does it do and what arguments does it 
> > require?
> 
> The qmp command is 'blockdev-snapshot-delete-internal-sync'.
> Purpose of this qmp command is to delete an internal snapshot within the
> disk. E.g. for qcow2 format, a internal disk snapshot will store snapshot
> related info within the disk. To delete this snapshot, one has to communicate
> with qemu (through qmp command), let qcow2 driver to handle that.
> 
>  { "execute": "blockdev-snapshot-delete-internal-sync",
>                        "arguments": { "device": "ide-hd0",
>                                                   "name": "snapshot0" }
> }

Thanks, I can see why this would be necessary if a domain was actively
using the snapshot chain.

Presumably if there is no active domain the calling application will
take some action directly to delete the snapshot (such as using qemu-img
perhaps?)

I suppose there is an added wrinkle which is that there might in fact be
multiple active domains using different snapshots within the chain
contained in the same qcow2 file? Or if qcow2 outlaws this we should
either consider a more general container or explicitly decide that we
don't support this sort of thing. TBH I'm not sure how multiple qemu
processes accessing the same qcow2 file could work, at least for
internal snapshots, and external ones I suppose are pretty trivial to
deal with.

So at the libxl layer we would want a function which takes a domid and a
libxl_device_disk and some sort of "handle" to the snapshot to remove.

I say "handle" rather than "string containing the name" since different
containers and/or backend types might have different manifestations of
such things, so it would have to be somewhat opaque from the libxl API
point of view.

I *think* (but I'm not sure) that the handle will be a function of the
disk format only and not the disk backend. My reasoning is that libxl
will need to know for each backend type how to speak to it (e.g. qdisk
== qmp) to delete a snapshot and will forward the handle on in the
request it makes to the backend, which will necessarily understand the
how snapshots are referenced for that disk container. Does that make
sense?

Given that it seems like the libxl function should take some sort of
parameter which depends on the disk format, we could just make that a
void*, but perhaps something more structured like:

        libxl_snapshot_handle = Struct("snapshot_handle", [
            ("u", KeyedUnion(None, libxl_disk_format, "format",
                [("qcow2", Struct(None, [("name", string)]),
                ])
            ])
        
So for qcow you would set
    handle.format = LIBXL_DISK_FORMAT_QCOW2
    handle.qcow2.name = "snapshot0"
and pass the result to the function.

handle.format would duplicate disk.format, and it would be an error for
them to be mismatched. Maybe we could just use an IDL Union instead of a
KeyedUnion to avoid that. KeyedUnion is nice because it enforces a
relationship between the related enum and union, but maybe in this case
it is uglier than it is helpful.

> > Perhaps it's something like "the snapshot chain of $disk may have 
> > changed, go and figure out what to do"? Or is it more complicated than 
> > that? 
> >  
> > > If we supply libxl_domain_snapshot_delete API, to handle snapshot-chain 
> > > (parent, children, sibling), it doesn't work without info about all  
> > snapshots. 
> > > Or, if we supply libxl_disk_snapshot_delete API, that's also weird with  
> > this 
> > > only-one disk snapshot API. 
> >  
> > So, the issue here is deleting a snapshot where there is an actively 
> > running domain using some other snapshot in the chain, is that right? 
> 
> I meant to say:
>         BASE -> SNAPSHOT A -> SNAPSHOT B
> When user want to delete  SNAPSHOT A, if flag indicates deleting
> all children, then SNAPSHOT B should be deleted too; if without this flag,
> then by default all changes to SNAPSHOT A should be merged to
> SNAPSHOT B. In any case, it should know the relationship between
> SNAPSHOT A and SNAPSHOT B.

If you want to delete SNAPSHOT A and all of its children then there
cannot be any active domains using A or its children, so would it be
necessary to go via libxl at all? (as hinted above perhaps such things
are better handled directly in the application).

If you are deleting A only and B is in active use then merging A into B
would be the job of B's backend, via the call/mechanism proposed above.
Does that seem reasonable?

> Well, after thinking again, I think maybe there is no problem here:
> libxl supplies libxl_domain_snapshot_delete API, that could help doing disk
> snapshot-delete work. In disk level, 'all changes to SNAPSHOT A should
> be merged to SNAPSHOT B' can be ensured by qemu automatically, libxl
> doesn't need to take care of that.
> Then xl or libvirt can do all other things, they scan the domain snapshot
> information to decide which snapshot is related and should be deleted,
> and refresh domain snapshot chain relationship after the work.

IOW it would be the applications responsibility to iterate over the
chain and delete individual things? That sounds reasonable and makes it
easier for us to ignore the possiblity of multiple VMs at the libxl API
layer.

> > Specifically we aren't interested in deleting a snapshot which is 
> > actually being used right now? Would that case be handled by destroying 
> > the domain in question first? 
> >  
> > So in the simplest case we might have: 
> >  
> >         BASE -> SNAPSHOT A -> SNAPSHOT B 
> >  
> > And there may be a domain running which is using any of BASE, A or B? Do 
> > we then need to be able to delete either of the other two snapshots? 
> >  
> > If a domain is running using SNAPSHOT A then I presume SNAPSHOT B can be 
> > trivially deleted without needing to communicate with that domain. 
> >  
> > If a domain is running using SNAPSHOT B and we want to delete SNAPSHOT A 
> > then the domain needs to know that it should rescan the chain to 
> > discover that it is has become: 
> >  
> >         BASE -> SNAPSHOT B 
> >  
> > Or is the qemu associated with the domain actually responsible for 
> > making that happen (i.e. folding the contents of SNAPSHOT A into either 
> > BASE or SNAPSHOT B as appropriate at runtime)? Or does that happen 
> > elsewhere?
> 
> If SNAPSHOT B is based on SNAPSHOT A, when A is deleted, changes made
> to A will be merged to B. Qemu can ensure that, at least for a qcow2 disk,
> qcow2 driver can ensure that.

Ack.

> But from Domain level, it is high level application to refresh the domain
> snapshot chain relationship. In libvirt, it is qemu_driver to rerefresh the
> chain relationship and update related qemuDomainSnapshotObj (update
> information of .parent, .sibling, .children, etc.).

Is the "qemu_driver" here a "qemu storage driver" or a "qemu vm
driver" (is there even such a distinction?)

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel