Xen Summit 2023: Design Session Notes: SMAPIv3

[Mark Syms <mark.syms@xxxxxxxxx>]

Slides used for scene setting -
https://github.com/xapi-project/xen-api/discussions/5080.

Example code in Github -
https://github.com/xapi-project/xen-api/tree/master/ocaml/xapi-storage/python/examples

Notes taken in the session
------------------------------------

Since scene setting slides were written SMAPIv3 has actually been
converted to python3 (pending merges etc)

How would you like SMAPIv3 evolve in future?

 - Gaps compared to other SR types:

     - Storage Migration (no implementation on either side, XenServer
keen on implementing this)

     - Change block tracking (start of an API definition exists for this)

Potential future thoughts to move the traditional LVM and FC SMAPIv1
SRs to SMAPIv3 SRs to resolve technical debt, would need storage
migration support.

Could other toolstacks (e.g. xl, libvirt) use SMAPIv3?

 - They're just python scripts, so nothing to prevent it, but the
current XenServer team has no particular experience with other
toolstacks - happy to assist, but would need someone else to lead.

Documentation - does it need improvement / is it clear, feedback invited.

Code for a plugin is very simple (e.g. example is 130 lines, a lot of
which is boilerplate)

Cycle for a disk attached to a VM is:

  open

  attach

  activate

  deactivate

  detach

  close

open/close are start and end of use of volume overall (VM activation lifecycle)

During live migration you may do e.g. migrating from H1 to H2:

H2: attach

H1: deactivate

H2: activate

H1: detach

The idea being activate/deactivate is in the critical region where the
VM is paused so should be as quick as possible, longer tasks can go in
attach/detach.

Multiple volume URIs can be returned to give multiple ways for a
consumer to access the volume

Q: Is there a way to expose SMAPIv3 SRs in a way for backup, e.g. for
incremental backup etc?

The change block tracking (CBT) functionality is relevant here, to
allow for a way of identifying what needs backing up etc.

- Could we use nbd as a universal way of exposing things?

Endpoints are implementation specific, so would need the SRs to
implement it, or something else to access another method and re-expose
it.

Storage motion support may also give more possibilities here as it'll
be a general API update, would be good to include other required
capabilities as part of this work, e.g. any new `implementation`s
required.

(Hoping to avoid breaking changes in API)

Vates have done an implementation of a zfs SR using SMAPIv3
(https://github.com/xcp-ng/xcp-ng-xapi-storage/pull/11), would be good
to review and see if it identifies any required API improvements etc.

Existing implementations don't implement `copy`, which would ideally
support an offloaded copy operation where the backend storage device /
mechanism can do the work. Needs toolstack work to plumb through and
handle cases where different SRs are in use that can't offload copy
between them etc.

Thanks to everyone who attended and contributed.

Mark.