Re: [Xen-devel] Xen Management API Draft, version 0.4

[Harry Butterworth <harry@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx>]

On the first reading, the bit of the document that I was most interested
in was the introduction as I was trying to establish the scope of the
API.  But it's just a one liner.  I think there needs to be considerable
work up front to set the scene and describe the background, context and
scope for the API.

I'd like to see a survey of existing solutions to the problem and a
discussion of the pros and cons of each.

One of the questions to answer up front is whether the API is for
managing a xen instance on a single node or a cluster of xen instances
on multiple nodes.  A picture of the whole system under consideration
and where the API fits into that would help.

It's just as important to describe an example for the kind of
environment that's expected to surround the API as the API and the
implementation itself.

Some of the text you have written below to introduce the document could
go into the introduction of the document itself.

The other part of the document that I found useful was the picture of
the domain lifecycle because it's a high level description of one part
of the model for supporting the use-cases that I'm interested in.  And I
can evaluate it against the use-cases to see if the system is going to
be useful.

The rest of the document is a description of the implementation.  This
part of the document is essential but really ought to fall out trivially
once you understand the scope and the use cases and the model for
supporting the uses cases.

So I think it would be good if the document was structured with an
introduction.  A bit of background on existing stuff.  A set of use
cases that the API is going to address.  A set of use cases that are for
future consideration.  A set of use cases that are not going to be
addressed.  A description of the scope and context of the API and how it
is intended to fit into a system that can meet the use cases that are
going to be addressed.  A description of the system model that supports
the use cases which would include diagrams like the domain lifecycle
diagram and then a description of the API used to drive the system
model.

Agreeing on the set of use cases and the scope of the API first will
make the model and implementation much easier to define.

With use-cases I think it's best if as many people as possible pool all
their ideas and then the complete set is filtered.  Some will be
duplicates, out of scope or too far out.  Some of the use cases will be
high level but require some support from the low level API.  Here's as
many as I can think of right now:

User buys new box wants to install xen and a single service.
User has existing box already running a service and not xen.  Wants to
create a new service on same box using xen for isolation.
User has existing boxes already running services and wants to work out
the cost of consolidation onto (how many?) fewer boxes using xen.
User has existing boxes already running services wants to actually
perform consolidation onto fewer boxes using xen.
The user wants to know how many xen boxes will be required.
User has existing xen infrastructure and wants to deploy a new service.
User has existing xen infrastructure but load is skewed.  Wants to
rebalance load across boxes.
User has existing virtual infrastructure from another provider and wants
to incrementally replace it by deploying xen.
User has existing xen infrastructure but wants to remove xen and migrate
back to native operation.
User has existing xen infrastructure and wants to migrate to virtual
infrastructure from another provider.
User has existing xen infrastructure and wants to safely evaluate the
effect of a xen upgrade.
User has existing xen infrastructure and wants to safely evaluate the
effect of upgrades to a set of interrelated services.
User has existing xen infrastructure and the xen hypervisor crashes. The
user wants a) the system operational again ASAP b) to know the root
cause c) a fix and d) to test the fix safely before deploying it in
production.  The xen developers want first failure data capture
debugging information.
User has existing xen infrastructure and a VM crashes. The user wants a)
the system operational again ASAP b) to know the root cause c) a fix and
d) to test the fix safely before deploying it in production.  Someone
wants first failure data capture information to debug the VM.
User has existing xen infrastructure and the system is performing
poorly.  The user wants to debug the problem and fix it.
The user wants to check up on the xen infrastructure to satisfy
themselves that it is all running OK.
User has existing xen infrastructure and a box needs to be powered off
for servicing.  The user wants to service the machine without disrupting
the services running on it.
User has existing xen infrastructure and a physical machine fails
completely.  The user wants to reestablish system operation ASAP.
User has existing xen infrastructure and a data centre is taken out. The
user wants to reestablish system operation at a new site ASAP.
User has existing xen infrastructure in multiple sites and wants a
disaster recovery solution that will get them up again ASAP if any site
is lost.
There is confidential information in a virtual machine.  User wants to
do a secure delete.
User has a virtual machine and wants to package it for distribution to
other users.
User is provided with a virtual machine package and wants to deploy it.
User wants to deploy a very large number of generally idle virtual
machines and dynamically balance resources such that the machines that
are active have enough resources to make progress.
User wants to deploy a virtual machine for the duration of a job on any
physical machine from a pool.
Multiple users want to do some of these use cases using the same
physical hardware at the same time with some protection from each other.
VM service provider wants to track resource usage of a VM for accounting
purposes.
Client of VM service provider wants to dynamically adjust resource
allocation according to workload to minimise cost.
User wants to maintain an archive of VMs so as to be able to recreate
historical configurations.
User wants a backup of a consistent set of virtual machines.
User's security is breached and primary systems are compromised.
Perhaps compromise is replicated by disaster recovery solution to
secondary site.  User wants to roll back to before the attack and then
reapply transaction logs.  Perhaps there were some VM configuration
changes or migrations during the period of interest.
User wants to perform some of these use-cases directly at the physical
machine.
User wants to perform some of these uses cases from a remote machine.
User wants to write scripts to automate some of these uses cases.
User wants to integrate the xen infrastructure with existing
infrastructure.

Harry.

On Mon, 2006-07-03 at 16:53 +0100, Ewan Mellor wrote:
> Attached is version 0.4 of the Xen Management API Draft.  This document will
> eventually define a fixed XML-RPC protocol for remote and local management of
> Xen-based systems.  I would like to invite you to study, give feedback on, and
> collaborate around this document, so that we will soon have for the first time
> an open source toolstack that exports a supported management API for Xen.
> 
> This document presents our ideas in terms of a data model, with an implied
> binding to XML-RPC calls. These XML-RPC calls can then be used (over one of a
> number of transports) to manage a Xen-based system.
> 
> The intention is to standardise both the data model and XML-RPC calls (one
> implying the other) and then the Xen project will guarantee that that wire
> protocol would be supported for the long term.
> 
> Behind that interface, we would then be free to improve Xend, and at the same
> time we give a solid foundation for third-party tools (in particular
> libvirt-based applications), GUIs, and so on. The API becomes effectively part
> of the "guarantees" of the Xen project.
> 
> 
> The XML-RPC calls will be the fixed standard, but we're also going to need
> bindings to that XML-RPC for Python (for xm), for C (for libvirt), and
> possibly for other languages too. These will be a thin translation from the
> host language's values and types onto the XML-RPC, so that they can be kept
> stable so that third-party applications can rely upon them in the long
> term. Cleverer facilities (such as libvirt) can then be built on top. These
> bindings will be open-source, and committed as libraries to the Xen trees for
> all to use.
> 
> 
> The latest version of the API definition, version 0.4, is attached.  This is
> an Open Preview Release, and we welcome any discussion about it and the issues
> surrounding it. We would love to see use cases for this API, so that we can
> check that it all makes sense, and welcome any feedback you might have.
> 
> Please join the xen-api mailing list if you are interested in this project.  I
> will collate all the feedback from that list and push out new versions of the
> document as and when.
> 
> In particular, please pay attention to Section 1.6, the to-do list.  All of
> these changes will be rolled into the document in the next couple of days.
> 
> This document, and its source, are available at the Xen wiki page XenApi:
> 
> http://wiki.xensource.com/xenwiki/XenApi
> 
> 
> This API is not yet fixed! The definition is still in flux, so please do not
> expect it to be stable.  Please do comment though!
> 
> 
> Many thanks,
> 
> Ewan.
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel