[Xen-devel] Copy on Write Filesystem Status

[stevegt@xxxxxxxxxxxxx]

On Tue, Nov 11, 2003 at 11:55:50PM +0000, Ian Pratt wrote:
> We're developing a copy-on-write file system [for virtual disks...]

How is this going?  If I can get this I can use Xen to replace User Mode
Linux...

Some related things that would help with that aim:

(1) Implement lazy allocation (as in the way a sparse file works) for
    vbd blocks.  (Or is this already implemented and I haven't found it in
    the list traffic?)  This would allow, say, 100 vbd's, 4Gb each, to be
    allocated on a 100Gb partition, as long as the average filesystem
    utilization in those vbd's stayed below 25%.  The extra redirection
    would incur a slight performance penalty, but there is a balancing
    economic driver -- see below.

(2) Also implement lazy allocation for COW backing store, unless that
    would already be taken care of by (1).

Example scenario -- an ISP sells Xen guests.  Each guest is sold as a
machine with "4Gb of disk space".  The Xen host hardware has 100Gb of
free physical disk.  The average guest filesystem is only about 25%
full, so the ISP can actually "overbook", i.e. provision 100 guests (100
vbd's, each with their own 4Gb filesystem) on this host.  

Lazy allocation and/or COW would make this possible.  A combination of
both would allow the ISP to efficiently keep a backing store for each of
several major Linux distros, giving the customers a choice.

This is something I can do now with User Mode Linux.  It works like
this:

# On the parent host, create an empty sparse file that will store our
# filesystem, 4GB in size:
#
$ dd if=/dev/zero of=ubd0 count=0 bs=1M seek=4096
0+0 records in
0+0 records out

# Note that this consumed zero data blocks so far:
#
$ ls -ls ubd0
0 -rw-r--r--    1 stevegt  stevegt  4294967296 Jan 15 21:01 ubd0

# Format it:
#
$ mkfs -j ubd0

# The superblock and copies etc. only used a few blocks:
#
$ ls -ls ubd0
99040 -rw-r--r--    1 stevegt  stevegt  4294967296 Jan 15 21:06 ubd0

# Mount it via loopback, copy in filesystem contents, etc.  At this
# point the filesystem is now usable as a UML root filesystem, and the
# parent will continue to allocate blocks as they are dirtied.  It's
# still pretty small:
#
$ ls -ls ubd0
202128 -rw-r--r--    1 stevegt  stevegt  4294967296 Jan 15 21:10 ubd0

# Start the UML:
#
$ linux ubd0=ubd0 ...

# Or better yet, start UML, automatically creating a new copy on write
# filesystem in a file named 'cow0', using the above ubd0 as read-only
# backing store:
#
$ chmod 444 ubd0 
$ linux ubd0=ubd0,cow0

# Later, check the sizes of backing store and COW -- note the huge
# amount of disk space we're saving by not allocating those blocks:
#
$ ls -ls ubd0 cow0
202128 -rw-r--r--    1 stevegt  stevegt  4294967296 Jan 15 21:10 ubd0
 14256 -rw-r--r--    1 stevegt  stevegt  4294967296 Jan 15 21:30 cow0


In other words, this all works right now in User Mode Linux.  These
semantics work well for hosting lots of independent guests who each
want their own root filesystem.  But the performance penalty of UML is
high enough that I'd like to be able to do this under Xen instead.  

I'm a strong Perl/Python guy (with rusty C and kernel internals), so am
likely to want to work on the provisioning tools side of making this
happen for Xen if you folks implement the lower layers.

Steve
-- 
Stephen G. Traugott  (KG6HDQ)
UNIX/Linux Infrastructure Architect, TerraLuna LLC
stevegt@xxxxxxxxxxxxx 
http://www.stevegt.com -- http://Infrastructures.Org 



-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/xen-devel