[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] High Number of VMs

On 09/09/2011 03:07 PM, Ciro Iriarte wrote:
Hi, I'm curious about how you guys deal with big virtualization
installations. To this date we only dealt with a small number of VM's
(~10)on not too big hardware (2xquad xeons+16GB ram). As I'm the
"storage guy" I find it quite convenient to present to the dom0s one
LUN per VM that makes live migration possible but without the cluster
file system or cLVM complexity. The thing is that Linux has a 255 SCSI
device limit apparently (255/2= ~128 with multipath) and that won't
scale in big installations (300 VMs for example).

Any experiences on this scale?

I run a couple hundred VM's across a handful of blades. I recommend going to fewer, larger LUNs, carving them up with LVM, and handing out LV's to your VM's. You don't actually need cLVM to do this! All the cluster infra does (for its nasty administrative overhead) is keep the LVM metadata (not your actual data) consistent through cluster-wide locks. You can manage yourself by, for example, making changes on one node and refreshing the other nodes with things like 'vgscan -ay'. I typically allocate LUNs 500GB at a time. You can invent means of keeping things consistent that work for your environment, just test them first.

If you really want data security -- preventing one node from hosing data that's accessible from another node -- you'll have to go with a cluster filesystem. That still doesn't help keep your multipathing and LVM configs consistent though, so I think you're better off just skipping that step.


John Madden / Sr UNIX Systems Engineer
Office of Technology / Ivy Tech Community College of Indiana

    Free Software is a matter of liberty, not price. To understand
    the concept, you should think of Free as in 'free speech,' not
    as in 'free beer.' -- Richard Stallman

Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.