Re: [Xen-users] Large server, Xen limitations

[Steve Thompson <smt@xxxxxxxxxxxx>]

On Thu, 1 Jan 2009, morten@xxxxxxxxxxxxxx wrote:

> we're contemplating getting a large new server, where we will run a 
> number of virtual servers.  Are there any things we need to keep in mind 
> in that case?  Are there limitations on what a Xen system can manage?
> We're talking about a 4 x Quad core CPU server with 64 GBs of RAM and a
> couple of terabytes of RAIDed SATA storage.
I have a 2 x quad core Dell PE2900 server with 24 GB memory and a couple 
of TB of RAID-1 SATA disk running CentOS 5.2 x86_64 and xen 3.0.3, and 
currently have 33 file-based guests running on it (mostly 32-bit and 
64-bit linux, some Windows). All guests have 1 vcpu and 512 MB memory. 
This setup has been running solidly for about 6 months. Some things I have 
noticed:
- HVM guests were a lot slower than PV guests, and there is a _lot_ of 
qemu-dm overhead on Dom0. In particular, 32-bit HVM guests were much 
slower than 64-bit HVM guests. Avoid HVM as much as possible, and if you 
can't (Windows), use the Xen PV drivers. My workload consists mostly of a 
software development environment, so I run a lot of make's (on Windows, 
under cygwin, sources and objects on Samba shares). I found that the Xen 
PV drivers on Windows improved performance by over three times (reduced 
one compilation from 6 hours to 100 minutes); Windows compile performance 
with cygwin is now about 80% of native speed.
- I converted most of my Linux guests to diskless guests with an NFS root, 
with the Dom0 as NFS server. Not only do the formerly-HVM guests run much 
faster, but they also run faster than file-based PV guests, and now 32-bit 
guests are a little faster than 64-bit guests (less stuff to read from 
NFS, I presume). The qemu-dm overhead on Dom0 is now essentially zero.
- With a large number of guests, you have to be more careful with their 
start/stop/start order to avoid the 'out of memory' error.
- Set xen.independent_wallclock on Linux guests if possible (Xen kernel), 
otherwise it will be next to impossible to keep the clocks in sync.
Steve

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users