Re: [Xen-users] Re: LVM on DRBD -OR- DRBD on LVM?

[xen-users@xxxxxxxxxxxxx]

Simon,

I think Matthew was making a recommendation based upon how you have your 
domU's setup  ... i.e. are your domU's living on LVM LV's or disk files 
in one big LV, multiple PV's (i.e. 1 for the boot/root of the dom0 and 
one (or more) for the domU's), etc.
We need to know how you have your primary server configured ... I'm not 
sure you can reliably mirror your entire root PV since the kernel needs 
to be up and have the DRBD driver in place to mount the device the 
kernel will be booting on (chicken and egg) ... it may be possible, but 
I've not thought about it long enough to decide.
What we've done is set up a pair of physically identical servers with a 
GigE crossover cable for the DRBD devices to sync across (and a 100M Eth 
for the public interface), and each guest has a set of LV's dedicated to 
it.  Each LV is created identically on both servers and are bound under 
a DRBD device.  The guest mounts the DRBD device, so all writes to the 
domU's "disk" gets replicated via DRBD to both servers.
Now, that said, that's not our entire story, as we have some more 
mirroring stuff under the LV's, but you get the point for where we have 
DRBD in place.
Each guest generally has 3 LV's associated with it, and only 2 of those 
replicated via DRBD:
   xen0:GuestA-root ->   drbd    ->  xen1:GuestA-root      (2gb)
   xen0:GuestA-swap <not replicated> xen1:GuestA-swap      (1gb)
   xen0:GuestA-data ->   drbd    ->  xen1:GuestA-data      (anywhere 
from 5gb to 100gb)
I felt that replicating swap was just a waste of resources as we can't 
do hot migration anyway, so upon reboot, anything in swap is gone anyhow.
Currently, we are running with 14 DRBD devices, but none of the guests 
are in production yet, so I don't have good load and performance data 
yet ... I'm still hunting for good performance monitoring tools (got one 
suggestion for ganglia, and I also found bandwidthd as well as the usual 
friends like cacti+snmpd), and I've been watching the privage GigE as 
well as /proc/drbd.
I'll let the list know how performance seems to be after it's all up and 
running!  :)
Oh, and currently, we are running all guests on the primary server, but 
we plan to distribute the guests across both servers after all is running.
Hope this helps,
Sam


Simon wrote:
> Hi Matthew,
>
> On 6/20/06, Matthew Palmer <mpalmer@xxxxxxxxxxx> wrote:
> > On Tue, Jun 20, 2006 at 02:49:48PM +1200, Simon wrote:
> > > In creating a "backup" dom0 server using drbd and using lvm partitions
> > > for the domU data, whats the best way to implement the drbd part?
> > >
> > > Should i have drbd mirroring the lvm partitions them selves or the
> > > data within them?
> >
> > Depends.  Primarily, on whether you've going to be sharing the entire PV
> > between both machines, and never sharing some LVs with any other 
> > host.  If
> > you have a separate VG (on it's own PV) for your domUs, and are just
> > mirroring your domUs for availability, then mirroring the PVs is fine.
> > Otherwise (if you've got the dom0 disk on the same VG) then you'll 
> > have to
> > mirror the LVs individually, and incur the overhead of lots-o-drbd.
> Im only concerned with two individual servers here:
>
> dom0(1) will hold all the domUs
> dom0(2) will only be used in a DR case when dom0(1) is down
>
> So reading your above comment.... LVM on DRBD?
>
> Simon
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users