Re: [Xen-API] Disaster Recovery of Xen XCP.

[James <jgreenlee@xxxxxxxxxxxx>]

I haven't found a reason for it...But the USB stick would randomly just 
"disappear."  One thing I noticed is that the logging on XCP was out of 
control.  HUGE gzip'd archives several a day.  I'll pull some log data and 
inquire later.

I found this site...

http://www.schirmacher.de/display/INFO/How+to+reattach+a+disk+to+XenServer

Which provided some info that was helpful for CLI commands.

Ultimately, I was able to get the USB stick to boot and write all the Metadata 
to the SR on my RAID controller.  I then loaded a stand-alone SATA drive, 
attached the RAID SR through xsconsole, and then restored the VM's from the 
Metadata that I had written to the RAID SR.

I had to clean up a few things but everything came back.  One thing that was 
somewhat puzzling was the date on my system had moved forward over a week.  The 
hardware clock thought it was Feb8th.  Apparently all of my VM's did as well as 
they all were freaked out about the date and forced a fsck on boot.

Not sure how the date got messed up.  One thing I noticed is that XCP kept 
setting my clock back 5 hours based on my timezone settings.  I don't recall 
that being an issue before, but I went into BIOS and set the hardware clock to 
be UTC, and then set XCP to be EST.  I need to sift through my VM's to make 
sure they're seeing the right time.

I'm very happy that I was able to save everything.

James
________________________________________
From: Denis Cardon [denis.cardon@xxxxxxxxxxxxxxxxxxxxxx]
Sent: Monday, January 28, 2013 5:29 AM
To: James
Cc: Xen-api@xxxxxxxxxxxxx
Subject: Re: [Xen-API] Disaster Recovery of Xen XCP.

Hi James,

> Over the past couple of weeks I've installed XCP onto a USB stick and have 
> built up my virtual data-center.  Now it would appear that the USB stick is 
> failing and I would like to recover rather than rebuild.

USB stick have very bad wear leveling, and you should avoid using them
if they is not mounted read only (you can take a look at the debian live
project to have system booting from usb mounted read only). Actually I
toasted a few usb stick with basic io write pattern.

I used to have a few XCP servers on usb stick/CF cards, and I switched
all of them no SSD because of disastreous wear leveling. Standard XCP
servers I/O pattern is not very intensive, and USB stick should
theoritically be ok, however in case of problem you can get intensive
logging and your USB stick will be toasted in no time.

> The good:
> This first showed up last night and I was able to shut down my VMs (at least 
> partially) and power cycle the server to get XCP back up and running.
> Also good (I think) all of the VMs and their related storage is hosted from a 
> hardware RAID controller.
>
> Whether I install another USB stick or use a normal hard drive, the issue is 
> the same:
>
> How do I get my VM's recognized on a new install of XCP?

XCP metadata (which define VMs setup, network config and corresponding
virtual disk) are written in a db on the XCP partition (which you lost
with your toasted USB stick). You can setup a daily backup using
xsconsole on your storage SR for disaster recovery. Alternatively, you
can do a xe pool-dump-database through cronjob to have your metadata on
your backup server.

> If I recall correctly, when I created the RAID array as a storage repository 
> for XCP, it warned me that all contents would be lost.  However I don't think 
> I need to "create" the SR...I just need to "import" the SR and then "import" 
> the VM's.
>
> With that in mind...Surely there's an "export" option somewhere in XCP??

Your new XCP install has no clue of what is inside your SR. I don't
remember the exact commands to introduce this existing SR to your new
server, but it should be something like
http://support.citrix.com/article/CTX121896 . Do backup first if you are
not familiar with all that stuff.

Cheers,

Denis

>
> Thanks,
>
> James
> _______________________________________________
> Xen-api mailing list
> Xen-api@xxxxxxxxxxxxx
> http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api
>


--
Denis Cardon
Tranquil IT Systems
Les Espaces Jules Verne, bâtiment A
12 avenue Jules Verne
44230 Saint Sébastien sur Loire
tel : +33 (0) 2.40.97.57.55
http://www.tranquil-it-systems.fr


_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api