[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] Strange failures of Xen 4.3.1, PVHVM storage VM, iSCSI and Windows+GPLPV VM combination

  • To: xen-users@xxxxxxxxxxxxx
  • From: Kuba <kuba.0000@xxxxx>
  • Date: Fri, 31 Jan 2014 01:08:31 +0100
  • Delivery-date: Fri, 31 Jan 2014 00:09:33 +0000
  • List-id: Xen user discussion <xen-users.lists.xen.org>

W dniu 2014-01-30 23:51, Adam Goryachev pisze:
On 31/01/14 00:50, Kuba wrote:
Dear List,

I am trying to set up a following configuration:
1. very simple Linux-based dom0 (Debian 7.3) with Xen 4.3.1 compiled
from sources,
2. one storage VM (FreeBSD 10, HVM+PV) with SATA controller attached
using VT-d, exporting block devices via iSCSI to other VMs and
physical machines,
3. one Windows 7 SP1 64 VM (HVM+GPLPV) with GPU passthrough (Quadro
4000) installed on a block device exported from the storage VM (target
on the storage VM, initiator on dom0).

Everything works perfectly (including PCI & GPU passthrough) until I
install GPLPV drivers on the Windows VM. After driver installation,
Windows needs to reboot, boots fine, displays a message that PV SCSI
drivers were installed and needs to reboot again, and then cannot
boot. Sometimes it gets stuck at "booting from harddrive" in SeaBIOS,
sometimes BSODs with "unmountable boot volume" message. All of the
following I tried without GPU passthrough to narrow down the problem.

The intriguing part is this:

1. If the storage VM's OS is Linux - it fails with the above symptoms.
2. If the block devices for the storage VM come directly from dom0
(not via pci-passthrough) - it fails.
2. If the storage VM is an HVM without PV drivers (e.g. FreeBSD
9.2-GENERIC) - it all works.
3. If the storage VM's OS is Linux with kernel compiled without Xen
guest support - it works, but is unstable (see below).
4. If the iSCSI target is on a different physical machine - it all works.
5. If the iSCSI target is on dom0 itself - it works.
6. If I attach the AHCI controller to the Windows VM and install
directly on the hard drive - it works.
7. If the block device for Windows VM is a disk, partition, file, LVM
volume or even a ZoL's zvol (and it comes from a dom0 itself, without
iSCSI)- it works.

If I install Windows and the GPLPV drivers on a hard drive attached to
dom0, Windows + GPLPV work perfectly. If I then give the same hard
drive as a block device to the storage VM and re-export it through
iSCSI, Windows usually boots fine, but works unstable. And by unstable
I mean random read/write errors, sometimes programs won't start,
ntdll.dll crashes, and after couple reboots Windows won't boot (just
like mentioned above).

The configurations I would like to achieve makes sense only with PV
drivers on both storage and Windows VM. All of the "components" seem
to work perfectly until all put together, so I am not really sure
where the problem is.

I would be very grateful for any suggestions or ideas that could
possibly help to narrow down the problem. Maybe I am just doing
something wrong (I hope so). Or maybe there is a bug that shows itself
only in such a particular configuration (hope not)?

IMHO, it sounds like a resource issue... the domU providing the iSCSI,
plus the dom0 plus the domU (windows VM) are all asking for CPU, IRQ's,
etc, and someone isn't getting enough in time. It doesn't really help,
but we use a physical iSCSI server, the dom0 then connects to the iSCSI
and provides the disks to the VM's. Maybe look at assigning specific
exclusive CPU's to each dom0 and domU's, and see if you can still
reproduce the issue. Also, make absolutely sure that you don't have two
VM's accessing the same iSCSI.


Dear Adam,

Thank you for your reply. I will try assigning specific cores to the VMs this weekend. I will also try to run all this on another machine and try using something different then Windows for the second VM (why didn't I think about it earlier?). When iSCSI target is on another physical machine, everything works like a charm, but the whole point is to make it all work on a single machine (everything faster than 1 Gbps is way beyond my reach, while iperf reports ~22 Gbps between dom0 and storage VM...)

Once again thank you for your suggestions, I will report back with more results. In the meantime please clarify one thing for me - is there something inherently wrong with what I'm trying to do?

Best regards,

Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.