Re: [Xen-users] Strange failures of Xen 4.3.1, PVHVM storage VM, iSCSI and Windows+GPLPV VM combination

[Kuba <kuba.0000@xxxxx>]

W dniu 2014-01-30 23:51, Adam Goryachev pisze:
> On 31/01/14 00:50, Kuba wrote:
> > Dear List,
> >
> > I am trying to set up a following configuration:
> > 1. very simple Linux-based dom0 (Debian 7.3) with Xen 4.3.1 compiled
> > from sources,
> > 2. one storage VM (FreeBSD 10, HVM+PV) with SATA controller attached
> > using VT-d, exporting block devices via iSCSI to other VMs and
> > physical machines,
> > 3. one Windows 7 SP1 64 VM (HVM+GPLPV) with GPU passthrough (Quadro
> > 4000) installed on a block device exported from the storage VM (target
> > on the storage VM, initiator on dom0).
> >
> > Everything works perfectly (including PCI & GPU passthrough) until I
> > install GPLPV drivers on the Windows VM. After driver installation,
> > Windows needs to reboot, boots fine, displays a message that PV SCSI
> > drivers were installed and needs to reboot again, and then cannot
> > boot. Sometimes it gets stuck at "booting from harddrive" in SeaBIOS,
> > sometimes BSODs with "unmountable boot volume" message. All of the
> > following I tried without GPU passthrough to narrow down the problem.
> >
> > The intriguing part is this:
> >
> > 1. If the storage VM's OS is Linux - it fails with the above symptoms.
> > 2. If the block devices for the storage VM come directly from dom0
> > (not via pci-passthrough) - it fails.
> > 2. If the storage VM is an HVM without PV drivers (e.g. FreeBSD
> > 9.2-GENERIC) - it all works.
> > 3. If the storage VM's OS is Linux with kernel compiled without Xen
> > guest support - it works, but is unstable (see below).
> > 4. If the iSCSI target is on a different physical machine - it all works.
> > 5. If the iSCSI target is on dom0 itself - it works.
> > 6. If I attach the AHCI controller to the Windows VM and install
> > directly on the hard drive - it works.
> > 7. If the block device for Windows VM is a disk, partition, file, LVM
> > volume or even a ZoL's zvol (and it comes from a dom0 itself, without
> > iSCSI)- it works.
> >
> > If I install Windows and the GPLPV drivers on a hard drive attached to
> > dom0, Windows + GPLPV work perfectly. If I then give the same hard
> > drive as a block device to the storage VM and re-export it through
> > iSCSI, Windows usually boots fine, but works unstable. And by unstable
> > I mean random read/write errors, sometimes programs won't start,
> > ntdll.dll crashes, and after couple reboots Windows won't boot (just
> > like mentioned above).
> >
> > The configurations I would like to achieve makes sense only with PV
> > drivers on both storage and Windows VM. All of the "components" seem
> > to work perfectly until all put together, so I am not really sure
> > where the problem is.
> >
> > I would be very grateful for any suggestions or ideas that could
> > possibly help to narrow down the problem. Maybe I am just doing
> > something wrong (I hope so). Or maybe there is a bug that shows itself
> > only in such a particular configuration (hope not)?
> IMHO, it sounds like a resource issue... the domU providing the iSCSI,
> plus the dom0 plus the domU (windows VM) are all asking for CPU, IRQ's,
> etc, and someone isn't getting enough in time. It doesn't really help,
> but we use a physical iSCSI server, the dom0 then connects to the iSCSI
> and provides the disks to the VM's. Maybe look at assigning specific
> exclusive CPU's to each dom0 and domU's, and see if you can still
> reproduce the issue. Also, make absolutely sure that you don't have two
> VM's accessing the same iSCSI.
>
> Regards,
> Adam
Dear Adam,

Thank you for your reply. I will try assigning specific cores to the VMs 
this weekend. I will also try to run all this on another machine and try 
using something different then Windows for the second VM (why didn't I 
think about it earlier?). When iSCSI target is on another physical 
machine, everything works like a charm, but the whole point is to make 
it all work on a single machine (everything faster than 1 Gbps is way 
beyond my reach, while iperf reports ~22 Gbps between dom0 and storage 
VM...)
Once again thank you for your suggestions, I will report back with more 
results. In the meantime please clarify one thing for me - is there 
something inherently wrong with what I'm trying to do?
Best regards,
Kuba

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users