Xen project Mailing List

Zir,

On Wed, Aug 28, 2013 at 12:38 PM, Zir Blazer <zir_blazer@xxxxxxxxxxx> wrote:

<snip>

Up to this point, everything is well. The problem is the next part...

As when you do passthrough of a device, neither other VMs nor the Hypervisor can use them without reassigning that device (And that device also needed a Soft Reset function or something like that if I did my homework properly), suddently I have to decide in what VMs I want my 3 GPUs and 2 Monitors (That should be physically connected to them, so video output will be of where the GPU is at that moment) to be at, and what could get away by using emulated Drivers (This should also apply to Audio, but I think than that one is fully emulable). Considering this, I suddently have to really think what goes where, which is what I can't decide.
Assuming I do passthrough of the 2 GPUs to the said Linux VM for mining, I wouldn't need a Monitor attached to either, and I could do passthrough of the IGP to the Windows gaming VM with Dual Monitors. However, in this case, I suppose that I wouldn't have video output from neither the Hypervisor nor any of the other VMs, including my everday Linux one, killing the whole point of it, unless that IGP can be automatically reassigned on-the-fly, which I doubt. This means that the most flexible approach, would be to leave the IGP with a single Monitor for the Hypervisor, and each 5770 to a Windows gaming VM, but then, I would be short on one Monitor. Basically, due to the fact that a Monitor is attached to a GPU that may or not be where I want the video output at, I may need to switch Monitors from output often, too. So in order to do it with only passthrough, I will have to take some decisions here.

So, I have to find other solutions. There are both that are very interesing, and technically both should be possible, through I don't know if anyone tried them.

The first one, is figuring out if you can route the GPU output somewhere else instead of that Video Card's own video output. As far that I am aware, there are some Software solutions that allows to do something like this: One is Windows-based Lucid Virtu, and the other nVidia Optimus Drivers. Both are conceptually the same: They switch between the IGP and the discrete GPU depending on the workload. However, the Monitor is always attached to the IGP output, and what they do, is copying the framebuffer from the Video Card to the integrated one, so you can use the discrete GPU for processing while redirecting it to the IGP's video output. If you can do something like this on Xen, it would be extremely useful, because I could have the two Monitors always attached to the IGP and simply reassign the GPUs to different VMs as needed.

Another possible solution, would be assuming that the virtual GPU technologies catch up, as I am aware that XenServer, that is based on Xen, is supposedly able to use a special GPU Hypervisor that allows a single physical GPU to be shared in several VMs simultaneously as a virtual GPU (In the same fashion that VMs currently see the vCPUs). This one sounds like THE ultimate solution. Officially, nVidia support this only on the GRID series, while AMD was going to release the Radeon Sky aimed for the same purpose, through I don't know what Software solutions it brings. However, it IS possible to mod Video Cards for them to be detected as their professional counterparts and maybe that allows the use of the advanced GPU virtualization technologies only available on these expensive series:

http://www.nvidia.com/object/grid-vgx-software.html
http://blogs.citrix.com/2013/08/26/preparing-for-true-hardware-gpu-sharing-for-vdi-with-xenserver-xendesktop-and-nvidia-grid/
http://www.eevblog.com/forum/projects/hacking-nvidia-cards-into-their-professional-counterparts/

I think that there are some people that likes to mod GeForces to Quadros because they're easier to passthrough in Xen. But I'm aiming one step above that should I want a GeForce @ Grid mod, as I think that full GPU virtualization would be a killer feature.

All my issues are regarding this last part. Do someone have any input regarding what can and can not be currently done? I will need something quite experimental to make my setup work as I intend it to. Another thing which could be a showstopper, is the 2 GB limit on VMs with VGA passthrough I have been hearing, throughI suppose will get fixed in some future Xen version. I'm looking for ideas and people that already tried this experiences to deal with it.

You might be interested in a recent announcement I saw on the xen-devel mailing list, around work on a graphics virtualization solution : http://lists.xenproject.org/archives/html/xen-devel/2013-09/msg00681.html

Regards,

David

Re: [Xen-users] Gettind ready for a computer build for IOMMU virtualization, need some input regarding sharing the GPU among VMs