[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] Hotplugging FPGA to running VM



Since I am new to the xen-users list, I was wondering if my question is more appropriate for the xen-devel list. Any thoughts?
 
Thanks.
-Anil


From: xen-users-bounces@xxxxxxxxxxxxx [mailto:xen-users-bounces@xxxxxxxxxxxxx] On Behalf Of Anil Rao
Sent: Thursday, May 31, 2012 11:25 AM
To: cyberhawk001@xxxxxxxxx; xen-users@xxxxxxxxxxxxx
Subject: Re: [Xen-users] Hotplugging FPGA to running VM

This is a proprietary FPGA device. Each control blade in our router chassis has one of these FPGAs and its primary function is to control certain dedicated hardware components on the blade that are responsible for establishing connectivity with the high-speed inter-blade fabric on the router. I am sorry, but I cannot divulge additional details regarding the FPGA.
 
Here is a more detailed description of my actual usage scenario.
 
1) The FPGA image must be downloaded onto the device before it can be enabled for use.
2) The VM in question is responsible for performing step (1), which means that the FPGA is not visible to Xen/Dom-0 when
    the VM is powered on.
3) After the VM performs step (1), the FPGA is seen in Dom-0. I use Xen pci-hotplug to assign the device to the VM.
    This operation completes successfully (as reported by the appropriate xl command).
4) However, the guest OS inside the VM cannot see the FPGA device after step (3).
5) If I shutdown the VM and re-start it, the guest OS inside the VM sees the FPGA device (and everything then proceeds normally).
 
The troublesome part here is that I need to shutdown and restart the VM. I would like to avoid this and rely on Xen pci-hotplug to make the device visible to the guest OS inside the VM.
 
Thanks.
-Anil
 

From: xen-users-bounces@xxxxxxxxxxxxx [mailto:xen-users-bounces@xxxxxxxxxxxxx] On Behalf Of cyberhawk001@xxxxxxxxx
Sent: Thursday, May 31, 2012 10:49 AM
To: xen-users@xxxxxxxxxxxxx
Subject: Re: [Xen-users] Hotplugging FPGA to running VM

Just out of curiosity, what kind of FPGA device is this? Is it an FPGA development board or something already assembled and functioning that contains an FPGA device(s)?

Well, just being curious... :)



Yes, the guest OS has PCI hotplug support. I am able to hotplug a variety of PCI devices to the VM. The only one that doesn't work is the FPGA device. Even for this one, the only time it doesn't work, is if the VM was powered-on prior to the FPGA device being brought on-line on the host. If the FPGA was present before the VM was powered-on everything works as expected.
 
It somehow seems like new devices that are introduced in the host, after a VM has been started on it, cannot be hotplugged into the VM.
 
Thanks.
-Anil
 
 

From: theubaz@xxxxxxxxx [mailto:theubaz@xxxxxxxxx] On Behalf Of John Sherwood
Sent: Wednesday, May 30, 2012 7:20 PM
To: Anil Rao
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-users] Hotplugging FPGA to running VM

Are you sure the guest OS has PCI hotplug support?

On Wed, May 30, 2012 at 10:14 PM, Anil Rao <anil.rao@xxxxxxxxxxxx> wrote:
Hi,
 
I am trying to assign a FPGA (which is a PCI device) to a running VM. The interesting thing here is that the FPGA is not present when the VM was powered-on. I.e. sometime after the VM was started, the FPGA was brought on-line on the host by downloading its image to the device.
 
So far my attempts at using Xen PCI-passthrough have not been successful. Although Xen reports that the pass-through operation was successful, which I have verified using the appropriate xl command, I am unable to see the FPGA from within the guest OS.
 
I have tried both manual rescanning of the PCI bus inside the guest as well as making use of the pciehp driver. The only way I can (reliably) see the FPGA from within the guest, is to shutdown and restart the VM and then redo the hotplug operation. I want to avoid a VM reboot and was wondering if this is at all possible and if so what do I need to do to realize it.
 
[The virtualization host is based on the default Xen setup that comes with SLES 11 SP2]
 
Thanks for any help on this matter.
 
-Anil
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.