[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] Issue with Win8 guest VGA Passthrough

Just to make sure I'm on the right path before continuing here: 
It looks like the Quadro FX 4800 and the GTX285 are using the GT200B GPU: 
GTX285 data and Quadro Data

The GTX 285 is an NV50: Tesla/NV50 info

Using Device IDs, I have that the GTX285 has an ID of 0x0165. The FX 4800 has an ID of 0x05FE

So it looks like I'm trying to modify the 4 bits 10-13. This is where my limited understanding breaks down, but I'm trying to find the right track here. The Fermi cards (NVC0) happen to use the same straps as the Tesla, it looks like (based on this: NV50 info)

So I'm going from 0000 0001 0110 0101 to 0000 0101 1111 1110

I need to modify xxxx x1xx 1xx1 1x10

That's not just an OR change anymore, and I'm lost at what he's doing for sure here, anyway. I'll keep looking at it, but this is a little beyond what I'm already familiar with. 

Any help here would be appreciated. 

On Wed, Feb 19, 2014 at 5:40 PM, Gordan Bobic <gordan@xxxxxxxxxx> wrote:
That was a long time ago. IIRC it required applying patches to Xen that ensured vBAR=pBAR and shortly after that was demonstrated to work, Nvidia modified the driver whitelist Quadro cards for virtualized operation. It is purely down to the device ID presented to the driver inside the VM. Modify the card's device ID, and the driver will initialize it even if it detects it is running virtualized.

Somebody mentioned on the list here that they got an unmodified card working by applying a patch that changes the behaviour the driver uses to detect whether it is in a VM, but I have not seen anything about it since.

Cards up to and including 4xx series can be BIOS-modded to change the device ID via the strap. Later cards include two sets of ID registers, one set by the hard-strap before the BIOS initializes, and one subject to being modified by the soft strap. This seems to have been designed specifically for Grid cards. The driver initializes the card based on the hard strap (to decide whether it will initialize it or not). Grid cards can emulate multiple different cards, for example a Quadro or a Tesla, which is effectively done by the soft strap. As far as I can work out, this is done by executing a BIOS payload in the VM to set the soft strap and perform suitable initialization, so a single Grid GPU can present different personalities to different VMs.

For your 285 it doesn't matter - just find a Quadro based on the same model GPU, get it's device ID, find where the device ID strap bits are for that generation of GPUs, re-flash the strap accordingly using nvflash and enjoy VGA passthrough. :)

It is probably only a matter of time before Nvidia do something to prevent this sort of thing in the driver, but we'll worry about it later. Older cards will always work with older drivers (i.e. anything Nvidia do in future drivers doesn't affect older cards since you don't need the new driver), and the current top of the like GTX780Ti is very easily modifiable into Quadro K6000 using a single 1206 33K-ohm resistor. So it would appear that VGA passthrough on the cheap is quite safe until at least the next generation of GPUs comes out in a year or two.

On 02/19/2014 10:29 PM, Kyle Davis wrote:
I'd seen people saying the GTX 285 worked for them with no changes to
the drivers, but I'll see if I can get this to work.


On Wed, Feb 19, 2014 at 4:12 PM, Gordan Bobic <gordan@xxxxxxxxxx
<mailto:gordan@xxxxxxxxxx>> wrote:

    GeForce card won't work - only Quadro/Tesla/Grid cards are
    whitelisted in the driver to make them work. You can modify your
    GTX285's device ID by changing the straps in it's BIOS to make it
    into a Quadro.

    This may be of interest:


    The straps on the 2xx series are a little different (check the
    nouveau strap documentation), but the general method will be similar.


    On 02/19/2014 08:54 PM, Kyle Davis wrote:

        Hello everyone,

        I'd posted this over at the new user forum on xenproject, and was
        directed here. Hopefully I'm not doing something stupid by
        copying and
        pasting my post from there, but here goes:

        I have a core i7-4770 on an ASRock Z87 Extreme 4 motherboard. I
        have a
        PCI-based USB hub attached to deal with the fact that all the
        USB ports
        on the motherboard are on the same controller for passthrough
        I am using Debian Jessie with Xen 4.3 and a kernel based on
        3.12.9. The
        card I'm trying to passthrough is an NVIDIA GTX 285 (I've found
        who say it worked fine for them, including on the boards here).

        I created a kernel included the flags suggested by the xen wiki,
        switched my toolstack over to xl (was on xm, and I'm unsure
        why), and
        followed the wiki's guides for setting up xen and VGA passthrough.

        I created a Windows 8 (HVM) guest that worked fine, but slowly,
        beginning the VGA passthrough guide. Changed my windows.cfg file
        to the

        bootloader ="/usr/bin/pygrub"
        memory =8192
        name ="windows8"
        vif =['bridge=xenbr0']

        device_model_version  ="qemu-xen-traditional"
        device_model_override ="/usr/bin/qemu-x86_64"
        acpi =1

        When the config worked, pci_permissive was commented out, as was
        the pci
        tag. The device override was set to qemu rather than
        qemu-x86_64, and
        the model version was qemu-xen rather than traditional. I've tried
        flipping things just to see if I could make it work to no avail.

        The error I get upon running the config is:

        libxl:  error:  libxl_dm.c:1275:device_model___spawn_outcome:

          domain2  devicemodel:  spawn failed (rc=-3)
        libxl:  error:
          libxl_create.c:1075:domcreate___devmodel_started:  device
        model did notstart:  -3

        libxl:  error:  libxl_dm.c:1306:libxl____destroy_device_model:

          Device  Model  already exited

        When I checked the log under /var/log/xen, I originally had
        -gfx_passthru: invalid option. After playing with the config
        file, I now
        have a hot mess:

        usage: qemu-x86_64 [options] program [arguments...]
        Linux CPU emulator (compiledfor  x86_64 emulation)

        Optionsand  associated environment variables:

        Argument      Env-variable      Description
        -h                              print this help
        -g port       QEMU_GDB          wait gdb connectionto  'port'
        -L path       QEMU_LD_PREFIXset  the  elf interpreter prefixto
        -s size       QEMU_STACK_SIZEset  the  stack sizeto  'size' bytes
        -cpu model    QEMU_CPU          select CPU (-cpu helpfor  list)

        -E var=value  QEMU_SET_ENV      sets targets environment
        variable (seebelow)
        -U var        QEMU_UNSET_ENV    unsets targets environment
        variable (seebelow)
        -0  argv0      QEMU_ARGV0        forces target process argv[0]to
          be 'argv0'
        -r uname      QEMU_UNAMEset  qemu uname releasestring  to  'uname'
        -B address    QEMU_GUEST_BASEset  guest_base addressto  'address'

        -R size       QEMU_RESERVED_VA  reserve 'size' bytesfor  guest
        virtual address $
        -ditem[,...] QEMU_LOG          enable loggingof  specified items
        (use '-d help$
        -D logfile    QEMU_LOG_FILENAMEwrite  logsto  'logfile' (default

        -p pagesize   QEMU_PAGESIZEset  the  host page sizeto  'pagesize'
        -singlestep   QEMU_SINGLESTEPrun  in  singlestep mode
        -strace       QEMU_STRACElog  system calls
        -version       QEMU_VERSION      displayversion  informationand

        QEMU_LD_PREFIX  = /etc/qemu-binfmt/x86_64
        QEMU_STACK_SIZE =8388608  byte

        You can use -Eand  -U optionsor  the  QEMU_SET_ENVand
        QEMU_UNSET_ENV environment variablesto  set  and  unset
        environment variablesfor  the  target process.
        Itis  possibleto  provide several variablesby  separating them
        by  commasin  getsubopt(3) style. Additionallyit  is  possibleto
        providethe  -Eand  -U options multipletimes.
        The following lines are equivalent:
              -E var1=val2 -E var2=val2 -U LD_PRELOAD -U LD_DEBUG
              -E var1=val2,var2=val2 -U LD_PRELOAD,LD_DEBUG

        Notethat  if  you provide several changesto  a single variable
        the  last  change will stayin  effect.

        Does anyone have some insight on this? I feel like I'm very close to
        getting my passthrough working, and that would be really awesome.

        Additionally, I tried to go graphically after being unsuccessful
        with my
        windows.cfg file, and tried the virtual machine manager. When I
        it, however, I get this:

        Unableto  connectto  libvirt.

        Failedto  connect socketto  '/var/run/libvirt/libvirt-__sock':

        No suchfile  or  directory

           - A Xen host kernel was booted
           - The Xen service has been started

        and under details :

        Unableto  connectto  libvirt.

        Failedto  connect socketto  '/var/run/libvirt/libvirt-__sock':

        No suchfile  or  directory

           - A Xen host kernel was booted
           - The Xen service has been started

        Libvirt URIis: xen:///

        Traceback (most recent calllast):

        line1027,in  _open_thread
              self.vmm = self._try_open()
        line1009,in  _try_open

        line105,in  openAuth
              if  retis  None:raise libvirtError('__virConnectOpenAuth()

        libvirtError: Failedto  connect socketto
          '/var/run/libvirt/libvirt-__sock': No suchfile  or  directory

        The virtual machine manager does not, thereby, connect to localhost

        I guess I should also mention that Windows 8 didn't always shut down
        properly, but it wasn't reporting errors and things like the GPLPV
        drivers seemed to install correctly.

        Thanks for any insight,

        Xen-users mailing list
        Xen-users@xxxxxxxxxxxxx <mailto:Xen-users@xxxxxxxxxxorg>

Xen-users mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.