Re: [Xen-users] can't passthrough video card

[bob evans <b.evans@xxxxxxxxx>]

Hi once again, I managed to get everything working! I had a very hard time to 
do this so I will share the steps that I followed and hope that it helps 
someone. This might help people even if they have different hardware than I do, 
but I think it will be the most help to people who use Ubuntu and who have 
Radeon graphics cards.

1. Install the dom0.

When installing dom0 you should use LVM. If you plan to use only virtual 
machines without using dom0 for anything else, you only need to make a 
partition for boot and root. Make the boot partition the first one, put root in 
the LVM. Your root should be about 15 gigabytes. Don't do anything with the 
rest of the space, you will use it for making a virtual drive for your domu. If 
you plan to use dom0 as your regular OS and only plan to make a Windows domu to 
play games then you should configure your drive however you want, but you 
should still leave some free space in the LVM for your Windows domu. How much 
space you leave for Windows is up to you, I think 100 gigabytes is enough. 
Using LVM like this will give you better performance than using a file.

2. Install xen.

The way to do this will probably depend on your dom0 operating system. 
https://help.ubuntu.com/community/Xen has some steps that you can follow if you 
use Ubuntu. 

sudo apt-get install xen-hypervisor-amd64
sudo sed -i 's/GRUB_DEFAULT=.*\+/GRUB_DEFAULT="Ubuntu GNU/Linux, with Xen 
hypervisor"/' /etc/default/grub
sudo update-grub
sudo sed -i 's/TOOLSTACK=.*\+/TOOLSTACK="xm"/' /etc/default/xen

The guide for Ubuntu is missing some steps. 

https://bugs.launchpad.net/ubuntu/+source/virt-manager/+bug/915954

sudo nano /etc/xen/xend-config.sxp
change (xend-unix-server no) to (xen-unix-server yes), uncomment 
(xen-unix-server yes) and (xend-unix-path /var/lib/xend/xend-socket)
ctrl-o saves ctrl-x exits

http://www.howtoforge.com/paravirtualization-with-xen-4.0-on-debian-squeeze-amd64

mv /etc/grub.d/10_linux /etc/grub.d/50_linux
update-grub2
update-grub
sudo reboot

3. install virt-manager, configure networking

virt-manager is very buggy and I almost don't like it, but I still used it for 
many steps. 

sudo apt-get install virt-manager 

Don't forget that your dom0 is a virtual machine, the virtual network 
interfaces are used by dom0 to connect to the internet. I don't know how you 
want to configure your networking but I will point you in the right direction. 
You can run virt-manager with the command virt-manager and then go to edit and 
then go to connection details and then go to network interfaces. You should 
read more about networking with Xen and figure out what you want, I had a lot 
of trouble with figuring out how to configure things. You should know that the 
Ubuntu network manager doesn't really do anything on Xen even though it looks 
like it does, I removed it from my system tray. You probably want to make a new 
network interface by clicking on the green plus sign and telling it to make a 
bridge that you add an eth interface to. If your internet stops working try 
restarting, this is very buggy. 

4. make sure that you have VT-d enabled, blacklist radeon drivers.

You need to restart and go to your BIOS. It should have an option for VT-d if 
your CPU and motherboard support it. AMD has something like VT-d called IOMMU. 
After you enable VT-d in your BIOS boot back into your dom0. Now you need to 
tell Xen to use VT-d. 

nano /etc/default/grub 

There are a few changes you need to make in this file. Here is what I have in 
mine

GRUB_CMDLINE_XEN="iommu=1 dom0_mem=2048M radeon.modeset=0"

iommu=1 tells Xen to use VT-d, dom0_mem tells Xen how much memory to let dom0 
use, radeon.modeset=0 tells Xen not to load the radeon driver. If you've got a 
radeon graphics card you probably want to add radeon.modeset=0 but if more than 
your graphics card uses that driver it will stop whatever else uses it from 
working. You can see all of your PCI devices that use the radeon driver with 
lspci -k, if only your graphics card is using it then you want to add that 
line. If you have an Nvidia graphics card I don't know what you need to do. I 
also put the same radeon.modeset=0 in my GRUB_CMDLINE_LINUX, I don't know if 
this is what stopped the driver from loading or if it was from putting it in 
the xen line. After doing this make sure to sudo update-grub, I also sudo 
update-grub2 but I don't know if I needed to. 

I also did nano /etc/modprobe.d/blacklist.conf and added "blacklist radeon" to 
the bottom of the file. I don't think this is what finally managed to blacklist 
the radeon driver, but I never changed it back. 

sudo reboot


5. Configure pci passthrough

lspci -k | grep radeon

there shouldn't be any output from this command, because hopefully your radeon 
driver didn't load. You need to start xen-pciback now, for me this was a module 
but I don't know why and I don't know how to make it part of my kernel.

modprobe xen-pciback

lspci -k

you should get a big list of all of your PCI devices, mine looks like this

02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] 
Tahiti XT [Radeon HD 7970/R9 280X]
Subsystem: Hightech Information System Ltd. Tahiti XT2 [Radeon HD 7970 GHz 
Edition]
Kernel driver in use: pciback
02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT HDMI 
Audio [Radeon HD 7970 Series]
Subsystem: Hightech Information System Ltd. Device aaa0
Kernel driver in use: pciback

pay special attention to the numbers in the xx:xx.x format. You need to find 
the devices that you want to passthrough to Windows. Probably you want to pass 
your VGA compatible controller and your Audio Device. These might have 
different names on your system. Both are part of your video card. I read that 
if the first numbers are not the same that you can't get this to work, for 
example 02:00.0 and 02:00.1 is ok but 03:00.0 and 02:00.0 isn't going to work. 
I don't know if this is true or not. If your video card has a VGA controller 
and an audio controller you need to passthrough both of them. 

Now you need to give the following command for each of your devices, replace 
the numbers with the ones you got from lspci 

echo 0000:02:00.1 > /sys/bus/pci/devices/0000:02:00.1/driver/unbind
echo 0000:02:00.1 > /sys/bus/pci/drivers/pciback/new_slot
echo 0000:02:00.1 > /sys/bus/pci/drivers/pciback/bind 

I noticed that after I blacklisted my radeon driver my radeon VGA compatible 
controller would automatically bind to xen-pciback, but I still had to unbind 
and bind all of my other devices. I'm still stuck on getting my USB controller 
to passthrough, now it gives me the same 'device not found' error that I had 
when I was first trying to passthrough my graphics card. At first I couldn't 
passthrough my graphics card but could passthrough my USB controller, now I can 
passthrough my graphics card but I can't passthrough my USB controller. Gordan 
Bobic suggested that I use xl instead of xm, this was the first thing I did 
that let me passthrough my graphics card but xl is very buggy compared to xm 
and I switched back. I think blacklisting the radeon driver is what let me use 
the echo commands but I don't know why my USB controller can't passthrough now.

6. Install the Windows HVM. 

I did this in virt-manager but I don't know if you should do this because it 
will break after you install the drivers. I will say how I did it and how I 
fixed it.

First you need to make a logical volume to install Windows to. In virt-manager 
go to edit and then go to connection details and then go to storage and then 
click on the name of your logical volume group and then click the New Volume 
button. You can name the volume whatever you want to and give it as much space 
as you want, I allocated all of the space right away. In virt-manager click on 
the create a new virtual machine button and go through the steps for making a 
windows HVM. I gave my Windows HVM 8 gigabytes of memory and 5 CPUs. On step 4 
select managed or other existing storage and click the browse button. Select 
the logical volume that you just made. On the last step make sure that you 
select the network interface you want, this is probably a bridge. Select to 
customize configuration before install and then click finish. A window will 
appear, click on the add hardware button. Now click on PCI host device and 
select the PCI device you want to passthrough. You need to do this for each of 
the PCI devices you want to passthrough. Make sure that you've already bound 
all of these devices to xen-pciback. Now finish and start the Windows 
installation. 


7. Configure Windows 

After you have Windows installed you need to install the drivers for your video 
card. Some people report trouble when using Catalyst Control Center, I only 
selected to install the video and audio drivers to try to avoid problems. It 
works for some people though.
After your drivers are installed you will be asked to reboot. Go ahead and 
reboot, you will probably boot into a BSOD though. At this point I tried 
various things to make this work. I repeatedly rebooted dom0 because 
virt-manager wasn't able to power the Windows HVM on again after the BSOD. 
Rebooting dom0 fixed this. Every time you reboot dom0 you need to do step 5 
again. You can make a script to do this if you want. No matter how many times I 
rebooted and tried to make it work I always ended up with a BSOD. At this point 
you need to stop using virt-manager.

cd /etc/xen
mkdir windows
cd windows
nano windows.conf

my windows.conf file looks like this, I copied almost all of it from a post I 
saw on a forum.

builder='hvm'
memory = 8192
vcpus=5
name = "win7"
vif = [ 'type=ioemu, bridge=xenbr0' ]
disk = [ 'phy:/dev/lvg/windows,xvda,w' ]
acpi = 1
boot="c"
sdl=0
serial='pty'
vnc=1
pci=[ '02:00.0', '02:00.1' ]

for memory and vcpus you should put what you want to give to your windows HVM. 
Vif and disk should be modified for your configuration. You also need to put 
the numbers of your PCI devices in pci, remember that you can find these 
numbers with lspci. You need to use a different name than you used when you 
made your Windows HVM. 

ctrl-o to save and ctrl-x to quit

xm create windows.conf

now Windows starts and you can go and look at it in virt-manager just like your 
first Windows HVM. Instead of BSOD you should now have your screen freeze 
during the loading Windows part of booting. Now you can switch your monitors 
input to whatever your Radeon card uses and you should go to full screen 
Windows! If you can't move your mouse you need to switch back to your regular 
video input and click into the vnc window. If you manage to passthrough a USB 
controller you can probably use a USB switch.

____________________________________________________________
FREE 3D MARINE AQUARIUM SCREENSAVER - Watch dolphins, sharks & orcas on your 
desktop!
Check it out at http://www.inbox.com/marineaquarium



_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users