Re: [Xen-devel] IOMMU and AMD 890fx

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

Hi Jens,

Thx for reporting your successful pci passthrough with an AMD IOMMU, i have put 
this motherboard in the xen vt-d/iommu wiki as a motherboard with the BIOS 
supporting the IOMMU.
http://wiki.xensource.com/xenwiki/VTdHowTo

--
Sander

> Hi!

> Thanks for the tips!

> With xen-4.0-testing.hg tree + 2.6.32-bpo.5-xen-amd64 (from debian lenny 
> backports) it works. So the 4.0.0 release could be buggy?
> When I compile and install xen-4.0-testing.hg tree it works as you 
> described.
> With 4.0.0 it did not work with any pci device, with xen-4.0-testing.hg 
> tree it works with the two I tested for the moment:

> 00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev 40)
> 03:00.0 Ethernet controller: Marvell Technology Group Ltd. Device 4381 
> (rev 11)

> (I just randomly take two devices).
> They appear in the hvm virtual machine and they are working.

> Great, thank you very much!

> Jens


> Am 17.06.2010 19:16, schrieb Wei Huang:
>> Hi Jens,
>>
>> Thanks for testing it out. I tested Xen 4.0 on my machine this morning.
>> The passthru did work. Here are the details:
>>
>> 1. I used Xen 4.0 (xen-4.0-testing.hg tree) + 2.6.18.8 Dom0
>> 2. My system is a workstation; so it doesn't have a HDA device. Here is
>> what I get from lspci (I skipped some of un-related devices):
>> ==========
>> ...
>> 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3b)
>> 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller
>> 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
>> 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
>> 00:14.5 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI2
>> Controller
>> ...
>> 01:00.0 VGA compatible controller: ATI Technologies Inc Device 949e
>> 01:00.1 Audio device: ATI Technologies Inc RV730XT Audio device [Radeon
>> HD 4670]
>> 02:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
>> Controller (rev 06)
>> 02:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet
>> Controller (rev 06)
>> 03:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet
>> Controller (Copper) (rev 06)
>> ==========
>> As you can see, I have two NICs (a two-port Intel 82571EB NIC and a
>> single-port 82572EI NIC). I am going to passthru 82571EB NIC to a Linux
>> guest VM.
>>
>> 3. Here is the output from "xm dmesg". Because you also see a similar
>> message, I think your IOMMU is working.
>> (XEN) AMD-Vi: IOMMU 0 Enabled.
>> (XEN) I/O virtualisation enabled
>> (XEN) I/O virtualisation for PV guests disabled
>>
>> FYI, here is my grub entry:
>> ==================
>> title 64bit Xen-unstable on SLES11
>> root (hd0,5)
>> kernel /boot/xen.gz dom0_mem=512M console=com2 com2=115200,8n1 iommu=1
>> module /boot/vmlinuz-2.6.18.8 root=/dev/hdb6 resume=/dev/hdb3
>> module /boot/initrd-2.6.18.8
>> ==================
>>
>> 4. After login, I removed 82571EB from dom0
>>
>>  > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/e1000e/unbind
>>  > echo -n "0000:02:00.1" > /sys/bus/pci/drivers/e1000e/unbind
>>  > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/pciback/new_slot
>>  > echo -n "0000:02:00.1" > /sys/bus/pci/drivers/pciback/new_slot
>>  > echo -n "0000:02:00.0" > /sys/bus/pci/drivers/pciback/bind
>>  > echo -n "0000:02:00.1" > /sys/bus/pci/drivers/pciback/bind
>>
>> 5. In guest (ttylinux) configure file, I use the following setting.
>> ...
>> pci=[ '02:00.0', '02:00.1' ]
>> ...
>>
>> 6. After ttylinux booted, I saw two Intel NICs. The IP address was
>> obtained automatically. See the attached image file.
>>
>> Same as what Konrad has mentioned, I would recommend two things: i)
>> instead of passthru HDA, try other independent PCIe devices first (such
>> as NICs); ii) configure the passthru devices in guest configure file,
>> instead of use it a hot-plug device. I can send you my guest configure
>> file if you need it.
>>
>> PS: I am pretty confident HDA passthru work because I tested it on an
>> desktop system before. Here is the list of devices I have passthru to a
>> Win7 guest VM on that system: (1) USB keyboard and mouse; (2) HDA audio;
>> (3) ATI graphics.
>>
>> Thanks,
>> -Wei
>>
>>
>>
>> Konrad Rzeszutek Wilk wrote:
>>> On Thu, Jun 17, 2010 at 12:09:31AM +0200, Jens Krehbiel-Gräther wrote:
>>>> Hi!
>>>>
>>>> I now have a Asus Crosshair IV for testing and this board (as I
>>>> postet earlier) supports iommu. Now I have installed Debian lenny
>>>> (5.0) and compiled xen 4.0.0.
>>>> Xen is running now and I have installed a hvm debian lenny to which
>>>> I would like to bind a pci device but that doesn't work.
>>>> Perhaps I am doing something wrong and you could help me?
>>>
>>> Is that the _only_ PCI device you have tried to assign? Did you try to
>>> assign other PCI devices which have the function number being zero?
>>>> and xm dmesg this:
>>>>
>>>> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device
>>>> (0:14.2) failed
>>>
>>> That is not very useful. Wish it gave you the return code at least.
>>> If you are comfortable with adding printks in the Xen hypervisor I
>>> would modify it a bit to see what was the 'ret' value and why
>>> 'intel_iommu_assign_device' function failed.
>>>
>>>> So can anyone tell me what I am doing wrong??
>>>
>>> Please also attach the full serial long, including the Xen one - and run
>>> it with 'loglvl=all iommu=verbose'.
>>>
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-devel



-- 
Best regards,
 Sander                            mailto:linux@xxxxxxxxxxxxxx


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel