Re: [Xen-users] pci passthrough under xen-3.3 without VT-d

["Todd Deshane" <deshantm@xxxxxxxxx>]

On Mon, Dec 15, 2008 at 6:36 PM, Benjamin Huntsman
<BHuntsman@xxxxxxxxxxxxxxxxxxxxx> wrote:
> Hi all!
>   I'm hoping someone here might be able to help.  I'm familiar with several 
> UNIX'es, though farily new to Linux...
> I'm running RHEL 5.2.  I've got the RHEL kernel headers installed, etc, and 
> compiled xen-3.3.
> Once that was built, I rebuilt libvirt, etc, the kernel, and all the modules. 
>  The whole mess boots and runs fine... mostly.
>
> My hardware is a Dell OptiPlex 745, Intel Core2 Duo, though the board doesn't 
> have VT-d capabilities.  Vitalization is otherwise enabled in the BIOS.  I 
> have a PCI NIC and PCI VGA card I want to pass through to a guest.  I have 
> pciback built as a module, and it's preloaded in my initrd.
>
> As I understand it though, even w/o VT-d, I should still be able to do PCI 
> passthrough, right?
>

Yes, as long as you understand the security implications (i.e driver
DMA to memory it doesn't own)


> Here's the config details (pardon the verbosity):
>
> from /boot/grub/menu.lst:
> ...
> title Red Hat Enterprise Linux Server (2.6.18-92.1.18.el6xen)(Xen-3.3.0)
>        root (hd0,1)
>        kernel /xen-3.3.0.gz iommu=1 vtd=1

I am pretty sure that you don't want the iommu line in here, since you
don't have
an iommu


>        module /vmlinuz-2.6.18-92.1.18.el5xen ro root=/dev/VolGroup00/LogVol00 
> rhgb quiet

instead of doing the modprobe below, you could instead pass the pciback.hide
on the dom0 command line. It should give a similar effect, but still
worth a shot.


>        module /initrd-2.6.18-92.1.18.el5xen.img
> ...
>
> from /etc/modprobe.conf:
> ...
> install e100 /sbin/modprobe pciback ; /sbin/modprobe --first-time 
> --ignore-install e100
> options pciback hide=(04:00.0)(04:02.0)
> ...
>
> [root@hx80kc1 ~]# dmesg | grep pciback
> pciback 0000:04:02.0: seizing device
> pciback 0000:04:00.0: seizing device
> [root@hx80kc1 ~]#
>
> [root@hx80kc1 ~]# xm dmesg
>  __  __            _____  _____  ___
>  \ \/ /___ _ __   |___ / |___ / / _ \
>  \  // _ \ '_ \    |_ \   |_ \| | | |
>  /  \  __/ | | |  ___) | ___) | |_| |
>  /_/\_\___|_| |_| |____(_)____(_)___/
>
> (XEN) Xen version 3.3.0 (root@xxxxxxxxxxxxxxx) (gcc version 4.1.2 20071124 
> (Red Hat 4.1.2-42)) Fri Dec  5 17:21:39 PST 2008
> (XEN) Latest ChangeSet: unavailable
> (XEN) Command line: iommu=1 vtd=1
> (XEN) Video information:
> (XEN)  VGA is text mode 80x25, font 8x16
> (XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
> (XEN) Disc information:
> (XEN)  Found 1 MBR signatures
> (XEN)  Found 1 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN)  0000000000000000 - 000000000009fc00 (usable)
> (XEN)  00000000000f0000 - 0000000000100000 (reserved)
> (XEN)  0000000000100000 - 00000000bfdffc00 (usable)
> (XEN)  00000000bfdffc00 - 00000000bfe01c00 (ACPI NVS)
> (XEN)  00000000bfe03c00 - 00000000bfe53c00 (reserved)
> (XEN)  00000000bfe53c00 - 00000000bfe55c00 (ACPI data)
> (XEN)  00000000bfe55c00 - 00000000c0000000 (reserved)
> (XEN)  00000000e0000000 - 00000000f0000000 (reserved)
> (XEN)  00000000fec00000 - 00000000fed00400 (reserved)
> (XEN)  00000000fed20000 - 00000000feda0000 (reserved)
> (XEN)  00000000fee00000 - 00000000fef00000 (reserved)
> (XEN)  00000000ffb00000 - 0000000100000000 (reserved)
> (XEN)  0000000100000000 - 000000013c000000 (usable)
> (XEN) System RAM: 4029MB (4126328kB)
> (XEN) ACPI: RSDP 000FEBF0, 0024 (r2 DELL  )
> (XEN) ACPI: XSDT 000FCE90, 006C (r1 DELL    B8K           14 ASL        61)
> (XEN) ACPI: FACP 000FCFB8, 00F4 (r3 DELL    B8K           14 ASL        61)
> (XEN) ACPI: DSDT FFF69C33, 4757 (r1   DELL    dt_ex     1000 INTL 20050624)
> (XEN) ACPI: FACS BFDFFC00, 0040
> (XEN) ACPI: SSDT FFF6E662, 009A (r1   DELL    st_ex     1000 INTL 20050624)
> (XEN) ACPI: APIC 000FD0AC, 0092 (r1 DELL    B8K           14 ASL        61)
> (XEN) ACPI: BOOT 000FD13E, 0028 (r1 DELL    B8K           14 ASL        61)
> (XEN) ACPI: ASF! 000FD166, 0092 (r32 DELL    B8K           14 ASL        61)
> (XEN) ACPI: MCFG 000FD1F8, 003E (r1 DELL    B8K           14 ASL        61)
> (XEN) ACPI: HPET 000FD236, 0038 (r1 DELL    B8K           14 ASL        61)
> (XEN) ACPI: TCPA 000FD492, 0032 (r1 DELL    B8K           14 ASL        61)
> (XEN) ACPI: SLIC 000FD26E, 0176 (r1 DELL    B8K           14 ASL        61)
> (XEN) Xen heap: 14MB (14624kB)
> (XEN) Domain heap initialised
> (XEN) Processor #0 6:15 APIC version 20
> (XEN) Processor #1 6:15 APIC version 20
> (XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 2126.508 MHz processor.
> (XEN) HVM: VMX enabled
> (XEN) CPU0: Intel(R) Core(TM)2 CPU          6400  @ 2.13GHz stepping 06
> (XEN) Booting processor 1/1 eip 8c000
> (XEN) CPU1: Intel(R) Core(TM)2 CPU          6400  @ 2.13GHz stepping 06
> (XEN) Total of 2 processors activated.
> (XEN) ENABLING IO-APIC IRQs
> (XEN)  -> Using new ACK method
> (XEN) checking TSC synchronization across 2 CPUs: passed.
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Brought up 2 CPUs
> (XEN) I/O virtualisation disabled
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN)  Xen  kernel: 64-bit, lsb, compat32
> (XEN)  Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 
> 0xffffffff807014e4
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN)  Dom0 alloc.:   0000000134000000->0000000136000000 (969614 pages to be 
> allocated)
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN)  Loaded kernel: ffffffff80200000->ffffffff807014e4
> (XEN)  Init. ramdisk: ffffffff80702000->ffffffff80ea6000
> (XEN)  Phys-Mach map: ffffffff80ea6000->ffffffff8161bc70
> (XEN)  Start info:    ffffffff8161c000->ffffffff8161c4a4
> (XEN)  Page tables:   ffffffff8161d000->ffffffff8162c000
> (XEN)  Boot stack:    ffffffff8162c000->ffffffff8162d000
> (XEN)  TOTAL:         ffffffff80000000->ffffffff81800000
> (XEN)  ENTRY ADDRESS: ffffffff80200000
> (XEN) Dom0 has maximum 2 VCPUs
> (XEN) Scrubbing Free RAM: .done.
> (XEN) Xen trace buffers: disabled
> (XEN) Std. Loglevel: Errors and warnings
> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to 
> Xen)
> (XEN) Freed 108kB init memory.
> (XEN) mtrr: type mismatch for c0000000,1000000 old: write-back new: 
> write-combining
> (XEN) mtrr: type mismatch for c0000000,1000000 old: write-back new: 
> write-combining
> [root@hx80kc1 ~]#
>
> [root@hx80kc1 ~]# lspci
> ...
> 04:00.0 Ethernet controller: Intel Corporation 82557/8/9/0/1 Ethernet Pro 100 
> (rev 05)
> ...
> 04:02.0 VGA compatible controller: nVidia Corporation NV5M64 [RIVA TNT2 Model 
> 64/Model 64 Pro] (rev 15)
> ...
> [root@hx80kc1 ~]#
>
> [root@hx80kc1 ~]# lspci -n
> ...
> 04:00.0 0200: 8086:1229 (rev 05)
> 04:02.0 0300: 10de:002d (rev 15)
> #
>
> # tail -n 10 /etc/xen/xend-pci-permissive.sxp
> # (unconstrained_dev_ids
> #     ('XXXX:XXXX:XXXX:XXXX'    # existing entry
> #      'YYYY:YYYY:YYYY:YYYY'    # new entry 1
> #      'ZZZZ:ZZZZ')             # new entry 2
> # )
> ###############################################################################
> (unconstrained_dev_ids
>     #('0123:4567:89AB:CDEF')
>     ('10de:002d')
> )
> [root@hx80kc1 ~]#
>
> [root@hx80kc1 ~]# grep pci /etc/xen/winxp
> pci = [ "04:00.0" ]
> [root@hx80kc1 ~]#
>
> [root@hx80kc1 ~]# xm pci-list-assignable-devices

I'm not sure it this command is setup to work with anything
other than the VT-d stuff.

> [root@hx80kc1 ~]#
>
> [root@hx80kc1 ~]# xm create /etc/xen/winxp
> Using config file "/etc/xen/winxp".
> Error: failed to assign device: maybe the platform doesn't support VT-d, or 
> VT-d
>  isn't enabled properly?


I think you are getting this error because you passed iommu=1

> [root@hx80kc1 ~]#
>
>
> --------------------------------------
> Okay, there's as much detail as I can think to include right now.  I'm 
> concerned that 'xm dmesg' shows the following message:
> (XEN) I/O virtualisation disabled
> Thoung I don't know if that would actually prevent PCI passthrough.
>
> So, how do I get the PCI card passed through to my hvm domain?  What am I 
> missing?
>

You can't pass to a HVM domain without VT-d. You can only pass to a PV guest.

Hope that helps.

Cheers,
Todd


-- 
Todd Deshane
http://todddeshane.net
http://runningxen.com

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users