Re: [Xen-users] PCI Passthrough not working

[Geert Janssens <info@xxxxxxxxxxxx>]

One thing I forgot to mention:
The domU kernel seems to be compiled with Xen PCI frontend built-in.

Regards,

Geert

On Wednesday 25 April 2007 23:05, Geert Janssens wrote:
> Hi,
>
> I have installed CentOS 5 with Xen on a new Dell SC440.
>
> I have hidden a network interface in dom0 to pass it on to domU by adding
> these lines in /etc/modprobe.conf:
> options pciback hide=(04:00.0)
> install tg3 /sbin/modprobe
>           pciback ; /sbin/modprobe --first-time --ignore-install tg3
> CentOS builds pciback as a modules, so I could not add pciback.hide in the
> grub configuration. I found the modprobe.conf setup in the XenSource wiki.
>
> I am not sure if this part works fine: the nic is still visible via lspci
> in dom0, although it is mentioned as disabled.
> # lspci -vb
> ...
> 04:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5754 Gigabit
> Ethernet PCI Express (rev 02)
>         Subsystem: Dell Unknown device 01df
>         Flags: fast devsel, IRQ 10
>         Memory at 00000000efbf0000 (64-bit, non-prefetchable) [disabled]
>         Capabilities: [48] Power Management version 3
>         Capabilities: [50] Vital Product Data
>         Capabilities: [58] Vendor Specific Information
>         Capabilities: [e8] Message Signalled Interrupts: 64bit+ Queue=0/0
> Enable-
>         Capabilities: [d0] Express Endpoint IRQ 0
> ...
>
> Also in dom0:
> # ls -l /sys/bus/pci/drivers/pciback
> total 0
> lrwxrwxrwx 1 root root    0 Apr 25 13:06
> 0000:04:00.0 -> ../../../../devices/pci0000:00/0000:00:1c.5/0000:04:00.0
> --w------- 1 root root 4096 Apr 25 13:06 bind
> lrwxrwxrwx 1 root root    0 Apr 25 13:06 module ->
> ../../../../module/pciback --w------- 1 root root 4096 Apr 25 13:06 new_id
> --w------- 1 root root 4096 Apr 25 13:06 new_slot
> -rw------- 1 root root 4096 Apr 25 13:06 permissive
> -rw------- 1 root root 4096 Apr 25 13:06 quirks
> --w------- 1 root root 4096 Apr 25 13:06 remove_slot
> -r-------- 1 root root 4096 Apr 25 13:06 slots
> --w------- 1 root root 4096 Apr 25 13:06 unbind
>
> seems to suggest that the nic is really bound to pciback. It's not
> in /sys/bus/pci/drivers/tg3 anyway.
>
> I then start a domU with the following configuration:
> memory = 256
> name = "CentOSFileserverLVM"
> vcpus = 2
> vif = [ 'mac=aa:cc:00:00:00:02, bridge=xenbr0' ]
> pci = [ '04:00.0' ]
> disk = [ 'phy:base/fwroot,xvda,w',
>          'phy:base/fwhome,xvdb1,w',
>          'phy:base/fwvar,xvdb2,w',
>          'phy:base/fwbackup,xvdb3,w',
>          'phy:base/fwswap,xvdb4,w' ]
> bootloader="/usr/bin/pygrub"
> bootargs="/dev/base/fwroot"
> root = "/dev/xvda2"
> vfb = [ 'type=vnc,
>          vnclisten=0.0.0.0,
>          vncunused=1,
>          display=localhost:12.0' ]
>
> This boots fine, but I don't find the nic in domU. lspci returns absolutely
> nothing. From reading messages on different forums (like this one:
> http://www.vdr-portal.de/board/thread.php?threadid=60330), I gather the
> boot messages for domU should contain some entries like
>
> > pcifront pci-0: Installing PCI frontend
> > pcifront pci-0: Creating PCI Frontend Bus 0000:03
>
> and
>
> > PCI: Enabling device 0000:03:04.0 (0000 -> 0003)
>
> or similar.
>
> It seems this is not the case for my setup. I have no idea what goes wrong
> and why. Did anybody else get stuck at this point, and managed to fix it ?
>
> Some additional info:
> dom0# uname -a
> Linux localhost.localdomain 2.6.18-8.1.1.el5xen #1 SMP Mon Apr 9 11:46:46
> EDT 2007 i686 i686 i386 GNU/Linux
>
> dom0# rpm -q xen
> xen-3.0.3-25.0.3.el5
>
> domU# uname -a
> Linux localhost.localdomain 2.6.18-8.1.1.el5xen #1 SMP Mon Apr 9 11:46:46
> EDT 2007 i686 i686 i386 GNU/Linux
>
> domU# dmesg  (I removed the SELinux audit entries for readability)
> # dmesg
> Linux version 2.6.18-8.1.1.el5xen (mockbuild@xxxxxxxxxxxxxxxxxxx) (gcc
> version 4.1.1 20070105 (Red Hat 4.1.1-52)) #1 SMP Mon Apr 9 11:46:46 EDT
> 2007 BIOS-provided physical RAM map:
>  Xen: 0000000000000000 - 0000000010800000 (usable)
> 0MB HIGHMEM available.
> 264MB LOWMEM available.
> NX (Execute Disable) protection: active
> On node 0 totalpages: 67584
>   DMA zone: 67584 pages, LIFO batch:15
> ACPI in unprivileged domain disabled
> Built 1 zonelists.  Total pages: 67584
> Kernel command line:  root=/dev/xvda2 ro root=LABEL=/ rhgb quiet
> Enabling fast FPU save and restore... done.
> Enabling unmasked SIMD FPU exception support... done.
> Initializing CPU#0
> CPU 0 irqstacks, hard=c071b000 soft=c06fb000
> PID hash table entries: 2048 (order: 11, 8192 bytes)
> Xen reported: 1862.047 MHz processor.
> Console: colour dummy device 80x25
> Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
> Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
> Software IO TLB disabled
> vmalloc area: d1000000-f4ffe000, maxmem 2d7fe000
> Memory: 251580k/270336k available (2017k kernel code, 10252k reserved, 824k
> data, 172k init, 0k highmem)
> Checking if this processor honours the WP bit even in supervisor mode...
> Ok. Calibrating delay using timer specific routine.. 4657.11 BogoMIPS
> (lpj=9314232)
> Security Framework v1.0.0 initialized
> SELinux:  Initializing.
> SELinux:  Starting in permissive mode
> selinux_register_security:  Registering secondary module capability
> Capability LSM initialized as secondary
> Mount-cache hash table entries: 512
> CPU: After generic identify, caps: bfebc3f1 20100000 00000000 00000000
> 0000e3bd 00000000 00000001
> CPU: After vendor identify, caps: bfebc3f1 20100000 00000000 00000000
> 0000e3bd 00000000 00000001
> CPU: L1 I cache: 32K, L1 D cache: 32K
> CPU: L2 cache: 2048K
> CPU: After all inits, caps: bfebc3f1 20100000 00000000 00000940 0000e3bd
> 00000000 00000001
> Checking 'hlt' instruction... OK.
> SMP alternatives: switching to UP code
> CPU 1 irqstacks, hard=c071c000 soft=c06fc000
> Brought up 1 CPUs
> sizeof(vma)=88 bytes
> sizeof(page)=32 bytes
> sizeof(inode)=340 bytes
> sizeof(dentry)=136 bytes
> sizeof(ext3inode)=492 bytes
> sizeof(buffer_head)=52 bytes
> sizeof(skbuff)=172 bytes
> checking if image is initramfs... it is
> Freeing initrd memory: 3071k freed
> Grant table initialized
> NET: Registered protocol family 16
> ACPI Exception (utmutex-0262): AE_BAD_PARAMETER, Thread C0ABCAA0 could not
> acquire Mutex [2] [20060707]
> SMP alternatives: switching to SMP code
> Initializing CPU#1
> migration_cost=3
> Brought up 2 CPUs
> PCI: setting up Xen PCI frontend stub
> ACPI: Interpreter disabled.
> Linux Plug and Play Support v0.97 (c) Adam Belay
> pnp: PnP ACPI: disabled
> xen_mem: Initialising balloon driver.
> usbcore: registered new driver usbfs
> usbcore: registered new driver hub
> PCI: System does not support PCI
> PCI: System does not support PCI
> NetLabel: Initializing
> NetLabel:  domain hash size = 128
> NetLabel:  protocols = UNLABELED CIPSOv4
> NetLabel:  unlabeled traffic allowed by default
> NET: Registered protocol family 2
> IP route cache hash table entries: 4096 (order: 2, 16384 bytes)
> TCP established hash table entries: 16384 (order: 5, 131072 bytes)
> TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
> TCP: Hash tables configured (established 16384 bind 8192)
> TCP reno registered
> IA-32 Microcode Update Driver: v1.14-xen <tigran@xxxxxxxxxxx>
> audit: initializing netlink socket (disabled)
> audit(1177532459.535:1): initialized
> VFS: Disk quotas dquot_6.5.1
> Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
> SELinux:  Registering netfilter hooks
> Initializing Cryptographic API
> ksign: Installing public key data
> Loading keyring
> - Added public key AA30C74FDDC4E19F
> - User ID: CentOS (Kernel Module GPG key)
> io scheduler noop registered
> io scheduler anticipatory registered
> io scheduler deadline registered
> io scheduler cfq registered (default)
> pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> rtc: IRQ 8 is not free.
> Non-volatile memory driver v1.2
> Linux agpgart interface v0.101 (c) Dave Jones
> RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize
> Xen virtual console successfully installed as xvc0
> Event-channel device installed.
> Console: switching to colour frame buffer device 100x37
> input: Xen Virtual Keyboard/Mouse as /class/input/input0
> Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
> ide: Assuming 50MHz system bus speed for PIO modes; override with idebus=xx
> Probing IDE interface ide0...
> Probing IDE interface ide1...
> Probing IDE interface ide2...
> Probing IDE interface ide3...
> Probing IDE interface ide4...
> Probing IDE interface ide5...
> ide-floppy driver 0.99.newide
> usbcore: registered new driver hiddev
> usbcore: registered new driver usbhid
> drivers/usb/input/hid-core.c: v2.6:USB HID core driver
> PNP: No PS/2 controller found. Probing ports directly.
> i8042.c: No controller found.
> mice: PS/2 mouse device common for all mice
> md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
> md: bitmap version 4.39
> TCP bic registered
> Initializing IPsec netlink socket
> NET: Registered protocol family 1
> NET: Registered protocol family 17
> Using IPI No-Shortcut mode
> XENBUS: Device with no driver: device/vif/0
> XENBUS: Device with no driver: device/vbd/51712
> XENBUS: Device with no driver: device/vbd/51729
> XENBUS: Device with no driver: device/vbd/51730
> XENBUS: Device with no driver: device/vbd/51731
> XENBUS: Device with no driver: device/vbd/51732
> Freeing unused kernel memory: 172k freed
> Write protecting the kernel read-only data: 355k
> USB Universal Host Controller Interface driver v3.0
> ohci_hcd: 2005 April 22 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI)
> Registering block device major 202
>  xvda: xvda1 xvda2
> kjournald starting.  Commit interval 5 seconds
> EXT3-fs: mounted filesystem with ordered data mode.
> security:  3 users, 6 roles, 1587 types, 172 bools, 1 sens, 1024 cats
> security:  59 classes, 49762 rules
> SELinux:  Completing initialization.
> SELinux:  Setting up existing superblocks.
> SELinux: initialized (dev xvda2, type ext3), uses xattr
> SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
> SELinux: initialized (dev debugfs, type debugfs), uses genfs_contexts
> SELinux: initialized (dev selinuxfs, type selinuxfs), uses genfs_contexts
> SELinux: initialized (dev mqueue, type mqueue), uses transition SIDs
> SELinux: initialized (dev devpts, type devpts), uses transition SIDs
> SELinux: initialized (dev eventpollfs, type eventpollfs), uses task SIDs
> SELinux: initialized (dev inotifyfs, type inotifyfs), uses genfs_contexts
> SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
> SELinux: initialized (dev futexfs, type futexfs), uses genfs_contexts
> SELinux: initialized (dev pipefs, type pipefs), uses task SIDs
> SELinux: initialized (dev sockfs, type sockfs), uses task SIDs
> SELinux: initialized (dev cpuset, type cpuset), not configured for labeling
> SELinux: initialized (dev proc, type proc), uses genfs_contexts
> SELinux: initialized (dev bdev, type bdev), uses genfs_contexts
> SELinux: initialized (dev rootfs, type rootfs), uses genfs_contexts
> SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts
> audit(1177532465.491:2): policy loaded auid=4294967295
> SELinux: initialized (dev usbfs, type usbfs), uses genfs_contexts
> input: PC Speaker as /class/input/input1
> Floppy drive(s): fd0 is unknown type 15 (usb?), fd1 is unknown type 15
> (usb?) Failed to obtain physical IRQ 6
> floppy0: no floppy controllers found
> lp: driver loaded but no devices found
> md: Autodetecting RAID arrays.
> md: autorun ...
> md: ... autorun DONE.
> device-mapper: ioctl: 4.11.0-ioctl (2006-09-14) initialised:
> dm-devel@xxxxxxxxxx
> EXT3 FS on xvda2, internal journal
> kjournald starting.  Commit interval 5 seconds
> EXT3 FS on xvda1, internal journal
> EXT3-fs: mounted filesystem with ordered data mode.
> SELinux: initialized (dev xvda1, type ext3), uses xattr
> kjournald starting.  Commit interval 5 seconds
> EXT3 FS on xvdb1, internal journal
> EXT3-fs: mounted filesystem with ordered data mode.
> SELinux: initialized (dev xvdb1, type ext3), uses xattr
> kjournald starting.  Commit interval 5 seconds
> EXT3 FS on xvdb2, internal journal
> EXT3-fs: mounted filesystem with ordered data mode.
> SELinux: initialized (dev xvdb2, type ext3), uses xattr
> kjournald starting.  Commit interval 5 seconds
> EXT3 FS on xvdb3, internal journal
> EXT3-fs: mounted filesystem with ordered data mode.
> SELinux: initialized (dev xvdb3, type ext3), uses xattr
> SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
> Adding 1048568k swap on /dev/xvdb4.  Priority:-1 extents:1 across:1048568k
> SELinux: initialized (dev binfmt_misc, type binfmt_misc), uses
> genfs_contexts NET: Registered protocol family 10
> lo: Disabled Privacy Extensions
> IPv6 over IPv4 tunneling driver
> process `sysctl' is using deprecated sysctl (syscall)
> net.ipv6.neigh.lo.retrans_time; Use net.ipv6.neigh.lo.retrans_time_ms
> instead.
> netfront: Initialising virtual ethernet driver.
> netfront: device eth0 has flipping receive path.
> SELinux: initialized (dev rpc_pipefs, type rpc_pipefs), uses genfs_contexts
> Bluetooth: Core ver 2.10
> NET: Registered protocol family 31
> Bluetooth: HCI device and connection manager initialized
> Bluetooth: HCI socket layer initialized
> Bluetooth: L2CAP ver 2.8
> Bluetooth: L2CAP socket layer initialized
> Bluetooth: RFCOMM socket layer initialized
> Bluetooth: RFCOMM TTY layer initialized
> Bluetooth: RFCOMM ver 1.8
> Bluetooth: HIDP (Human Interface Emulation) ver 1.1
> SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
> SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
> SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
> eth0: no IPv6 routers present
>
> If additional information is needed, please ask.
>
> Thank you,
>
> Geert Janssens

-- 
Kobalt W.I.T.
Web & Information Technology
Brusselsesteenweg 152
1850 Grimbergen

Tel  : +32 479 339 655
Email: info@xxxxxxxxxxxx

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users