[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] PCI Passthrough not working



Hi,

I have installed CentOS 5 with Xen on a new Dell SC440.

I have hidden a network interface in dom0 to pass it on to domU by adding 
these lines in /etc/modprobe.conf:
options pciback hide=(04:00.0)
install tg3 /sbin/modprobe 
          pciback ; /sbin/modprobe --first-time --ignore-install tg3
CentOS builds pciback as a modules, so I could not add pciback.hide in the 
grub configuration. I found the modprobe.conf setup in the XenSource wiki.

I am not sure if this part works fine: the nic is still visible via lspci in 
dom0, although it is mentioned as disabled.
# lspci -vb
...
04:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5754 Gigabit 
Ethernet PCI Express (rev 02)
        Subsystem: Dell Unknown device 01df
        Flags: fast devsel, IRQ 10
        Memory at 00000000efbf0000 (64-bit, non-prefetchable) [disabled]
        Capabilities: [48] Power Management version 3
        Capabilities: [50] Vital Product Data
        Capabilities: [58] Vendor Specific Information
        Capabilities: [e8] Message Signalled Interrupts: 64bit+ Queue=0/0 
Enable-
        Capabilities: [d0] Express Endpoint IRQ 0
...

Also in dom0:
# ls -l /sys/bus/pci/drivers/pciback
total 0
lrwxrwxrwx 1 root root    0 Apr 25 13:06 
0000:04:00.0 -> ../../../../devices/pci0000:00/0000:00:1c.5/0000:04:00.0
--w------- 1 root root 4096 Apr 25 13:06 bind
lrwxrwxrwx 1 root root    0 Apr 25 13:06 module -> ../../../../module/pciback
--w------- 1 root root 4096 Apr 25 13:06 new_id
--w------- 1 root root 4096 Apr 25 13:06 new_slot
-rw------- 1 root root 4096 Apr 25 13:06 permissive
-rw------- 1 root root 4096 Apr 25 13:06 quirks
--w------- 1 root root 4096 Apr 25 13:06 remove_slot
-r-------- 1 root root 4096 Apr 25 13:06 slots
--w------- 1 root root 4096 Apr 25 13:06 unbind

seems to suggest that the nic is really bound to pciback. It's not 
in /sys/bus/pci/drivers/tg3 anyway.

I then start a domU with the following configuration:
memory = 256
name = "CentOSFileserverLVM"
vcpus = 2
vif = [ 'mac=aa:cc:00:00:00:02, bridge=xenbr0' ]
pci = [ '04:00.0' ]
disk = [ 'phy:base/fwroot,xvda,w',
         'phy:base/fwhome,xvdb1,w',
         'phy:base/fwvar,xvdb2,w',
         'phy:base/fwbackup,xvdb3,w',
         'phy:base/fwswap,xvdb4,w' ]
bootloader="/usr/bin/pygrub"
bootargs="/dev/base/fwroot"
root = "/dev/xvda2"
vfb = [ 'type=vnc,
         vnclisten=0.0.0.0,
         vncunused=1,
         display=localhost:12.0' ]

This boots fine, but I don't find the nic in domU. lspci returns absolutely 
nothing. From reading messages on different forums (like this one: 
http://www.vdr-portal.de/board/thread.php?threadid=60330), I gather the boot 
messages for domU should contain some entries like 
> pcifront pci-0: Installing PCI frontend
> pcifront pci-0: Creating PCI Frontend Bus 0000:03
and 
> PCI: Enabling device 0000:03:04.0 (0000 -> 0003)
or similar.

It seems this is not the case for my setup. I have no idea what goes wrong and 
why. Did anybody else get stuck at this point, and managed to fix it ?

Some additional info:
dom0# uname -a
Linux localhost.localdomain 2.6.18-8.1.1.el5xen #1 SMP Mon Apr 9 11:46:46 EDT 
2007 i686 i686 i386 GNU/Linux

dom0# rpm -q xen
xen-3.0.3-25.0.3.el5

domU# uname -a
Linux localhost.localdomain 2.6.18-8.1.1.el5xen #1 SMP Mon Apr 9 11:46:46 EDT 
2007 i686 i686 i386 GNU/Linux

domU# dmesg  (I removed the SELinux audit entries for readability)
# dmesg
Linux version 2.6.18-8.1.1.el5xen (mockbuild@xxxxxxxxxxxxxxxxxxx) (gcc version 
4.1.1 20070105 (Red Hat 4.1.1-52)) #1 SMP Mon Apr 9 11:46:46 EDT 2007
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 0000000010800000 (usable)
0MB HIGHMEM available.
264MB LOWMEM available.
NX (Execute Disable) protection: active
On node 0 totalpages: 67584
  DMA zone: 67584 pages, LIFO batch:15
ACPI in unprivileged domain disabled
Built 1 zonelists.  Total pages: 67584
Kernel command line:  root=/dev/xvda2 ro root=LABEL=/ rhgb quiet
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
CPU 0 irqstacks, hard=c071b000 soft=c06fb000
PID hash table entries: 2048 (order: 11, 8192 bytes)
Xen reported: 1862.047 MHz processor.
Console: colour dummy device 80x25
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Software IO TLB disabled
vmalloc area: d1000000-f4ffe000, maxmem 2d7fe000
Memory: 251580k/270336k available (2017k kernel code, 10252k reserved, 824k 
data, 172k init, 0k highmem)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
Calibrating delay using timer specific routine.. 4657.11 BogoMIPS 
(lpj=9314232)
Security Framework v1.0.0 initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
selinux_register_security:  Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 512
CPU: After generic identify, caps: bfebc3f1 20100000 00000000 00000000 
0000e3bd 00000000 00000001
CPU: After vendor identify, caps: bfebc3f1 20100000 00000000 00000000 0000e3bd 
00000000 00000001
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 2048K
CPU: After all inits, caps: bfebc3f1 20100000 00000000 00000940 0000e3bd 
00000000 00000001
Checking 'hlt' instruction... OK.
SMP alternatives: switching to UP code
CPU 1 irqstacks, hard=c071c000 soft=c06fc000
Brought up 1 CPUs
sizeof(vma)=88 bytes
sizeof(page)=32 bytes
sizeof(inode)=340 bytes
sizeof(dentry)=136 bytes
sizeof(ext3inode)=492 bytes
sizeof(buffer_head)=52 bytes
sizeof(skbuff)=172 bytes
checking if image is initramfs... it is
Freeing initrd memory: 3071k freed
Grant table initialized
NET: Registered protocol family 16
ACPI Exception (utmutex-0262): AE_BAD_PARAMETER, Thread C0ABCAA0 could not 
acquire Mutex [2] [20060707]
SMP alternatives: switching to SMP code
Initializing CPU#1
migration_cost=3
Brought up 2 CPUs
PCI: setting up Xen PCI frontend stub
ACPI: Interpreter disabled.
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI: disabled
xen_mem: Initialising balloon driver.
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: System does not support PCI
PCI: System does not support PCI
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
NET: Registered protocol family 2
IP route cache hash table entries: 4096 (order: 2, 16384 bytes)
TCP established hash table entries: 16384 (order: 5, 131072 bytes)
TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
TCP: Hash tables configured (established 16384 bind 8192)
TCP reno registered
IA-32 Microcode Update Driver: v1.14-xen <tigran@xxxxxxxxxxx>
audit: initializing netlink socket (disabled)
audit(1177532459.535:1): initialized
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
SELinux:  Registering netfilter hooks
Initializing Cryptographic API
ksign: Installing public key data
Loading keyring
- Added public key AA30C74FDDC4E19F
- User ID: CentOS (Kernel Module GPG key)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
rtc: IRQ 8 is not free.
Non-volatile memory driver v1.2
Linux agpgart interface v0.101 (c) Dave Jones
RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize
Xen virtual console successfully installed as xvc0
Event-channel device installed.
Console: switching to colour frame buffer device 100x37
input: Xen Virtual Keyboard/Mouse as /class/input/input0
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 50MHz system bus speed for PIO modes; override with idebus=xx
Probing IDE interface ide0...
Probing IDE interface ide1...
Probing IDE interface ide2...
Probing IDE interface ide3...
Probing IDE interface ide4...
Probing IDE interface ide5...
ide-floppy driver 0.99.newide
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
PNP: No PS/2 controller found. Probing ports directly.
i8042.c: No controller found.
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
TCP bic registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
Using IPI No-Shortcut mode
XENBUS: Device with no driver: device/vif/0
XENBUS: Device with no driver: device/vbd/51712
XENBUS: Device with no driver: device/vbd/51729
XENBUS: Device with no driver: device/vbd/51730
XENBUS: Device with no driver: device/vbd/51731
XENBUS: Device with no driver: device/vbd/51732
Freeing unused kernel memory: 172k freed
Write protecting the kernel read-only data: 355k
USB Universal Host Controller Interface driver v3.0
ohci_hcd: 2005 April 22 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI)
Registering block device major 202
 xvda: xvda1 xvda2
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
security:  3 users, 6 roles, 1587 types, 172 bools, 1 sens, 1024 cats
security:  59 classes, 49762 rules
SELinux:  Completing initialization.
SELinux:  Setting up existing superblocks.
SELinux: initialized (dev xvda2, type ext3), uses xattr
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev debugfs, type debugfs), uses genfs_contexts
SELinux: initialized (dev selinuxfs, type selinuxfs), uses genfs_contexts
SELinux: initialized (dev mqueue, type mqueue), uses transition SIDs
SELinux: initialized (dev devpts, type devpts), uses transition SIDs
SELinux: initialized (dev eventpollfs, type eventpollfs), uses task SIDs
SELinux: initialized (dev inotifyfs, type inotifyfs), uses genfs_contexts
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev futexfs, type futexfs), uses genfs_contexts
SELinux: initialized (dev pipefs, type pipefs), uses task SIDs
SELinux: initialized (dev sockfs, type sockfs), uses task SIDs
SELinux: initialized (dev cpuset, type cpuset), not configured for labeling
SELinux: initialized (dev proc, type proc), uses genfs_contexts
SELinux: initialized (dev bdev, type bdev), uses genfs_contexts
SELinux: initialized (dev rootfs, type rootfs), uses genfs_contexts
SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts
audit(1177532465.491:2): policy loaded auid=4294967295
SELinux: initialized (dev usbfs, type usbfs), uses genfs_contexts
input: PC Speaker as /class/input/input1
Floppy drive(s): fd0 is unknown type 15 (usb?), fd1 is unknown type 15 (usb?)
Failed to obtain physical IRQ 6
floppy0: no floppy controllers found
lp: driver loaded but no devices found
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
device-mapper: ioctl: 4.11.0-ioctl (2006-09-14) initialised: 
dm-devel@xxxxxxxxxx
EXT3 FS on xvda2, internal journal
kjournald starting.  Commit interval 5 seconds
EXT3 FS on xvda1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: initialized (dev xvda1, type ext3), uses xattr
kjournald starting.  Commit interval 5 seconds
EXT3 FS on xvdb1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: initialized (dev xvdb1, type ext3), uses xattr
kjournald starting.  Commit interval 5 seconds
EXT3 FS on xvdb2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: initialized (dev xvdb2, type ext3), uses xattr
kjournald starting.  Commit interval 5 seconds
EXT3 FS on xvdb3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: initialized (dev xvdb3, type ext3), uses xattr
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
Adding 1048568k swap on /dev/xvdb4.  Priority:-1 extents:1 across:1048568k
SELinux: initialized (dev binfmt_misc, type binfmt_misc), uses genfs_contexts
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
IPv6 over IPv4 tunneling driver
process `sysctl' is using deprecated sysctl (syscall) 
net.ipv6.neigh.lo.retrans_time; Use net.ipv6.neigh.lo.retrans_time_ms 
instead.
netfront: Initialising virtual ethernet driver.
netfront: device eth0 has flipping receive path.
SELinux: initialized (dev rpc_pipefs, type rpc_pipefs), uses genfs_contexts
Bluetooth: Core ver 2.10
NET: Registered protocol family 31
Bluetooth: HCI device and connection manager initialized
Bluetooth: HCI socket layer initialized
Bluetooth: L2CAP ver 2.8
Bluetooth: L2CAP socket layer initialized
Bluetooth: RFCOMM socket layer initialized
Bluetooth: RFCOMM TTY layer initialized
Bluetooth: RFCOMM ver 1.8
Bluetooth: HIDP (Human Interface Emulation) ver 1.1
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
eth0: no IPv6 routers present

If additional information is needed, please ask.

Thank you,

Geert Janssens
-- 
Kobalt W.I.T.
Web & Information Technology
Brusselsesteenweg 152
1850 Grimbergen

Tel  : +32 479 339 655
Email: info@xxxxxxxxxxxx

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.