[Xen-devel] VTd not showing PCI device in VM

["Rafael Ayala" <rafael_ayala@xxxxxxxxxxxxxx>]

Moving my question to the developers list per Todd's suggestion. 

Attached are the console trace and the xend log.  For the traces I rebooted the server and started the trace.  I let Xen boot, started my VM, shut the VM down.  I realized I had a typo in my pci command, so you see a second VM start and then shutdown.  The second VM started should hopefully have the details to figure out my problem since the first VM never was told of the pci device to grab.  Thanks for the help.

-----------------------------------------
Rafael E. Ayala
Senior Software Engineer
Xyratex International, Inc.

-----Original Message-----
From: Todd Deshane [mailto:deshantm@xxxxxxxxx]
Sent: Wednesday, December 10, 2008 6:18 PM
To: Rafael Ayala
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-users] VTd not showing PCI device in VM

On Wed, Dec 10, 2008 at 6:00 PM, Rafael Ayala <rafael_ayala@xxxxxxxxxxxxxx> wrote:
> I'm having trouble actually seeing a PCI device in my VMs.  I've
> resolved several of my issues using previous posts and using the VTdHowTo wiki page.
> I have both VT and VTd BIOS options enabled.  I have pciback hiding
> the devices and xm can list and assign the devices to VMs.  I don't
> see the PCI devices in my VMs though.  I have tried a windows and a
> Linux vm, without success. I also made sure I removed the drivers for
> the devices to make sure dom0 was not using them.
>
> Grub command:
> title Xen 3.3 (2.6.18.8)
>     root (hd0,1)
>     kernel /boot/xen-3.3.0.gz vtd=1 iommu=1
>     module /boot/vmlinuz-2.6.18.8-xen
> root=/dev/disk/by-id/scsi-SATA_ST3250310NS_9SF0LZAP-part2 vga=0x317
> resume=/dev/sda1 splash=silent showopts
> pciback.hide=(01:00.0)(01:00.1)(06:00.0)
>
>     module /boot/initrd-2.6.18.8-xen
>
> The PCI devices I'm trying to assign:
> # lspci
> 01:00.0 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre
> Channel Host Adapter (rev 03)
> 01:00.1 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre
> Channel Host Adapter (rev 03) 06:00.0 SCSI storage controller: LSI
> Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
>
> Xen says I can assign them
> # xm pci-list-assignable-devices
> 0000:06:00.0
> 0000:01:00.0 0000:01:00.1
>
> Create my VM:
> # xm create w2k3.vxm pci=01:00.0 pci=01:00.1 Using config file
> "./w2k3.vxm".
> Started domain w2k3
>
> I can confirm Xen thinks I have assigned the devices # xm pci-list
> w2k3
> domain   bus   slot   func
> 0x0  0x01   0x00    0x0
> 0x0  0x01   0x00    0x1
>
> # xm create rhel5.vxm.vxm pci=01:00.0 pci=01:00.1 Using config file
> "./rhel5.vxm".
> Started domain rhel5
>
> # xm pci-list rhel5
> domain   bus   slot   func
> 0x0  0x06   0x00    0x0
>
> Again, neither windows (device manager) nor Linux (lspci) VMs can see
> the assigned devices.  Below is "xm dmesg" showing VTd is enabled. 
> Another curious fact is that I can assign the same devices to 2 VMs. 
> I would have expected an error, but perhaps it is allowed to share
> devices.  Anyone have any thoughts on what I'm missing?
>

From the looks of it, you have done your homework and it looks like it *should* work.

I would recommend trying to hook up a serial console to the Xen box and getting good debug info and sending it to xen-devel using the latest xen-unstable pull.

here is a serial console example:

kernel          /boot/xen-3.4-unstable.gz loglvl=all guest_loglvl=all
com1=115200,8n1 console=com1
module          /boot/vmlinuz-2.6.18.8-xen root=/dev/sda1 ro
console=hvc0 earlyprintk=xen xencons=ttyS0,115200

You will of course want to add in the iommu stuff too.

Hope that helps.

Cheers,
Todd

--
Todd Deshane
http://todddeshane.net
http://runningxen.com

______________________________________________________________________
This email may contain privileged or confidential information, which should only be used for the purpose for which it was sent by Xyratex. No further rights or licenses are granted to use such information. If you are not the intended recipient of this message, please notify the sender by return and delete it. You may not use, copy, disclose or rely on the information contained in it.

Internet email is susceptible to data corruption, interception and unauthorised amendment for which Xyratex does not accept liability. While we have taken reasonable precautions to ensure that this email is free of viruses, Xyratex does not accept liability for the presence of any computer viruses in this email, nor for any losses caused as a result of viruses.

Xyratex Technology Limited (03134912), Registered in England & Wales, Registered Office, Langstone Road, Havant, Hampshire, PO9 1SA.

The Xyratex group of companies also includes, Xyratex Ltd, registered in Bermuda, Xyratex International Inc, registered in California, Xyratex (Malaysia) Sdn Bhd registered in Malaysia, Xyratex Technology (Wuxi) Co Ltd registered in The People's Republic of China and Xyratex Japan Limited registered in Japan.
______________________________________________________________________

Attachment: xm.log
Description: xm.log

Attachment: Capture.txt
Description: Capture.txt

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel