[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] SR-IOV problems - HVM cannot access network




On 03/01/2011 02:41 PM, Rose, Gregory V wrote:
-----Original Message-----
From: David White [mailto:dwhite@xxxxxxxxxxxxx]
Sent: Tuesday, March 01, 2011 2:05 PM
To: Rose, Gregory V
Cc:xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] SR-IOV problems - HVM cannot access network

  [snip}

The guest driver does not seem to be the source of this difference,
since these qemu logs are logged before HVM grub menu appears (and hence
before guest kernel is loaded)

Why would a physical function invoke MSI-X but not a virtual function?
A lot of reasons actually :-).  SR-IOV is a complete platform solution and
feature.  It requires full support from all PCIe devices from the root complex
to the endpoint device as well as proper BIOS programming and OS support.

I don't know that Ubuntu actually supports SR-IOV and I don't know which
platform you're using and whether it supports SR-IOV.

Is it something that pciback is doing?

When I bind a PF to pciback, the xen-pciback driver shows:
[ 1390.090693] pciback 0000:04:00.1: seizing device
[ 1390.095417] xen_allocate_pirq: returning irq 17 for gsi 17
[ 1390.101021] Already setup the GSI :17
[ 1390.104717] pciback 0000:04:00.1: PCI INT B ->  GSI 17 (level, low) ->
IRQ 17
[ 1390.111807] pciback 0000:04:00.1: PCI INT B disabled

but when I bind a VF to pciback, the driver shows:
[ 1439.411763] pciback 0000:04:10.0: seizing device
[ 1439.416462] pciback 0000:04:10.0: enabling device (0000 ->  0002)

(note no INT or GSI messages)

could pciback be the source of the HVM INTx problem?
Well it's certainly possible but another good possibility is that the machine
doesn't actually implement the SR-IOV feature correctly in the BIOS.  What
machine are you using?  Is there a BIOS setting to enable SR-IOV and if so are
you sure you've enabled it?

Also, would it be possible to get a register dump of the VF device from both
the Dom0 host domain and the DomU guest domain using the Intel ethregs
utility?  You can get that from source forge.

- Greg


Thanks for your help so far.

Ubuntu Maverick is only running in the guest HVM domains.

I have two boards I am testing with, both with Intel 5520 chipset.  I'm using
two 82576 nics from two different vendors, one attached to either board.  Both
systems have AMI bios.  One system (call it systemA) has bios options for both
VT-d and SR-IOV, and both are enabled, as is I/OAT.  The other system (systemB)
has bios options for VT-d and "Coherency support" (which seemed to help me
enumerate the VFs), and both are enabled (no mention of I/OAT).

SystemA runs Debian Squeeze and the xen kernel, vmm and tools from that distro.
SystemB runs Fedora14 with the well-known "jeremy" kernel
(http://git.kernel.org/?p=linux/kernel/git/jeremy/xen.git;a=shortlog;h=xen/stable-2.6.32.x),
and xen-4.0.1 hypervisor built from source (unaltered).

Both systems are experiencing the same behavior.

attached are dom0 and hvm ethregs output for the VF using '-o -m'
flags.  This one was taken from the Debian host ("systemA")

-dave

Attachment: ethregs.VF.dom0
Description: Text document

Attachment: ethregs.VF.hvm
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.