Re: [Xen-devel] Intel S3420GPLX VT-d VF question

["Michael A. Collins" <mike@xxxxxxxxxxx>]

On Wed, 8 Dec 2010 12:50:35 -0500, Konrad Rzeszutek Wilk 
<konrad.wilk@xxxxxxxxxx> wrote:
> > The kernel I am using is myoung's pre-built rpm from 
> > http://myoung.fedorapeople.org/dom0/x86_64/kernel-2.6.32.26-174.2.xendom0.fc12.x86_64.rpm.
> Sweet. That is quite up-to-date.
>
> > I am also using the 4.0.1 rpms from
> > http://pasik.reaktio.net/fedora/xen-4.0.1-6.fc14.src.rpm.
> Uhh, try a more modern version. The xen-unstable.hg - you will have
> to compile it yourself. Pasi's awesome PVOPS Wiki mentions the 
> appropiate
> steps and I think Boris's blog has details as well.
>
> > I am following the instructions from
> > http://wiki.xen.org/xenwiki/Fedora13Xen4Tutorial.
> >
> > The pci_pt_e820_access=on setting was taken from the following blog: 
> > http://blogs.oracle.com/wim/2010/10/playing_with_sr_iov_in_oracle.html.
> Oh gosh. That is with 2.6.18 and Xen 3.4.
>
> > I was under the impression that that option had to be set for the
> > VFs to be seen by the kernel.
> Well, they were workarounds for that verison of kernel. It might be
> that they are required in 2.6.32.. but lets assume that those
> are actually fixed.
>
> > I have not tried it without, but will do that tonight.  The
> > blacklist settings was referenced in the above article, so I stuck
> > it in.  I will try to load the igbvf tonight as well.
> Yeah, try it. If it works you should see suddenly seven extra NICs
> (ifconfig -a) in your box. If that works then we know the kernel
> driver is working.
> > I do not see the 7 Virtual Functions in lspci.
> Ah! Those "01:00.0 Ethernet controller: Intel Corporation 82576
> Gigabit Network Connection (rev 01)
> 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
> Connection (rev 01)
> 01:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function
> (rev 01)
> 01:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function
> (rev 01)
> 01:10.4 Ethernet controller: Intel Corporation 82576 Virtual Function
> (rev 01)
> 01:10.6 Ethernet controller: Intel Corporation 82576 Virtual Function
> (rev 01)
> 01:11.0 Ethernet controller: Intel Corporation 82576 Virtual Function
> (rev 01)
> 01:11.2 Ethernet controller: Intel Corporation 82576 Virtual Function
> (rev 01)
> 01:11.4 Ethernet controller: Intel Corporation 82576 Virtual Function
> (rev 01)"
>
> must show up. That is the proof that your max_ig.. parameter worked
> and the card
> is now able to do SR-IOV. You might have need to upgrade your lspci -
> I think I had
> it as "Unknown device" at some point.
>
> Anyhow, if you see those extra devices, _then_ you can blacklist the
> igvf module.
>
> > Thanks for the reply, I will try without the e820 option and loading
> > the igbvf module tonight if that works, I'll let you know.  If not,
> > I'll respond with the full xen dmesg output.
> Take your time.
Seems as the igb module was loaded in the initramfs and was not 
reloaded using the options specified in my igb.conf file under 
/etc/modprobe.d.  I rmmod igb and did a modprobe igb max_vfs=7 and 
suddenly all the Virtual Functions showed up.  So that was the problem 
all along.  Until I figure out how to edit the initramfs to load igb 
with max_vfs option, I'm putting the rmmod and modprobe in the rc.local 
file, that seems to work well and it gets executed before xend.  I am 
build xen-4.0-testing to replace the rpms I used.  Once all that is done 
and I test passing through one of the Virtual Functions to a vm, I'll 
report back.
Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel