Re: [Xen-devel] RFC: Xen VMDq patch for ixgbe

[Steven Smith <steven.smith@xxxxxxxxxxxxx>]

> Maybe, you are not running in VMDq mode.
> Do you see the following message when you run "dmesg" in dom0?
> "Netchannel2 using vmq mode for guest n"
> 
> I just noticed the tip of the current netchannel2 does not include the 
> modifications of a previous changeset. I suspect this may have been lost in a 
> merge with latest xen code. Without this missing code MSI is disabled by Xen 
> which causes VMDq to be disabled.
> I needed to appply this patch (attached) maually to enable MSI and be able to 
> run in VMDq mode.
Oops, sorry about that.  I've (re-)applied the patch to tip and pushed
it out.

Steven.



> > -----Original Message-----
> > From: Williams, Mitch A [mailto:mitch.a.williams@xxxxxxxxx] 
> > Sent: Wednesday, January 21, 2009 4:21 PM
> > To: Santos, Jose Renato G; xen-devel@xxxxxxxxxxxxxxxxxxx
> > Cc: steven.smith@xxxxxxxxxxxxx; Ronciak, John
> > Subject: RE: [Xen-devel] RFC: Xen VMDq patch for ixgbe
> > 
> > I'll take a look.  I've been running this code, so I 
> > (obviously) didn't see any failure.
> > 
> > They've just updated ixgbe on sourceforge, so I'm spinning a 
> > new patch that will work with that driver.  I'll probably get 
> > that out tomorrow.  I'll double-check to make sure we're not 
> > allocating memory before the queue is activated.
> > 
> > 
> > -Mitch 
> > 
> > >-----Original Message-----
> > >From: Santos, Jose Renato G [mailto:joserenato.santos@xxxxxx]
> > >Sent: Wednesday, January 21, 2009 4:17 PM
> > >To: Williams, Mitch A; xen-devel@xxxxxxxxxxxxxxxxxxx
> > >Cc: steven.smith@xxxxxxxxxxxxx; Ronciak, John
> > >Subject: RE: [Xen-devel] RFC: Xen VMDq patch for ixgbe
> > >
> > > 
> > >Mitch,
> > >
> > >It seems this last patch is not working properly I get a 
> > kernel panic 
> > >when the driver module loads (see panic message below).
> > >Apparently the driver is trying to allocate guest memory
> > >(vmq_alloc_skb()) before the queue is allocated to a guest 
> > (there is no 
> > >guest running when the ixgbe module is loaded).
> > >This is probably something easy to fix
> > >
> > >I am able to use the previous version of the patch (version
> > >1.3.31.3 attached) with no problem. Not sure what changed 
> > from version 
> > >1.3.31.3 to this new version 1.3.47. I am also attaching the buggy 
> > >patch for  version 1.3.47 that I am using so you can verify if I am 
> > >using the right patch ...
> > >
> > >Could you please take a look at this?
> > >
> > >Thanks
> > >
> > >Renato
> > >
> > >=====================================
> > >
> > >Unable to handle kernel paging request at 0000000000007800 RIP: 
> > > [<ffffffff804600a4>] vmq_alloc_skb+0x64/0x1f0 PGD 79807067 PUD 
> > >79710067 PMD 0
> > >Oops: 0000 [1] SMP
> > >CPU 0
> > >Modules linked in: video thermal fan button battery 
> > asus_acpi ac ixgbe
> > >Pid: 0, comm: swapper Not tainted 2.6.18.8-xen0 #73
> > >RIP: e030:[<ffffffff804600a4>]  [<ffffffff804600a4>] 
> > >vmq_alloc_skb+0x64/0x1f0
> > >RSP: e02b:ffffffff80793ca0  EFLAGS: 00010206
> > >RAX: ffff88007acc5dd8 RBX: 0000000000000000 RCX: 0000000000080000
> > >RDX: ffff88007b5d7740 RSI: 0000000000000001 RDI: ffff88007a450000
> > >RBP: ffffffff80793cd0 R08: 0000000000000001 R09: ffff88007a450c80
> > >R10: 000000000000003f R11: 000000000000012c R12: 0000000000007800
> > >R13: 0000000000000000 R14: 0000000000000500 R15: 00000000000005f4
> > >FS:  00002ba4a6b1eda0(0000) GS:ffffffff80737000(0000) 
> > >knlGS:0000000000000000
> > >CS:  e033 DS: 0000 ES: 0000
> > >Process swapper (pid: 0, threadinfo ffffffff8074c000, task
> > >ffffffff8064d440)
> > >Stack:  ffff88007a450000 ffffc200118f0000 0000000000000000 
> > >0000000000000000  ffff88007b279070 ffff88007a450500 ffffffff80793d30 
> > >ffffffff88000d2f  000003ff80793d50 ffff880075df0000 ffff88007a450000 
> > >ffff88007fe90800 Call Trace:
> > > <IRQ> [<ffffffff88000d2f>] :ixgbe:ixgbe_alloc_rx_buffers+0x15f/0x2e0
> > > [<ffffffff8800320b>] :ixgbe:ixgbe_clean_rx_irq+0x9eb/0xaa0
> > > [<ffffffff8800798e>] :ixgbe:ixgbe_clean_rxonly_many+0xbe/0x210
> > > [<ffffffff8800de3e>] :ixgbe:__kc_adapter_clean+0x2e/0x50
> > > [<ffffffff8052eba4>] net_rx_action+0xc4/0x1c0  [<ffffffff80239eec>] 
> > >__do_softirq+0x9c/0x140  [<ffffffff8020b604>] 
> > call_softirq+0x1c/0x28  
> > >[<ffffffff8020d7cc>] do_softirq+0x6c/0x100  [<ffffffff80239d48>] 
> > >irq_exit+0x48/0x50  [<ffffffff80430b92>] 
> > evtchn_do_upcall+0x232/0x250  
> > >[<ffffffff8020b13a>] do_hypervisor_callback+0x1e/0x2c  <EOI> 
> > >[<ffffffff802063aa>] hypercall_page+0x3aa/0x1000  
> > [<ffffffff802063aa>] 
> > >hypercall_page+0x3aa/0x1000  [<ffffffff8020eed2>] 
> > >raw_safe_halt+0xc2/0xf0  [<ffffffff80209b15>] xen_idle+0x75/0x90  
> > >[<ffffffff8020926a>] cpu_idle+0xba/0xe0  [<ffffffff802073b6>] 
> > >rest_init+0x26/0x30  [<ffffffff807568f5>] start_kernel+0x265/0x270  
> > >[<ffffffff8075623d>] _sinittext+0x23d/0x250
> > >
> > >
> > >Code: 4d 39 a6 00 73 00 00 4d 8d ae e8 72 00 00 0f 84 58 01 
> > 00 00 RIP  
> > >[<ffffffff804600a4>] vmq_alloc_skb+0x64/0x1f0  RSP <ffffffff80793ca0>
> > >CR2: 0000000000007800
> > > <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
> > > (XEN) Domain 0 crashed: 'noreboot' set - not rebooting.
> > >
> > >
> > >> -----Original Message-----
> > >> From: Williams, Mitch A [mailto:mitch.a.williams@xxxxxxxxx]
> > >> Sent: Friday, January 16, 2009 3:57 PM
> > >> To: Williams, Mitch A; xen-devel@xxxxxxxxxxxxxxxxxxx
> > >> Cc: steven.smith@xxxxxxxxxxxxx; Ronciak, John; Santos, 
> > Jose Renato G
> > >> Subject: RE: [Xen-devel] RFC: Xen VMDq patch for ixgbe
> > >> 
> > >> Whoops!  Attached the wrong patch file.  Please use this one.
> > >> 
> > >> Sorry for the confusion.
> > >> -Mitch
> > >> 
> > >> >-----Original Message-----
> > >> >From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> > >> >[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Mitch 
> > >> >Williams
> > >> >Sent: Friday, January 16, 2009 3:06 PM
> > >> >To: xen-devel@xxxxxxxxxxxxxxxxxxx
> > >> >Cc: steven.smith@xxxxxxxxxxxxx; Ronciak, John;
> > >> joserenato.santos@xxxxxx
> > >> >Subject: [Xen-devel] RFC: Xen VMDq patch for ixgbe
> > >> >
> > >> >Attached is a patch to enable Xen VMDq (AKA Netchannel2
> > >vmq) for the
> > >> >ixgbe driver.  This is intended for testing and development
> > >purposes
> > >> >and should not be considered to be production-quality code.
> > >> >
> > >> >Please note that this does NOT apply to the Xen Linux kernel; it 
> > >> >applies to the ixgbe 1.3.47 release, available from 
> > >> >http://sourceforge.net/projects/e1000/.  You'll obviously
> > >> also need an
> > >> >Intel 82598-based 10 Gigabit network card and some sort of link 
> > >> >partner.  This will build against the current Netchannel2 source 
> > >> >available from http:xenbits.xen.org/ext/netchannel2.
> > >You'll need to
> > >> >enable "Net channel 2 support for multi-queue devices" in
> > >> your kernel
> > >> >config.
> > >> >
> > >> >To enable VMDq functionality, load the driver with the 
> > command-line 
> > >> >parameter VMDQ=<num queues>, as in:
> > >> >
> > >> >$ insmod ~/ixgbe-1.3.47/src/ixgbe.ko VMDQ=8
> > >> >
> > >> >You can then set up PV domains to use the device by
> > >> modifying your VM
> > >> >configuration file from
> > >> >     vif = [ '<whatever>' ]
> > >> >to
> > >> >     vif2 = [ 'pdev=<netdev>' ]
> > >> >where <netdev> is the interface name for your 82598 board,
> > >> e.g peth0 in
> > >> >dom0.
> > >> >
> > >> >Known issues (at least, known by me):
> > >> >1) Must manually attach bridge device after starting domU vm.
> > >> >Netchannel2 backend devices show up as ethNN, not vifN.M, so the 
> > >> >scripts don't automatically attach the interface.  Once your
> > >> VM starts,
> > >> >do ifconfig -a to see which new interface got added.  Then
> > >> use "brctl
> > >> >addif" to add this new interface the the bridge.
> > >> >2) No broadcast replication.  This is a big one.  Incoming
> > >> broadcasts
> > >> >will ONLY go to dom0.  This means that your VMs can send ARP
> > >> requests
> > >> >and initiate IP sessions to outside machines, but outside 
> > machines 
> > >> >cannot initiate connections because the ARP requests don't
> > >go to the
> > >> >domU VMs.
> > >> >3) No loopback.  VMs cannot communicate with other VMs (including
> > >> >dom0) on the same machine.
> > >> >
> > >> >Once I get this out, I'll start working on a proper 
> > backport of the 
> > >> >driver into the Xen kernel (2.6.18.8) tree.  I'll remove as
> > >> much of the
> > >> >compatibility cruft as is prudent and properly integrate it into
> > >> >the Kbuild stuff.   When that's done, I'll send a complete 
> > >> patchset to
> > >> >this list, including signed-off-by lines which can then be
> > >> checked in
> > >> >to Mercurial.
> > >> >
> > >> >Please review and comment, and if possible test.
> > >> >
> > >> >Thanks,
> > >> >Mitch
> > >> >
> > >
> >

Attachment: signature.asc
Description: Digital signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel