Xen project Mailing List

On Wed, Feb 12, 2014 at 1:25 PM, Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx> wrote:

On Tue, Feb 11, 2014 at 08:04:04AM -0500, Mike Neiderhauser wrote:
> So I went ahead to tested the setup I am trying to achieve using xen. This
> setup basically requires two isolated machines that can be used for network
> testing. On the hvm mentioned above, this testing fails due to something I
> cannot wrap my head around. I believe it is still related to the PCI
> passthrough of a device and I believe it is related to the libxl error
> mentioned above. Can anyone shed some light on what is going on? Is it a
> driver issue? (Broadcom Corporation NetXtreme II BCM5716 Gigabit Ethernet)

That looks like this issue:

igb and bnx2: "NETDEV WATCHDOG: transmit queue timed out" when skb has huge linear buffer

(http://lkml.org/lkml/2014/1/30/358) ?

>
> [79464.816085] ------------[ cut here ]------------
> [79464.816093] WARNING: at
> /build/buildd/linux-lts-raring-3.8.0/net/sched/sch_generic.c:254
> dev_watchdog+0x262/0x270()
> [79464.816094] Hardware name: HVM domU
> [79464.816096] NETDEV WATCHDOG: eth1 (bnx2): transmit queue 1 timed out
> [79464.816096] Modules linked in: esp6(F) ah6(F) xfrm6_mode_tunnel(F)
> xfrm_user(F) xfrm4_tunnel(F) tunnel4(F) ipcomp(F) xfrm_ipcomp(F) authenc(F)
> esp4(F) ah4(F) xfrm4_mode_tunnel(F) deflate(F) zlib_deflate(F) ctr(F)
> twofish_generic(F) twofish_avx_x86_64(F) twofish_x86_64_3way(F)
> twofish_x86_64(F) twofish_common(F) camellia_generic(F)
> camellia_aesni_avx_x86_64(F) camellia_x86_64(F) serpent_avx_x86_64(F)
> serpent_sse2_x86_64(F) glue_helper(F) serpent_generic(F)
> blowfish_generic(F) blowfish_x86_64(F) blowfish_common(F)
> cast5_avx_x86_64(F) cast5_generic(F) cast_common(F) des_generic(F) xcbc(F)
> rmd160(F) crypto_null(F) af_key(F) xfrm_algo(F) 8021q(F) garp(F) stp(F)
> llc(F) ipmi_msghandler(F) autofs4(F) ghash_clmulni_intel(F) aesni_intel(F)
> ablk_helper(F) cryptd(F) lrw(F) aes_x86_64(F) xts(F) gf128mul(F) cirrus(F)
> ttm(F) drm_kms_helper(F) nfsd(F) nfs_acl(F) drm(F) auth_rpcgss(F)
> sysimgblt(F) nfs(F) psmouse(F) sysfillrect(F) syscopyarea(F) fscache(F)
> lockd(F) microcode(F) joydev(F) xen_kbdfront(F) i2c_piix4(F) serio_raw(F)
> lp(F) mac_hid(F) sunrpc(F) parport(F) floppy(F) bnx2(F) [last unloaded:
> ipmi_msghandler]
> [79464.816139] Pid: 0, comm: swapper/1 Tainted: GF
> 3.8.0-29-generic #42~precise1-Ubuntu
> [79464.816140] Call Trace:
> [79464.816142] <IRQ> [<ffffffff81059b0f>] warn_slowpath_common+0x7f/0xc0
> [79464.816149] [<ffffffff8135b9d4>] ? timerqueue_add+0x64/0xb0
> [79464.816151] [<ffffffff81059c06>] warn_slowpath_fmt+0x46/0x50
> [79464.816154] [<ffffffff81076794>] ? wake_up_worker+0x24/0x30
> [79464.816157] [<ffffffff81602062>] dev_watchdog+0x262/0x270
> [79464.816160] [<ffffffff810771f0>] ? __queue_work+0x2d0/0x2d0
> [79464.816161] [<ffffffff81601e00>] ? pfifo_fast_dequeue+0xe0/0xe0
> [79464.816164] [<ffffffff8106995b>] call_timer_fn+0x3b/0x150
> [79464.816167] [<ffffffff8144f5a1>] ? add_interrupt_randomness+0x41/0x190
> [79464.816170] [<ffffffff8106b427>] run_timer_softirq+0x267/0x2c0
> [79464.816173] [<ffffffff810ee3c9>] ? handle_irq_event_percpu+0xa9/0x210
> [79464.816175] [<ffffffff81601e00>] ? pfifo_fast_dequeue+0xe0/0xe0
> [79464.816177] [<ffffffff81062620>] __do_softirq+0xc0/0x240
> [79464.816180] [<ffffffff810b67cd>] ? tick_do_update_jiffies64+0x9d/0xd0
> [79464.816184] [<ffffffff816fdd5c>] call_softirq+0x1c/0x30
> [79464.816188] [<ffffffff81016775>] do_softirq+0x65/0xa0
> [79464.816189] [<ffffffff810628fe>] irq_exit+0x8e/0xb0
> [79464.816193] [<ffffffff8140a125>] xen_evtchn_do_upcall+0x35/0x50
> [79464.816195] [<ffffffff816fdeed>] xen_hvm_callback_vector+0x6d/0x80
> [79464.816196] <EOI> [<ffffffff81084008>] ? hrtimer_start+0x18/0x20
> [79464.816201] [<ffffffff81045136>] ? native_safe_halt+0x6/0x10
> [79464.816204] [<ffffffff8101cc33>] default_idle+0x53/0x1f0
> [79464.816206] [<ffffffff8101dad9>] cpu_idle+0xd9/0x120
> [79464.816209] [<ffffffff816d10fe>] start_secondary+0xc3/0xc5
> [79464.816210] ---[ end trace 48cf6b13be16e0ae ]---
> [79464.816214] bnx2 0000:00:05.0 eth1: <--- start FTQ dump --->
> [79464.816220] bnx2 0000:00:05.0 eth1: RV2P_PFTQ_CTL 00010000
> [79464.816225] bnx2 0000:00:05.0 eth1: RV2P_TFTQ_CTL 00020000
> [79464.816258] bnx2 0000:00:05.0 eth1: RV2P_MFTQ_CTL 00004000
> [79464.816263] bnx2 0000:00:05.0 eth1: TBDR_FTQ_CTL 00004000
> [79464.816268] bnx2 0000:00:05.0 eth1: TDMA_FTQ_CTL 00010000
> [79464.816272] bnx2 0000:00:05.0 eth1: TXP_FTQ_CTL 00010000
> [79464.816276] bnx2 0000:00:05.0 eth1: TXP_FTQ_CTL 00010000
> [79464.816280] bnx2 0000:00:05.0 eth1: TPAT_FTQ_CTL 00010000
> [79464.816285] bnx2 0000:00:05.0 eth1: RXP_CFTQ_CTL 00008000
> [79464.816289] bnx2 0000:00:05.0 eth1: RXP_FTQ_CTL 00100000
> [79464.816293] bnx2 0000:00:05.0 eth1: COM_COMXQ_FTQ_CTL 00010000
> [79464.816297] bnx2 0000:00:05.0 eth1: COM_COMTQ_FTQ_CTL 00020000
> [79464.816302] bnx2 0000:00:05.0 eth1: COM_COMQ_FTQ_CTL 00010000
> [79464.816306] bnx2 0000:00:05.0 eth1: CP_CPQ_FTQ_CTL 00004000
> [79464.816308] bnx2 0000:00:05.0 eth1: CPU states:
> [79464.816321] bnx2 0000:00:05.0 eth1: 045000 mode b84c state 80001000
> evt_mask 500 pc 8001288 pc 8001288 instr 38640001
> [79464.816335] bnx2 0000:00:05.0 eth1: 085000 mode b84c state 80005000
> evt_mask 500 pc 8000a5c pc 8000a5c instr 10400016
> [79464.816349] bnx2 0000:00:05.0 eth1: 0c5000 mode b84c state 80000000
> evt_mask 500 pc 8004c14 pc 8004c14 instr 32050003
> [79464.816362] bnx2 0000:00:05.0 eth1: 105000 mode b8cc state 80000000
> evt_mask 500 pc 8000a9c pc 8000a94 instr 8c420020
> [79464.816375] bnx2 0000:00:05.0 eth1: 145000 mode b880 state 80000000
> evt_mask 500 pc 8009c00 pc 800d948 instr 30420040
> [79464.816389] bnx2 0000:00:05.0 eth1: 185000 mode b8cc state 80008000
> evt_mask 500 pc 8000c58 pc 8000c58 instr 1092823
> [79464.816392] bnx2 0000:00:05.0 eth1: <--- end FTQ dump --->
> [79464.816394] bnx2 0000:00:05.0 eth1: <--- start TBDC dump --->
> [79464.816399] bnx2 0000:00:05.0 eth1: TBDC free cnt: 32
> [79464.816401] bnx2 0000:00:05.0 eth1: LINE CID BIDX CMD VALIDS
> [79464.816410] bnx2 0000:00:05.0 eth1: 00 001000 00e8 00 [0]
> [79464.816420] bnx2 0000:00:05.0 eth1: 01 001000 00e8 00 [0]
> [79464.816429] bnx2 0000:00:05.0 eth1: 02 000800 afc8 00 [0]
> [79464.816438] bnx2 0000:00:05.0 eth1: 03 000800 afb8 00 [0]
> [79464.816447] bnx2 0000:00:05.0 eth1: 04 000800 afd8 00 [0]
> [79464.816456] bnx2 0000:00:05.0 eth1: 05 000800 afe0 00 [0]
> [79464.816465] bnx2 0000:00:05.0 eth1: 06 000800 afe8 00 [0]
> [79464.816474] bnx2 0000:00:05.0 eth1: 07 000800 afd0 00 [0]
> [79464.816485] bnx2 0000:00:05.0 eth1: 08 001000 3510 00 [0]
> [79464.816494] bnx2 0000:00:05.0 eth1: 09 000800 aec0 00 [0]
> [79464.816504] bnx2 0000:00:05.0 eth1: 0a 001000 3530 00 [0]
> [79464.816514] bnx2 0000:00:05.0 eth1: 0b 000800 aec8 00 [0]
> [79464.816523] bnx2 0000:00:05.0 eth1: 0c 000800 aed0 00 [0]
> [79464.816559] bnx2 0000:00:05.0 eth1: 0d 001000 34f8 00 [0]
> [79464.816570] bnx2 0000:00:05.0 eth1: 0e 001000 3500 00 [0]
> [79464.816580] bnx2 0000:00:05.0 eth1: 0f 001000 3518 00 [0]
> [79464.816590] bnx2 0000:00:05.0 eth1: 10 1fbc00 2fe8 7d [0]
> [79464.816599] bnx2 0000:00:05.0 eth1: 11 1ab780 fff8 7d [0]
> [79464.816608] bnx2 0000:00:05.0 eth1: 12 17ff00 b908 f7 [0]
> [79464.816618] bnx2 0000:00:05.0 eth1: 13 0cb700 ff40 d7 [0]
> [79464.816627] bnx2 0000:00:05.0 eth1: 14 177a80 efe0 03 [0]
> [79464.816637] bnx2 0000:00:05.0 eth1: 15 037d80 9f88 72 [0]
> [79464.816646] bnx2 0000:00:05.0 eth1: 16 1bae00 eef8 ce [0]
> [79464.816657] bnx2 0000:00:05.0 eth1: 17 1bbc80 a7f8 df [0]
> [79464.816666] bnx2 0000:00:05.0 eth1: 18 17e180 6aa8 e4 [0]
> [79464.816675] bnx2 0000:00:05.0 eth1: 19 07ff80 6e50 dd [0]
> [79464.816683] bnx2 0000:00:05.0 eth1: 1a 1fda80 f790 6e [0]
> [79464.816694] bnx2 0000:00:05.0 eth1: 1b 151580 d7b0 fc [0]
> [79464.816703] bnx2 0000:00:05.0 eth1: 1c 1b9f80 cef8 6b [0]
> [79464.816712] bnx2 0000:00:05.0 eth1: 1d 1ebf00 ffa8 df [0]
> [79464.816723] bnx2 0000:00:05.0 eth1: 1e 1e7e00 ff78 ef [0]
> [79464.816731] bnx2 0000:00:05.0 eth1: 1f 166e80 fbd8 aa [0]
> [79464.816733] bnx2 0000:00:05.0 eth1: <--- end TBDC dump --->
> [79464.817101] bnx2 0000:00:05.0 eth1: DEBUG: intr_sem[0] PCI_CMD[00100406]
> [79464.817239] bnx2 0000:00:05.0 eth1: DEBUG: PCI_PM[19002008]
> PCI_MISC_CFG[92000088]
> [79464.817248] bnx2 0000:00:05.0 eth1: DEBUG: EMAC_TX_STATUS[00000008]
> EMAC_RX_STATUS[00000000]
> [79464.817252] bnx2 0000:00:05.0 eth1: DEBUG: RPM_MGMT_PKT_CTRL[40000088]
> [79464.817257] bnx2 0000:00:05.0 eth1: DEBUG:
> HC_STATS_INTERRUPT_STATUS[01fc0003]
> [79464.817261] bnx2 0000:00:05.0 eth1: DEBUG: PBA[00000003]
> [79464.817264] bnx2 0000:00:05.0 eth1: <--- start MCP states dump --->
> [79464.817270] bnx2 0000:00:05.0 eth1: DEBUG: MCP_STATE_P0[0003611e]
> MCP_STATE_P1[0003611e]
> [79464.817278] bnx2 0000:00:05.0 eth1: DEBUG: MCP mode[0000b880]
> state[80000000] evt_mask[00000500]
> [79464.817286] bnx2 0000:00:05.0 eth1: DEBUG: pc[08003ebc] pc[0800d994]
> instr[32020020]
> [79464.817288] bnx2 0000:00:05.0 eth1: DEBUG: shmem states:
> [79464.817296] bnx2 0000:00:05.0 eth1: DEBUG: drv_mb[01030027]
> fw_mb[00000027] link_status[0008506b]
> [79464.817301] drv_pulse_mb[0000338a]
> [79464.817306] bnx2 0000:00:05.0 eth1: DEBUG: dev_info_signature[44564907]
> reset_type[01005254]
> [79464.817310] condition[0003611e]
> [79464.817318] bnx2 0000:00:05.0 eth1: DEBUG: 000001c0: 01005254 42530083
> 0003611e 00000000
> [79464.817328] bnx2 0000:00:05.0 eth1: DEBUG: 000003cc: 00000000 00000000
> 00000000 00000000
> [79464.817357] bnx2 0000:00:05.0 eth1: DEBUG: 000003dc: 00000000 00000000
> 00000000 00000000
> [79464.817367] bnx2 0000:00:05.0 eth1: DEBUG: 000003ec: 00000000 00000000
> 00000000 00000000
> [79464.817372] bnx2 0000:00:05.0 eth1: DEBUG: 0x3fc[00000000]
> [79464.817374] bnx2 0000:00:05.0 eth1: <--- end MCP states dump --->
> [79464.872988] bnx2 0000:00:05.0 eth1: NIC Copper Link is Down
>
>
>
> On Mon, Feb 10, 2014 at 1:47 PM, Konrad Rzeszutek Wilk <
> konrad.wilk@xxxxxxxxxx> wrote:
>
> > On Mon, Feb 10, 2014 at 01:29:22PM -0500, Mike Neiderhauser wrote:
> > > Thanks for the answers on the timeline.
> > >
> > > When I start the HVM with th broadcom adapter, I get this message back.
> > > Parsing config from /etc/xen/ubuntu-hvm-1.cfg
> > > libxl: error: libxl_pci.c:990:libxl__device_pci_reset: The kernel doesn't
> > > support reset from sysfs for PCI device 0000:05:00.0
> > > libxl: error: libxl_pci.c:990:libxl__device_pci_reset: The kernel doesn't
> > > support reset from sysfs for PCI device 0000:05:00.1
> > >
> > > However, the devices appear in the HVM. Is this something that I should
> > be
> > > concerned about?
> >
> > No. Xen pciback does the reset automatically.
> >
> > Actually we might want to ditch that reporting in libxl, or maybe just
> > implement a stub function in xen-pciback so that libxl will be happy.
> >
> > >
> > >
> > >
> > > On Mon, Feb 10, 2014 at 12:11 PM, George Dunlap <dunlapg@xxxxxxxxx>
> > wrote:
> > >
> > > > On Sat, Feb 8, 2014 at 5:42 PM, Mike Neiderhauser
> > > > <mikeneiderhauser@xxxxxxxxx> wrote:
> > > > > Works like a charm. I do not have physical access to the computer
> > this
> > > > > weekend to verify that the cards are isolated, but the HVM starts and
> > > > > appears to be working well.
> > > > >
> > > > > When do you think Xen 4.4 will be released. The article I read
> > > > mentioned it
> > > > > will be released in 2014 (hinting towards the end of February). I
> > also
> > > > read
> > > > > 'When it is ready.'
> > > > >
> > > > > Any timeline would be great.
> > > >
> > > > I'm afraid that's about all we can give. :-) We've locked down
> > > > development for 2 months now and are working on finding and fixing
> > > > bugs. If there are no more blocker bugs or other unforeseen delays,
> > > > it should be out by the end of February. But there are necessarily
> > > > significant unknowns, so we can't make any promises.
> > > >
> > > > -George
> > > >
> >

Re: [Xen-devel] Xen 4.3 PCI passthrough possible bug