Re: [Xen-devel] [PATCH] blkfront: Move blkif_interrupt into a tasklet.

[Jeremy Fitzhardinge <jeremy@xxxxxxxx>]

 On 09/23/2010 09:08 AM, Andrew Jones wrote:
> On 09/03/2010 12:46 AM, Jeremy Fitzhardinge wrote:
>>  On 08/22/2010 11:54 PM, Daniel Stodden wrote:
>>> Response processing doesn't really belong into hard irq context.
>>>
>>> Another potential problem this avoids is that switching interrupt cpu
>>> affinity in Xen domains can presently lead to event loss, if
>>> RING_FINAL_CHECK is run from hard irq context.
>> I just got this warning from a 32-bit pv domain.  I think it may relate
>> to this change.  The warning is
>>
>> void blk_start_queue(struct request_queue *q)
>> {
>>      WARN_ON(!irqs_disabled());
>>
>>
>> Oddly, I only saw this pair once at boot, and after that the system
>> seemed fine...
>>
>> [    4.376451] ------------[ cut here ]------------
>> [    4.377415] WARNING: at /home/jeremy/git/linux/block/blk-core.c:337 
>> blk_start_queue+0x20/0x36()
>> [    4.377415] Modules linked in: xfs exportfs xen_blkfront [last unloaded: 
>> scsi_wait_scan]
>> [    4.377415] Pid: 0, comm: swapper Not tainted 2.6.32.21 #32
>> [    4.377415] Call Trace:
>> [    4.377415]  [<c1039f74>] warn_slowpath_common+0x65/0x7c
>> [    4.377415]  [<c11b3ae1>] ? blk_start_queue+0x20/0x36
>> [    4.377415]  [<c1039f98>] warn_slowpath_null+0xd/0x10
>> [    4.377415]  [<c11b3ae1>] blk_start_queue+0x20/0x36
>> [    4.377415]  [<edc74712>] kick_pending_request_queues+0x1c/0x2a 
>> [xen_blkfront]
>> [    4.377415]  [<edc74ec4>] blkif_do_interrupt+0x176/0x189 [xen_blkfront]
>> [    4.377415]  [<c103e063>] tasklet_action+0x63/0xa8
>> [    4.377415]  [<c103f2d5>] __do_softirq+0xac/0x152
>> [    4.377415]  [<c103f3ac>] do_softirq+0x31/0x3c
>> [    4.377415]  [<c103f484>] irq_exit+0x29/0x5c
>> [    4.377415]  [<c121a1b6>] xen_evtchn_do_upcall+0x29/0x34
>> [    4.377415]  [<c100a027>] xen_do_upcall+0x7/0xc
>> [    4.377415]  [<c10023a7>] ? hypercall_page+0x3a7/0x1005
>> [    4.377415]  [<c10065a9>] ? xen_safe_halt+0x12/0x1f
>> [    4.377415]  [<c10042cb>] xen_idle+0x27/0x38
>> [    4.377415]  [<c100877e>] cpu_idle+0x49/0x63
>> [    4.377415]  [<c14a6427>] rest_init+0x53/0x55
>> [    4.377415]  [<c179c814>] start_kernel+0x2d4/0x2d9
>> [    4.377415]  [<c179c0a8>] i386_start_kernel+0x97/0x9e
>> [    4.377415]  [<c179f478>] xen_start_kernel+0x576/0x57e
>> [    4.377415] ---[ end trace 0bfb98f0ed515cdb ]---
>> [    4.377415] ------------[ cut here ]------------
>> [    4.377415] WARNING: at /home/jeremy/git/linux/block/blk-core.c:245 
>> blk_remove_plug+0x20/0x7e()
>> [    4.377415] Modules linked in: xfs exportfs xen_blkfront [last unloaded: 
>> scsi_wait_scan]
>> [    4.377415] Pid: 0, comm: swapper Tainted: G        W  2.6.32.21 #32
>> [    4.377415] Call Trace:
>> [    4.377415]  [<c1039f74>] warn_slowpath_common+0x65/0x7c
>> [    4.377415]  [<c11b3961>] ? blk_remove_plug+0x20/0x7e
>> [    4.377415]  [<c1039f98>] warn_slowpath_null+0xd/0x10
>> [    4.377415]  [<c11b3961>] blk_remove_plug+0x20/0x7e
>> [    4.377415]  [<c11b39ca>] __blk_run_queue+0xb/0x5e
>> [    4.377415]  [<c11b3af4>] blk_start_queue+0x33/0x36
>> [    4.377415]  [<edc74712>] kick_pending_request_queues+0x1c/0x2a 
>> [xen_blkfront]
>> [    4.377415]  [<edc74ec4>] blkif_do_interrupt+0x176/0x189 [xen_blkfront]
>> [    4.377415]  [<c103e063>] tasklet_action+0x63/0xa8
>> [    4.377415]  [<c103f2d5>] __do_softirq+0xac/0x152
>> [    4.377415]  [<c103f3ac>] do_softirq+0x31/0x3c
>> [    4.377415]  [<c103f484>] irq_exit+0x29/0x5c
>> [    4.377415]  [<c121a1b6>] xen_evtchn_do_upcall+0x29/0x34
>> [    4.377415]  [<c100a027>] xen_do_upcall+0x7/0xc
>> [    4.377415]  [<c10023a7>] ? hypercall_page+0x3a7/0x1005
>> [    4.377415]  [<c10065a9>] ? xen_safe_halt+0x12/0x1f
>> [    4.377415]  [<c10042cb>] xen_idle+0x27/0x38
>> [    4.377415]  [<c100877e>] cpu_idle+0x49/0x63
>> [    4.377415]  [<c14a6427>] rest_init+0x53/0x55
>> [    4.377415]  [<c179c814>] start_kernel+0x2d4/0x2d9
>> [    4.377415]  [<c179c0a8>] i386_start_kernel+0x97/0x9e
>> [    4.377415]  [<c179f478>] xen_start_kernel+0x576/0x57e
>> [    4.377415] ---[ end trace 0bfb98f0ed515cdc ]---
>>
>>      J
>>
> Hi Jeremy,
>
> Any developments with this? I've got a report of the exact same warnings
> on RHEL6 guest. See
>
> https://bugzilla.redhat.com/show_bug.cgi?id=632802
>
> RHEL6 doesn't have the 'Move blkif_interrupt into a tasklet' patch, so
> that can be ruled out. Unfortunately I don't have this reproducing on a
> test machine, so it's difficult to debug.  The report I have showed that
> in at least one case it occurred on boot up, right after initting the
> block device. I'm trying to get confirmation if that's always the case.
>
> Thanks in advance for any pointers you might have.
>

Yes, I see it even after reverting that change as well.  However I only
see it on my domain with an XFS filesystem, but I haven't dug any deeper
to see if that's relevant.

Do you know when this appeared?  Is it recent?  What changes are in the
rhel6 kernel in question?

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel