[Xen-devel] Re: xen-swiotlb

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

Monday, August 16, 2010, 4:43:53 PM, you wrote:

> On Sun, Aug 15, 2010 at 11:28:39PM +0200, Sander Eikelenboom wrote:
>> Hi Konrad,
>> 
>> After i have exhausted all kernel debug options without getting a pointer to 
>> my freezes, i have added some printk's to all functions in swiotlb-xen.c
>> I see a lot of calls to xen_swiotlb_dma_mapping_error, which doesn't seem to 
>> be good ?

> The driver looks to be actually testing the value, which is great. Some
> of the older drivers (r8169) don't even do that.
>> 
>> Although all the errors the device works fine (grabs video), but eventually 
>> the machine freezes, probably due to overwriting some mem it shouldn't.

> Looking at the output, the physical addresses that DMA-ed are:

> 0x1f2962dc0
> 0x1f24f2e68

> and they look to be called quite often. In fact, there looks to be a
> loop that does something like this:

> again:
>   p = kmalloc(..)

>   dma = pci_map_single(p)
>   pci_dma_mapping_error(dma);
>   /* get some data.. */
>   /* parse the: (pipe 0x80000280): IN:  c0 00 00 00 0c 00 01 00 */
>   pci_unmap_sg(dma);
>   goto again;

> As the virtual address sent to pci_map_single looks to be sequentially
> increasing.

> It might be:
>  a). the pci_dma_mapping_error is used incorrectly, ie, it is used
>      as !pci_dma_mapping_error,  but I doubt that - the Linux kernel
>      has soo many exampples of how to proper use that.
>  b). The pci_dma_mapping_error implementation in Xen-SWIOTLB is busted,
>      but I can't see how. The logic is basically 'return !addr' so, if
>      you have addr = 0xf200000', you will get 0, which is the proper
>      return value.
>  c). the xhci driver does something similar to the pseudo-code I've
>      pointed out. It is missing a kfree somewhere.

> Can you point me to the git tree for the xhci and I can take a look
> there? Also could you send me yor debug patch - that will help in
> finding the culprit.

I suspect the xhci drivers, because usb2 (ohci/ehci) works fine, but it's all 
hard to find due to the freeze without debug output.
And for linux, xen and pci-passthrough is still somewhat a corner case :(

xhci-isoc patches required for webcams/videograbbers to work got merged in 
2.6.36 merge window, together with xen-swiotlb.
(isoc patch series by "Xu, Andiry" <Andiry.Xu@xxxxxxx> , maintainer of the 
complete xhci tree is Sarah Sharp <sarah.a.sharp@xxxxxxxxxxxxxxx>)


So what i have as domU kernel is:
- from Linus tree 2.6.36-pre with latest commit 
8357422d4bf33bc2c35884d4016c3fc9efbbc1d2
- pulled your devel/xen-pcifront-0.5 tree, and fixed the merge conflicts due to 
the pv on hvm patches, in the same way Jeremy did for his 2.6.36 branch.
    Conflicts:
        drivers/xen/events.c
        include/xen/events.h

- Added a patch for xhci isoc that fixes another bug. (attached 
0001-xHCI-update-ring-dequeue-pointer-when-process-missed.patch)
- Added a patch that shows extra debug info for xhci from the author of the 
xhci-isoc pathes (attached isoc_length5.patch)

- Changed some debug lines from dbg to warn level, (just enabling xhci-debug in 
kernel config floods the logs too fast, so i just enabled them in xhci-mem.c)
  (Attached a patch with all my changes to xhci*, including the 2 patches above 
applied.)

- Added some printk's to swiotlb-xen.c to see which functions were used, and 
let some of them print the address as well, in the hope i could find some debug 
info there.(attached)

Apart from fixing the xhci in the end, is there a way to prevent xen from 
freezing altogether without leaving a trace ?
Even an Oops is much easier to debug than a freeze. Due to the nature of DMA 
that could perhaps be difficult, although there is an DMA API ...

Had a fruitful LinuxCon ?

--
Sander

Attachment: all-xhci-patches
Description: Binary data

Attachment: swiotlb-xen-debug-patch
Description: Binary data

Attachment: 0001-xHCI-update-ring-dequeue-pointer-when-process-missed.patch
Description: Binary data

Attachment: isoc_length5.patch
Description: Binary data

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel