Re: [Xen-devel] Hikey: Enable Xen + Mainline Linux Kernel

[Leo Yan <leo.yan@xxxxxxxxxx>]

Hi all,

On Tue, Jan 15, 2019 at 10:49:58AM +0800, Leo Yan wrote:

[...]

> [    1.807591] Modules linked in:
> [    1.810717] CPU: 4 PID: 1 Comm: swapper/0 Not tainted 
> 5.0.0-rc2-00001-g5b47dea3757c #3
> [    1.818691] Hardware name: HiKey Development Board (DT)
> [    1.823983] pstate: 40000005 (nZcv daif -PAN -UAO)
> [    1.828848] pc : xen_swiotlb_alloc_coherent+0x64/0x1e8
> [    1.834044] lr : dma_alloc_attrs+0xf4/0x110
> [    1.838289] sp : ffff000010073a50
> [    1.841671] x29: ffff000010073a50 x28: 0000000000000007 
> [    1.847047] x27: ffff000011150068 x26: ffff80001b6ddd60 
> [    1.852429] x25: ffff000010caaa70 x24: 0000000000000000 
> [    1.857800] x23: 0000000000001000 x22: 0000000000001000 
> [    1.863177] x21: 0000000000000000 x20: ffff80001c2edc10 
> [    1.868553] x19: ffff0000111fd000 x18: ffffffffffffffff 
> [    1.873930] x17: 0000000000000000 x16: 0000000000000000 
> [    1.879306] x15: ffff0000111fd6c8 x14: ffff0000900737b7 
> [    1.884683] x13: ffff0000100737c5 x12: ffff000011215000 
> [    1.890060] x11: 0000000005f5e0ff x10: ffff0000111fd940 
> [    1.895436] x9 : 0000000000000000 x8 : ffff80001bb0e700 
> [    1.900813] x7 : 0000000000000000 x6 : 0000000000000000 
> [    1.906189] x5 : ffff0000105bdbb8 x4 : 0000000000000000 
> [    1.911566] x3 : 00000000006000c0 x2 : ffff80001b6ddd60 
> [    1.916943] x1 : 0000000000001000 x0 : 0000000000000000 
> [    1.922326] Process swapper/0 (pid: 1, stack limit = 0x(____ptrval____))
> [    1.929084] Call trace:
> [    1.931602]  xen_swiotlb_alloc_coherent+0x64/0x1e8
> [    1.936456]  dma_alloc_attrs+0xf4/0x110
> [    1.940359]  dmam_alloc_attrs+0x64/0xb8
> [    1.944264]  dw_mci_probe+0x5f8/0xb00
> [    1.947990]  dw_mci_pltfm_register+0xa0/0xd0
> [    1.952327]  dw_mci_k3_probe+0x2c/0x38

Some update for this issue after dig a bit for related code; with
below simple hacking, the kernel can boot successfully to rootFS:

diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index f6ded992c183..31c7e17f0fe5 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -196,7 +196,8 @@ static inline int dma_mmap_from_global_coherent(struct 
vm_area_struct *vma,

 static inline bool dma_is_direct(const struct dma_map_ops *ops)
 {
-       return likely(!ops);
+       return true;
+       //return likely(!ops);
 }

Though this minor code tweaking can workaround the kernel panic, but
it's not a formal fixing; if we look into the kernel code, we can see
firstly the kernel will initialize dma operation pointer in
arch/arm64/mm/dma-mapping.c, arch_setup_dma_ops():

void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
                        const struct iommu_ops *iommu, bool coherent)
{
        dev->dma_coherent = coherent;
        __iommu_setup_dma_ops(dev, dma_base, size, iommu);

#ifdef CONFIG_XEN
        if (xen_initial_domain()) {
                dev->archdata.dev_dma_ops = dev->dma_ops;   // since 
'dev->dma_ops' is NULL,
                                                            // so 
dev->archdata.dev_dma_ops
                                                            // will be 
initialized as NULL
                dev->dma_ops = xen_dma_ops;
        }
#endif
}

So we can see 'dev->archdata.dev_dma_ops' will be NULL and
'dev->dma_ops' is assigned to xen_dma_ops;

In dw mmc driver init function, it will run into below flow:

  dw_mci_init_dma()
    `> dmam_alloc_coherent()
         `-> dmam_alloc_attrs()
               `-> dma_alloc_attrs()
                     `-> xen_dma_ops::alloc()
                           `-> xen_swiotlb_alloc_coherent()
                                 `-> xen_alloc_coherent_pages()
                                       `-> xen_get_dma_ops()

So xen_get_dma_ops() will try to retrieve pointer from
'dev->archdata.dev_dma_ops' but because it's NULL so at the end
introduces kernel panic will NULL pointer.

Seems to me, we should check two pointers in dma_is_direct(), one
is 'dev->dma_ops' and another is 'dev->archdata.dev_dma_ops', should
both of them are not NULL pointers then we can run into xen_alloc_xxx
related function, otherwise it should fallback to use
dma_direct_alloc() to allocate dma pages?

Also very welcome if you could work on formal fixing and at my side
I am glad to test it!

Thanks,
Leo Yan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel