RE: Linux pin_user_pages_fast fails on Xen

[Stefano Stabellini <stefano.stabellini@xxxxxxx>]

The problem is that drivers/xen/privcmd.c:privcmd_mmap sets VM_IO |
VM_PFNMAP, and either flag would cause check_vma_flags to return
-EFAULT.

Do you know if it works if you remove VM_IO | VM_PFNMAP from
privcmd_mmap?

Juergen, do you think the flags are necessary and useful? Any
suggestions?


On Mon, 12 Sep 2022, NK, JESHWANTHKUMAR (JESHWANTH KUMAR) wrote:
> Missed to update the Flag details:
> 
> Flag for DMA Mapped VA - 0x0C0644BB
> Flag for Local VA        -  0x08100073
> 
> 
> VM_IO and VM_PFNMAP  - Set in DMA mapped VA but not in local VA.
> 
> Regards,
> Jeshwanth
> 
> -----Original Message-----
> From: NK, JESHWANTHKUMAR (JESHWANTH KUMAR) 
> Sent: Tuesday, September 13, 2022 11:05 AM
> To: 'Stefano Stabellini' <stefano.stabellini@xxxxxxx>
> Cc: Stabellini, Stefano <stefano.stabellini@xxxxxxx>; 
> boris.ostrovsky@xxxxxxxxxx; xen-devel@xxxxxxxxxxxxxxxxxxxx; Rangasamy, 
> Devaraj <Devaraj.Rangasamy@xxxxxxx>; Pandeshwara krishna, Mythri 
> <Mythri.Pandeshwarakrishna@xxxxxxx>; SK, SivaSangeetha (Siva Sangeetha) 
> <SivaSangeetha.SK@xxxxxxx>; Thomas, Rijo-john <Rijo-john.Thomas@xxxxxxx>; 
> jgross@xxxxxxxx
> Subject: RE: Linux pin_user_pages_fast fails on Xen
> 
> [AMD Official Use Only - General]
> 
> Hi Stefano,
> 
> https://elixir.bootlin.com/linux/v5.16/source/mm/gup.c#L975 is the -EFAULT 
> returning for our current use case.
> 
> access_ok is fine.
> 
> Regards,
> Jeshwanth
> 
> -----Original Message-----
> From: Stefano Stabellini <stefano.stabellini@xxxxxxx>
> Sent: Tuesday, September 13, 2022 6:56 AM
> To: NK, JESHWANTHKUMAR (JESHWANTH KUMAR) <JESHWANTHKUMAR.NK@xxxxxxx>
> Cc: Stabellini, Stefano <stefano.stabellini@xxxxxxx>; 
> boris.ostrovsky@xxxxxxxxxx; xen-devel@xxxxxxxxxxxxxxxxxxxx; NK, 
> JESHWANTHKUMAR (JESHWANTH KUMAR) <JESHWANTHKUMAR.NK@xxxxxxx>; Rangasamy, 
> Devaraj <Devaraj.Rangasamy@xxxxxxx>; Pandeshwara krishna, Mythri 
> <Mythri.Pandeshwarakrishna@xxxxxxx>; SK, SivaSangeetha (Siva Sangeetha) 
> <SivaSangeetha.SK@xxxxxxx>; Thomas, Rijo-john <Rijo-john.Thomas@xxxxxxx>; 
> jgross@xxxxxxxx
> Subject: Re: Linux pin_user_pages_fast fails on Xen
> 
> On Sat, 10 Sep 2022, Juergen Gross wrote:
> > On 09.09.22 22:25, Stefano Stabellini wrote:
> > > On Fri, 9 Sep 2022, Juergen Gross wrote:
> > > > On 09.09.22 04:11, Stefano Stabellini wrote:
> > > > > Adding more people in CC
> > > > > 
> > > > > On Thu, 8 Sep 2022, Stefano Stabellini wrote:
> > > > > > Hi Juergen,
> > > > > > 
> > > > > > A colleague is seeing a failure on x86 in Linux Dom0. The 
> > > > > > failure is pin_user_pages_fast with addresses that correspond 
> > > > > > to foreign memory
> > > > > > pages:
> > > > > > 
> > > > > > - QEMU maps a domU address using dma_memory_map
> > > > > > (xen_map_cache)
> > > > > > - QEMU calls an IOCTL to the TEE subsystem with the Virtual Address
> > > > > >     returned by dma_memory_map
> > > > > > - Linux tee_shm_register->pin_user_pages_fast Returns -14 - 
> > > > > > drivers/tee/tee_shm.c
> > > > > > 
> > > > > > Once upon a time it used to be the case that 
> > > > > > get_user_pages_fast would fail on Xen because we didn't have a 
> > > > > > struct page corresponding to foreign memory mappings. But that 
> > > > > > hasn't been the case for years now.
> > > > > > 
> > > > > > Any other ideas why it would fail?
> > > > 
> > > > I think we can expect that access_ok() isn't failing.
> > > > 
> > > > I assume the mapping was done allowing writes (sorry for paranoia mode)?
> > >   I was told it was verified: QEMU could read and write to the VA 
> > > returned by dma_memory_map. From /proc/<qemu-pid>/maps, the VA 
> > > assigned after the mapping is pointing to /dev/xen/privcmd.
> > > 
> > > 
> > > > Other than that I'm not having enough memory management skills. It 
> > > > might be related to mmap()-ed foreign pages having _PAGE_SPECIAL 
> > > > set, though.
> > > 
> > > Do we still set PAGE_SPECIAL for foreign mapped pages? It looks like 
> > > it is not there anymore? If PAGE_SPECIAL is not there, then they 
> > > really should look like regular pages?
> > 
> > See the call of pte_mkspecial() in remap_area_pfn_pte_fn() (mmu_pv.c).
> 
> The kernel version is 5.16 and the return code is -EFAULT. Is it the 
> following -EFAULT the one that triggers?
> 
> mm/gup.c:internal_get_user_pages_fast:
> 
>       if (unlikely(!access_ok((void __user *)start, len)))
>               return -EFAULT;
>