[Xen-devel] Re: GNTTABOP_unmap_grant_ref

[Mark Williamson <mark.williamson@xxxxxxxxxxxx>]

> I'm trying fix my code running in a domU to avoid Xen calling
> domain_crash() in mm.c line 615.  This occurs when a domain "implicitly
> unmaps a granted PTE".  The somewhat abridged version of events:
>
> 1) Remote domU shares a page
> 2) app in local domU calls mmap()
> 3) driver mmap code calls GNTTAB_map_grant_ref() and returns the mapping
> to the app
> 4) app accesses the shared page without problem
> 5) app is done, calls munmap()
> 6) Xen kills the domain in mm.c line 615.

Linux doesn't understand that the memory you've mmap-ed is "magic" and it's 
trying to simply zap the PTEs that refer to it.  Because those were installed 
using the grant tables mechanisms, it looks like a domain bug to Xen - you 
can't mix managing PTEs using grant tables *and* writable pagetables (side 
note: I'd like to remove this restriction at some point, since XenFS is going 
to run up against it).

I'd expect that this *might* be a problem if the system starts swapping... if 
Linux thinks it can fool around with your PTEs, it almost certainly thinks it 
can do other undesirable stuff.  We need to keep the memory management 
routines well away from granted memory in this case!

> I have not yet found a way to make the GNTTABOP_unmap_grant_ref() call
> in time to avoid the crash.  The driver release() function is too late
> and there's no hook in struct file_operations to handle unmap
> explicitly

I agree with Muli - it's a property of the VMA, not of the file, etc.It's 
unfortunate hooking the destructor didn't work - but all is not lost.

You need to to take away control of that memory from the core memory 
management algorithms.  In the olden days you might have tried setting the 
RESERVED bit for the struct page(s) in question.  This is no longer the Done 
Thing - it's a bit more complex...

See:
http://lwn.net/Articles/160501/ (for background info from the recent past)
http://lwn.net/Articles/162860/ (for the current state of affairs)

You'll need a 2.6.15 kernel for this (like the one in our merge tree) but it 
appears to be The Way Of The Future, for now.  You probably want something 
like VM_PFNMAP.  The case for you will be slightly different, since you're 
not mapping in device memory and won't be using the usual IO remapping 
functions.

A scan through the code might reveal some useful insight, since this has the 
look of a poorly documented interface - probably worth checking out the 
original changeset in kernel.org's gitweb to see what changed together.

> Mark Williamson's blktap driver is calling 
> GNTTAB_unmap_grant_ref, but it's unclear to me if the same unmap()'ing
> problem exists there too.

I can't take credit for blktap, although I wish I could!  IIRC, Andy Warfield 
wrote most of it.  My name might have got into the copyright somewhere, but 
I've never worked on the tap functionality.

As a result I also can't give specific advice on the code, but I'd think it is 
probably a very good guide regarding mapping foreign pages into userspace.

Cheers,
Mark

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel