[Xen-devel] The mfn of the frame, that holds a mlock-ed PV domU usermode page, can change

[Rafal Wojtczuk <rafal@xxxxxxxxxxxxxxxxxxxxxx>]

Hello,
Would someone on the list enlight me on the following issue, possibly related 
to mfn management in the PV guest.
Environment: xen-3.4.3, pvops 2.6.32.9 in dom0 and domU, all 64bit.
Usermode code
(if you are interested, at 
http://gitweb.qubes-os.org/gitweb/?p=mainstream/gui.git;a=blob;f=vchan/vchan/init.c;h=cb2fb851c3b97804b115dbf58fd47a30d6d0a8a3;hb=HEAD)
in PV domU does the following:
1) gets a page via
ring_ref_v=mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE |MAP_ANON,-1, 0);
2) mlock(ring_ref_v, 4096)
3) determines the mfn number of the frame holding ring_ref_v via a call to 
u2mfn driver
(http://gitweb.qubes-os.org/gitweb/?p=mainstream/gui.git;a=blob;f=vchan/u2mfn/u2mfn.c;h=6ff113c07c50ef078ab04d9e61d2faab338357e7;hb=HEAD)
the driver basically does get_user_pages+kmap+virt_to_mfn (and later
kunmap+put_page for cleanup).

Then, the
usermode code in dom0 does xc_map_foreign_range on the returned mfn, and can
communicate with the code in domU over this shared page...
...but sometimes, apparently the page that backs ring_ref_v changes: if the 
domU application calls u2mfn ioctl again with ring_ref_v argument, it 
returns a different mfn.
And naturally the code in dom0 reads garbage from the address returned by
its pevious call to xc_map_foreign_range.

I find it puzzling. Is this behaviour normal/expected ?
Mlock man pages say "All pages that contain a part of the specified address 
range are guaranteed to be resident in RAM", not "be resident at the same
RAM location", but why would a frame backing a mlock-ed memory be changed ?
Is there some memory defragmentation going on ? Or maybe only frame->frame
number function changes (but why would it) ?

Anyway, this behaviour causes problems, as you can see in
http://www.qubes-os.org/trac/ticket/16#comment:4
It would be nice if the mfn of a frame that holds a given mlock-ed usermode 
page could be made constant. 

If you can offer some insight, that would be helpful, particularly:
1) Why this does not happen to pages allocated in kernel mode (if it did, it
would break the split drivers model) ?
2) Can this frame-changing behaviour be switched off at Xen/kernel level?
3) Would using grant tables (instead of brutal xc_map_foreign_range) change 
the situation ?
BTW, for Qubes it is necessary to map PV domU usermode pages in dom0; 
particularly, map X server composition buffers.

Regards,
Rafal Wojtczuk
The Qubes OS Project
http://qubes-os.org

Attachment: pgp0V2fZs4uPW.pgp
Description: PGP signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel