Re: [Xen-devel] xc_evtchn_status fails with EFAULT on HVM, the same on PV works

[Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>]

On Sat, Jan 14, 2017 at 01:47:49AM +0000, Andrew Cooper wrote:
> On 13/01/2017 20:32, Marek Marczykowski-Górecki wrote:
> > On Fri, Jan 13, 2017 at 07:54:01PM +0000, Andrew Cooper wrote:
> >> This is a guest kernel bug.  The guest kernel has used SMAP to say
> >> "please disallow all userspace accesses", and Xen is respecting this
> >> when trying to follow the pointer in the hypercall.
> > Hmm, if that's the case, isn't the above patch also effectively breaking
> > SMAP?
> 
> No.
> 
> > I see the purpose of SMAP is to prevent _accidental_ access to
> > arbitrary, guest controlled memory.
> 
> This is correct.  The point of SMAP is to prevent accidental access to
> user memory.
> 
> The ABI of the privcmd hypercall ioctl() is to pass values straight to
> the hypervisor, and it is a fact that most of these hypercalls contain
> pointers to hypercall buffers, which reside in userspace.
> 
> (There is a separate thread ongoing questioning whether the ABI is
> appropriate, but irrespective of those results, this is how the ABI
> currently behaves.  A concerned person might note that anyone with an
> open privcmd fd can issue any hypercall, including the ones only
> intended for kernels, such as set_trap_table, update_va_mapping and iret.)
> 
> Therefore, it is an expected consequence that a hypercall passed blindly
> from a userspace agent contains userspace pointers which Xen must follow
> to complete the hypercall successfully.
> 
> > For example because of some memory
> > corruption bug in the hypervisor. If such a bug could be triggered using
> > a hypercall, then the above patch also makes SMAP useless.  Patching
> > copy_from_guest/copy_to_guest on the other hand would only allow such
> > guest controlled memory access when hypervisor is really supposed to
> > access guest memory.
> 
> I don't follow this argument.
> 
> In a native situation, SMAP raises #PF if hardware tries to follow a
> user pointer outside of an area whitelisted by the kernel.  (The
> copy_*_guest path suppresses #PF, opting instead to return -EFAULT.)
> 
> The choice of supervisor vs user in a pagewalk is a single bit, and Xen
> (for better or worse, but realistically as a consequence of SMAP being
> ~10 years younger than HVM guests) accesses pointers from hypercalls as
> a supervisor entity when walking the pagetables.  The key point here is
> that Xen must emulate the hardware pagewalker when trying to find the
> data being pointed to.
> 
> If Xen has a bug causing spurious accesses to HVM guests, then all bets
> are already off wrt memory corruption, because real hardware isn't used
> to make the access.
> 
> As indicated before, we could deliberately break the Xen pagewalker to
> ignore SMAP.  However, that would be identical to "pretend the guest
> actually whitelisted this access". 

I think there is important difference between "whitelist all accesses
to guest user memory for the hypercall time" vs "whitelist accesses done
through copy_from_guest/copy_to_guest only". If I understand correctly
above description, modifying privcmd_call would also suppress #PF when
Xen would be tricked to access guest memory outside of copy_*_guest. Am
I missing something?

Anyway, if someone can access /dev/xen/privcmd and issue hypercalls,
probably also can whitelist userspace access... So the practical
difference is very small - apart from that I control Xen version, but
not necessary the kernel. And I think this applies to many more Xen
users (from which only I've tripped over this issue...). What is the
point of SMAP if guest can disable it? Is it only about protecting
hypervisor when attacker _does not_ control guest kernel?

> This would fix your symptoms, but
> open up a hole in the case that userspace somehow managed to trick Xen
> into thinking a legitimate hypercall had been made, at which point Xen
> would ignore the last level of protection that SMAP was supposed to offer.
> 
> >> This bug doesn't
> >> manifest for PV guests, and you are probably the first person to do any
> >> serious hypercall work from HVM userspace.
> > That's interesting. I'm just trying to use slightly modified
> > libxenchan...
> 
> And I believe that you are first person (in 6 years of being a part of
> upstream Xen) who I have positively identified as having used
> libxenchan.  (There have been a few comments along the line of "oh yeah
> - there is that libxenchan thing which might be worth looking at".)
> 
> It does raise some questions though.  Linux has (what is supposed to be
> a) perfectly good /dev/xen/evtchn, for interacting with event channels. 
> Why is libxenchan open-coding the hypercalls using a lower level
> interface?  Can it be modified to use the higher level interfaces?

Actually, vanilla libxenvchan do use /dev/xen/evtchn. I have a wrapper
library which additional check - if remote domain is still alive.
Otherwise there are multiple cases when libxenvchan will simply hang if
remote domain disappear without proper cleanup. Some of those cases are
covered by xengntshr_share_page_notify/xengnttab_map_grant_ref_notify
(namely: those cases when only application crashes, but kernel is still
alive). But not all...

The actual check use xc_evtchn_status() and observe result - if it get
unbound/interdomain status then the remote domain is still alive, but if
it get -ESRCH - then it's dead and application should consider the
connection broken.
Is there any better way to check if a domain with given ID is alive? Or
better - get a notification of its death (this all from domU)?

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

Attachment: signature.asc
Description: PGP signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel