Re: [Xen-devel] [PATCH 00/12] libxl: fork: SIGCHLD flexibility

[Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>]

Ian Jackson writes ("Re: [Xen-devel] [PATCH 00/12] libxl: fork: SIGCHLD 
flexibility"):
> I have just looked at the code in vireventpoll.c and there is nothing
> that stops this being a problem.

I think it's a problem with fd callbacks too.

fd callbacks aren't always removed by the very same handler; they
might be removed by pretty much any arbitrary code (although it will
be in _some_ handler).

It isn't correct to defer removal of an fd callback because it is
important that by the time the removal callback returns, the fd is no
longer being polled by the event loop.  That includes being polled by
_any_ instance of the event loop perhaps in another thread.  Neither
libxl nor libvirt have any machinery to wake up other instances of the
event loop in other threads to change their fd sets, and to wait until
that's done.

libxl could (and perhaps should) grow such a thing in its own event
loop, but thinking about how to write it, it's going to be quite
tedious![1]

Changing the identity of the file objects referred to by fds being
polled on in another thread has unfortunate behaviours.  In
experiments done by a friend of mine we discovered that:
 * Linux keeps a secret handle onto the old open-file struct
 * FreeBSD honours the change immediately
 * Solaris returns EBADF even if the operation is dup2 to
   dup a new open file object onto the existing fd
(After discussion we agreed that a close reading of SuS permits only
the FreeBSD behaviour if the open file objects refer to plain files
which in practice means that the Solaris behaviour is buggy.  I think
the Linux behaviour is mad and the FreeBSD behaviour correct.)

And of course if the fd remains being polled on after the internal
deregistration callback has returned, libxl will feel free to close
the fd.  It might then be reused for any arbitrary object, which might
cause malfunctions if polled in the old way.  And in any case poll
might return EBADF which will probaby crash the event loop.

I'm trying to think of a way for libvirt to handle all of this without
duplicating the horrible synchronisation/wakeup thing I'm
contemplating for libxl.

Ian.

[1] Add a ctx-wide list of pollers, one for every libxl thread in
poll.  This list has to be covered by its own lock.

When fd deregistration occurs, we take this lock, wake up all the
pollers, release the lock, and then wait (perhaps with a condition
variable) for the pollers to acknowledge that they have left poll().

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel