“Backend has not unmapped grant” errors

[Demi Marie Obenour <demi@xxxxxxxxxxxxxxxxxxxxxx>]

I recently had a VM’s /dev/xvdb stop working with a “backend has not
unmapped grant” error.  Since /dev/xvdb was the VM’s private volume,
that rendered the VM effectively useless.  I had to kill it with
qvm-kill.

The backend of /dev/xvdb is dom0, so a malicious backend is clearly not
the cause of this.  I believe the actual cause is a race condition, such
as the following:

1. GUI agent in VM allocates grant X.
2. GUI agent tells GUI daemon in dom0 to map X.
3. GUI agent frees grant X.
4. blkfront allocates grant X and passes it to dom0.
5. dom0’s blkback maps grant X.
6. blkback unmaps grant X.
7. GUI daemon maps grant X.
8. blkfront tries to revoke access to grant X and fails.  Disaster
   ensues.

What could be done to prevent this race?  Right now all of the
approaches I can think of are horribly backwards-incompatible.  They
require replacing grant IDs with some sort of handle, and requiring
userspace to pass these handles to ioctls.  It is also possible that
netfront and blkfront could race against each other in a way that causes
this, though I suspect that race would be much harder to trigger.

This has happened more than once so it is not a fluke due to e.g. cosmic
rays or other random bit-flips.

Marek, do you have any suggestions?
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab

Attachment: signature.asc
Description: PGP signature