[Xen-devel] xSplice prototype

[Ross Lagerwall <ross.lagerwall@xxxxxxxxxx>]

Hi all,

I've gone ahead and implemented a bunch of xSplice stuff based on 
Konrad's xsplice branch. It is sitting in my github repository:
https://github.com/rosslagerwall/xen tagged prototype-v1
This is obviously prototype code, but please try it out. You'll also 
need this tool to build patches: 
https://github.com/rosslagerwall/xsplice-build tagged prototype-v1

Much of the work is implementing a basic version of the Linux kernel 
module loader. So the code does:
* Loading of xsplice modules.
* Copying allocated sections into a new executable region of memory.
* Resolving symbols.
* Applying relocations.
* Patching of altinstructions.
* Special handling of bug frames and exception tables.
* Unloading of xsplice modules.

The other main bit of this work is applying and reverting the patches 
safely. As implemented, the code is patched with each CPU waiting in the 
return-to-guest path (i.e. with no stack) which appears to be the safest 
way of patching. It should mean that stack checking is not required.
All of the following should work:
* Applying patches safely.
* Reverting patches safely.
* Replacing patches safely (e.g. reverting any applied patches and 
applying a new patch).
* Bug frames as part of modules. This means adding or changing WARN, 
ASSERT, BUG, and run_in_exception_handler works correctly. Line number 
only changes _are ignored_.
* Exception tables as part of modules. E.g. wrmsr_safe and copy_to_user 
work correctly when used in a patch module.
* Hook load and unload functions which run at patch apply and revert 
time respectively.
* Shadow variables. A minimal bit of infrastructure to attach a new 
variable to an existing data structure.

Limitations
===========
The above is enough to fully implement an update system where multiple 
source patches are combined (using combinediff) and built into a single 
binary which then atomically replaces any existing loaded patches (this 
is why I added a REPLACE operation). This is the approach used by kPatch 
and kGraft. Multiple completely independent patches can also be loaded 
but unexpected interactions may occur.
As it stands, the patches are statically linked which means that 
independent patches cannot be linked against one another (e.g. if one 
introduces a new symbol). Using the combinediff approach above fixes this.
Backtraces containing functions from a patch module do not show the 
symbol name.
There is no checking that a patch which is loaded is built for the 
correct hypervisor.
Binary patching works at the function level.


Design thoughts
===============
Combining patches at the source level is relatively easy. Multiple 
binary patches applied at runtime is tricky. I'm not convinced that it 
is necessarily a good idea. Based on the discussion so far, the sanest 
way of doing this that I can think of is:
* Each hypervisor has an embedded build id.
* Each binary patch has an embedded build id.
* The hypervisor should expose its build id and the build id of every 
loaded binary patch.
* Each binary patch specifies one or more build ids on which it depends. 
These build ids can be either a hypervisor build id or another patch 
build id. The dependencies could either identified automatically or by a 
developer.
* The userspace tool enforces dependency management (user can optionally 
force patch apply). I don't see any reason to involve the
hypervisor for dependency management.
Implementing this scheme will require dynamically linking the binary 
patches.
The CHECK phase seems unnecessary to me. I would think that any safety 
checking that needs to be done would be done atomically at the point of 
patch apply (or revert). Given the implemented system of applying 
patches, I'm not sure if any safety checking need be done at all.

Testing
=======
In case it's not clear what the workflow is, here is a sample transcript:

$ mkdir ~/work
$ cd ~/work
$ git clone git://github.com/rosslagerwall/xen.git
$ cd xen
$ git checkout prototype-v1
$ # Build a debug Xen and tools (including misc/xen-xsplice), install on 
a host and reboot
$ # Write a patch
$ git diff > ~/work/test1.patch
$ git reset --hard
$ # Write another patch
$ git diff > ~/work/test2.patch
$ git reset --hard
$ # Write another patch
$ git diff > ~/work/test3.patch
$ git reset --hard

$ cd ~/work
$ git clone git://github.com/rosslagerwall/xsplice-build.git
$ cd xsplice-build
$ git checkout prototype-v1
$ make
$ ./xsplice-build -s ~/work/xen -p ~/work/test1.patch -o out1 
--xen-debug --debug
$ ./xsplice-build -s ~/work/xen -p ~/work/test2.patch -o out2 
--xen-debug --debug
$ ./xsplice-build -s ~/work/xen -p ~/work/test3.patch -o out3 
--xen-debug --debug
$ # copy out*/test*.xsplice onto the host


On the host:
# xen-xsplice upload test1 test1.xsplice
# xen-xsplice upload test2 test2.xsplice
# xen-xsplice upload test3 test3.xsplice
# xen-xsplice check test1
# xen-xsplice check test2
# xen-xsplice check test3

# xen-xsplice apply test1
# # Verify test1 is applied
# xen-xsplice apply test2
# # Verify test2 is also applied
# xen-xsplice replace test3
# # Verify test3 is applied and test1 and test2 are not
# xen-xsplice revert test3
# # Verify test3 is not applied

# xen-xsplice unload test1
# xen-xsplice unload test2
# xen-xsplice unload test3

Thanks,
--
Ross Lagerwall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel