Xen Summit 2023 Design Session: Communicating guest informations to the management stack

[Yann Dirson <yann.dirson@xxxxxxxx>]

Here are my notes from memory, organized by topic (and somehow enriched 
with more details in places)

# context

Guest-management stacks typically need the guest to communicate some 
information back. This essentially happens through Xenstore and some of 
it has been normalized in the Xen docs.

Some of those pieces of information are filled by PV drivers (eg. 
~/control/feature-poweroff), others like information on network 
configuration are more commonly filled by a userspace daemon (eg. 
xe-guest-utilities for XAPI-based platforms). People in need to publish 
such info even turn to forked old versions of those tools for lack of a 
central solution.

We have started to work on a prototype guest agent (written in Rust) 
with the goal of providing such an official guest service within the Xen 
Project, that would be easily portable and use low resources.


# guest agent Rust prototype

Question was asked why we selected Rust: aside from the language's 
well-known selling-points this was largely a political decision, 
intended to send a signal that the Xen project values such new technologies.

There is a separate Xenserver guest agent for Windows guests.  The Rust 
agent could likely get implementations for writing to the Windows idiom 
for Xenstore (through WMI), and to get notified of network config 
changes, which would still allow to share the rest (policy, xenstore 
layout, likely other things in the future).

It has been pointed out the Xenserver Windows guest agent has support 
for other features not available in the Linux version, like Copy-paste 
(which currently is problematic, using Xenstore to stream 
potentially-large amounts of data).  Such features should be made 
better, and available everywhere.

Another lesser-known feature of the Xenserver windows guest agent allows 
(depending on a guest registry key for enabling the feature) to let dom0 
execute commands in the guest.  It was noted the concept is not that far 
for QubesOS qrexec mechanism.


# NIC information

The most visible info provided by guest tools is external IP addresses. 
This is useful for scripts in dom0 to discover it to communicate with 
guests through a given VNIC, and for users to get the information 
without having to manually log through the VM console.

Today addresses are reported for VIF and SRIOV-passthrough interfaces, 
but not for those on PCI-passthrough, USB-passthrough[0] or bridges that 
include such interfaces.  Reporting more info in a backward-compatible 
way would make it easier for the rest of the world.

For additional passed-through devices we need a way for the guest to 
identify the devices to report about (MAC address?  hw topology?), and 
we'll have to dedicate a new xenstore subtree to those.

For bridged VNICs the bridge IP can accurately be reported for the VNIC 
itself, as if it were not bridged.  But confusion can occur if 2 VNICs 
get bridged together, as they would both report the same IP.  This could 
be made better by adding more information to the VNIC in addition to 
addresses, e.g. adding a guest-unique identifier for the bridge 
interface, so the toolstack can deduce the existence of the bridge.


[0] although we likely don't care much about that one in the server 
space, desktop xen distros could have a need for this - edge computing 
and homelab possibly too, eg. for the RaspberryPi Ethernet


# control of memory ballooning

Current Xenserver guest agent sets a `feature-balloon` flag.  It is not 
meant as a feature flag though, but signals to the XAPI balloon 
controller (squeezed) that the guest has booted far enough that all its 
memory has been allocated, and that it can now "calibrate its behaviour" 
to be able to decide when a balloon driver has reached the goal it was set.

My evaluation was that the balloon driver would better report to 
xenstore when it has reached its target, rather than having the balloon 
controller doing guesswork.  It was added that it would be useful for 
the balloon driver to report impossibility to reach a target, and 
possibly other conditions.

Guest agent would still have to write this node for backward compatibility.

It was mentioned that a "guest booted" report could be more generally 
useful.  (we did not discuss any particular use, or whether that would 
be a job for guest tools or PV drivers)


# standardization of xenstore interface

Usage of xenstore by the guest agent and PV drivers was documented after 
the fact in the xenstore-paths document in Xen source tree.  That could 
be cleaned up and made more useful.

My proposed patch to standardize the permissions on `~/control/` as set 
by libxl was noted as problematic for XAPI: standardization should leave 
the current situation legal and allow both alternatives.



Thanks to everyone who attended and contributed.

-- 
Yann Dirson | Vates Platform Developer
XCP-ng & Xen Orchestra - Vates solutions
w: vates.tech | xcp-ng.org | xen-orchestra.com


Yann Dirson | Vates Platform Developer

XCP-ng & Xen Orchestra - Vates solutions
web: https://vates.tech