Re: [PATCH] xen/evtchn: Add design for static event channel signaling for domUs..

[Julien Grall <julien@xxxxxxx>]

Hi Rahul,

Title: s/../.../

On 23/03/2022 15:43, Rahul Singh wrote:
> in dom0less system. This patch introduce the new feature to support the
s/introduce/introduces/
s/the new/a/

> signaling between two domUs in dom0less system.
Did you intend to add a newline before the second sentence?

> Signed-off-by: Rahul Singh <rahul.singh@xxxxxxx>
> ---
>   docs/designs/dom0less-evtchn.md | 96 +++++++++++++++++++++++++++++++++
>   1 file changed, 96 insertions(+)
>   create mode 100644 docs/designs/dom0less-evtchn.md
>
> diff --git a/docs/designs/dom0less-evtchn.md b/docs/designs/dom0less-evtchn.md
> new file mode 100644
> index 0000000000..6a1b7e8c22
> --- /dev/null
> +++ b/docs/designs/dom0less-evtchn.md
> @@ -0,0 +1,96 @@
> +# Signaling support between two domUs on dom0less system
> +
> +## Current state: Draft version
> +
> +## Proposer(s): Rahul Singh, Bertrand Marquis
> +
> +## Problem Statement:
> +
> +The goal of this work is to define a simple signaling system between Xen guests
> +in dom0less systems.
> +
> +In dom0less system, we cannot make use of xenbus and xenstore that are used in
> +normal systems with dynamic VMs to communicate between domains by providing a
> +bus abstraction for paravirtualized drivers.
> +
> +One possible solution to implement the signaling system between domUs is based
> +on event channels.
> +
> +## Proposal:
> +
> +Event channels are the basic primitive provided by Xen for event notifications.
> +An event channel is a logical connection between 2 domains (more specifically
> +between dom1,port1 and dom2,port2). They essentially store one bit of
> +information, the event of interest is signalled by transitioning this bit from
> +0 to 1. An event is an equivalent of a hardware interrupt.
> +
> +Notifications are received by a guest via an interrupt from Xen to the guest,
> +indicating when an event arrives (setting the bit). 
I am a bit confused with the description. Are you trying to explain the 
event channel in layman term? If not, then event channel protocol is 
more complicated than that (in particular for fifo).
> Further notifications are
> +masked until the bit is cleared again.
I think "masked" is confusing here.

The event channel differentiate "mask" vs "pending". When sending an 
event, the pending bit will be set to 1. If it wasn't already pending 
and the mask bit is clear, then we will notify the guest.
If the pending bit is already set, then we will ignore.

In fact, the event channel is acting similarly to an edge interrupt. I 
wrote similarly, because IIRC they are behaving slightly differently 
(see [1] for more details).
> When a domain wants to wait for data it
> +will block until an event arrives, and then send an event to signal that data
> +has been consumed.
> Events are delivered asynchronously to guests and are
> +enqueued when the guest is not running.
s/guest/domain/ to stay consistent and also include dom0/hardware domain.

> +
> +Event channel communication will be established statically between two domU
> +guests before unpausing the domains after domain creation. Event channel
> +connection information between domUs will be passed to XEN via device tree
> +node.
Why are we limiting ourself to domUs?

> +
> +Under the /chosen node, there needs to be sub nodes with compatible
> +"xen,evtchn" that descibes the event channel connection between two domUs.
s/descibes/describes/

> +
> +The event channel sub-node has the following properties:
> +
> +- compatible
> +
> +    "xen,evtchn"
> +
> +- xen,evtchn
> +
> +    The property is four numbers of tuples of
> +    (local-port-domU1,domU1-phandle,local-port-domU2,domU2-phandle) where:
This is quite difficult to read. Can we add some space before/after each 
comma?
> +
> +    local-port-domU1 is an integer value that will be used to allocte local
s/allocte/allocate/

> +    port for domU1 to send an event notification to the remote domain.
The port will be used for sending but also receiving event notification.

Also, I would suggest to replace "remote domain" with "domU2". So it is 
more explicit.
> +
> +    domU1-phandle is a single phandle to an domain to which local-port-domU1
s/an domain/a domain/ I think.

> +    will be allocated.
> +
> +    local-port-domU2 is an integer value that will be used to allocte local
s/allocte/allocate/

> +    port for domU2 to send an event notification to the remote domain.
Same as above for "remote domain".

> +
> +    domU2-phandle is a single phandle to an domain to which local-port-domU2
> +    will be allocated.
> +
> +Example:
> +
> +    chosen {
> +        ....
> +
> +        domU1: domU1 {
> +            ......
> +        };
> +
> +        domU2: domU2 {
> +            ......
> +        };
> +
> +        evtchn@1 {
> +            compatible = "xen,evtchn";
> +            xen,evtchn = <0xa &domU1 0xb &domU2>;
> +        };
> +
> +        evtchn@2 {
> +            compatible = "xen,evtchn";
> +            xen,evtchn = <0xc &domU1 0xd &domU2>;
> +        };
> +    };
> +
> +In above example two event channel comunication will be established between
> +domU1 and domU2.
> +
> +    domU1 (port 0xa) <-----------------> domU2 (port 0xb)
> +    domU1 (port 0xc) <-----------------> domU2 (port 0xd)
> +
> +domU1 and domU2 can send the signal to remote domain via hypercall
> +EVTCHNOP_send(.) on local port.
Cheers,

[1] 
https://lore.kernel.org/lkml/dbfd87e9-48fc-f641-9e24-ddb6c4f61135@xxxxxxx/
--
Julien Grall