[Xen-devel] [PATCH v9 6/7] docs: documentation about static shared memory regions

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

From: Zhongze Liu <blackskygg@xxxxxxxxx>

Author: Zhongze Liu <blackskygg@xxxxxxxxx>

Add docs to document the motivation, usage, use cases and other
relevant information about the static shared memory feature.

This is for the proposal "Allow setting up shared memory areas between VMs
from xl config file". See:

  https://lists.xen.org/archives/html/xen-devel/2017-08/msg03242.html

The corresponding device tree binding is described by
Documentation/devicetree/bindings/reserved-memory/xen,shared-memory.txt.

Signed-off-by: Zhongze Liu <blackskygg@xxxxxxxxx>
Signed-off-by: Stefano Stabellini <stefanos@xxxxxxxxxx>

Cc: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
Cc: Wei Liu <wei.liu2@xxxxxxxxxx>
Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>
Cc: Julien Grall <julien.grall@xxxxxxx>
Cc: xen-devel@xxxxxxxxxxxxx

---
Changes in v9:
- rename master to owner and slave to borrower
- add reference to device tree spec in commit message and in the doc

Changes in v6:
- add clarifications on memory allocation

Changes in v5:
- fix typos
---
 docs/man/xl-static-shm-configuration.pod.5 | 266 +++++++++++++++++++++++++++++
 docs/man/xl.cfg.pod.5.in                   |   8 +
 docs/misc/xenstore-paths.markdown          |  47 +++++
 3 files changed, 321 insertions(+)
 create mode 100644 docs/man/xl-static-shm-configuration.pod.5

diff --git a/docs/man/xl-static-shm-configuration.pod.5 
b/docs/man/xl-static-shm-configuration.pod.5
new file mode 100644
index 0000000..69c7577
--- /dev/null
+++ b/docs/man/xl-static-shm-configuration.pod.5
@@ -0,0 +1,266 @@
+=head1 NAME
+
+xl-static-shm-configuration - XL Static Shared Memory Configuration Syntax
+
+
+(B<NOTE>: This is currently only available to ARM guests.)
+
+=head1 DESCRIPTION
+
+The static_shm option allows users to statically setup shared memory regions
+among a group of VMs, enabling guests without grant table support to do
+shm-based communication. See
+Documentation/devicetree/bindings/reserved-memory/xen,shared-memory.txt
+in Linux for the corresponding device tree binding.
+
+Every shared region is:
+
+=over 4
+
+* Uniquely identified by a string that is no longer than 128 characters, which
+is called an B<identifier> in this document.
+
+* Backed by exactly one domain, which is called a B<owner> domain, and all
+the other domains who are also sharing this region are called B<borrower>s.
+
+=back
+
+=head1 SYNTAX
+
+This document specifies syntax of the static shared memory configuration in
+the xl config file. It has the following form:
+
+    static_shm = [ "SSHM_SPEC", "SSHM_SPEC", ... ]
+
+where each C<SSHM_SPEC> is in this form:
+
+    [<key>=<value>,]*
+
+Valid examples of C<SSHM_SPEC> are:
+
+    id=ID1, begin=0x100000, size=0x100000, role=owner, cache_policy=x86_normal
+    id=ID1, offset = 0, begin=0x500000, size=0x100000, role=borrower, prot=rw
+    id=ID2, begin=0x300000, size=0x100000, role=owner
+    id=ID2, offset = 0x10000, begin=0x690000, size=0x110000, role=borrower
+    id=ID2, offset = 0x10000, begin=0x690000, size=0x110000, role=borrower
+
+These might be specified in the domain config file like this:
+
+    static_shm = ["id=ID2, offset = 0x10000, begin=0x690000, size=0x110000,
+role=borrower"]
+
+
+More formally, the string is a series of comma-separated keyword/value
+pairs. Each parameter may be specified at most once. Default values apply if
+the parameter is not specified.
+
+=head1 Parameters
+
+=over 4
+
+=item B<id>
+
+=over 4
+
+=item Description
+
+The unique identifier of the shared memory region.
+
+Every identifier could appear only once in each xl config file.
+
+=item Supported values
+
+A string that contains alphanumerics and "_"s, and is no longer than 128
+characters.
+
+=item Default value
+
+None, this parameter is mandatory.
+
+=back
+
+=item B<begin>/B<size>
+
+=over 4
+
+=item Description
+
+The boundaries of the shared memory area.
+
+=item Supported values
+
+Same with B<offset>.
+
+=item Default Value
+
+None, this parameter is mandatory.
+
+=back
+
+=item B<offset>
+
+=over 4
+
+=item Description
+
+Can only appear when B<role> = borrower. If set, the address mapping will not
+start from the beginning the backing memory region, but from the middle
+(B<offset> bytes away from the beginning) of it. See the graph below:
+
+With B<offset> = 0, the mapping will look like:
+
+  backing memory region:     #########################################
+                             |                       |
+                             |                       |
+                             |                       |
+                             V                       V
+  borrower's shared region:  #########################
+
+With B<offset> > 0:
+
+  backing memory region:      #########################################
+                              |<-- offset -->||                       |
+                                              |                       |
+                                              |                       |
+                                              V                       V
+  borrower's memory region:                   #########################
+
+=item Supported values
+
+Decimals or hexadecimals with a prefix "0x", and should be the multiple of the
+hypervisor page granularity (currently 4K on both ARM and x86).
+
+=item Default value
+
+0x0
+
+=back
+
+=item B<role>
+
+=over 4
+
+=item Description
+
+The backing area would be taken from one domain, which we will mark as
+the "owner domain", and this domain should be created prior to any
+other borrower domains that depend on it. The owner's shared memory range
+is NOT allocated in addition to its regular memory. Hence, it is usually
+a good idea to choose a subrange of the regular guest memory allocation,
+which starts at GUEST_RAM0_BASE, see xen/include/public/arch-arm.h.
+
+The "borrower domain" maps the memory of the owner. The address of said
+mapping should not be overlapping with the normal memory allocation of
+the borrower domain.
+
+This argument specifies the role of this domain.
+
+=item Supported values
+
+owner, borrower
+
+=item Default value
+
+borrower
+
+=back
+
+=item B<prot>
+
+=over 4
+
+=item Description
+
+When B<role> = owner, this means the largest set of stage-2 permission flags
+that can be granted to the borrower domains. When B<role> = borrower, this 
means the
+stage-2 permission flags of the shared memory area.
+
+=item Supported values
+
+Currently only 'rw' is supported.
+
+=item Default value
+
+rw
+
+=back
+
+=item B<cache_policy>
+
+=over 4
+
+=item Description
+
+The stage-2 cacheability/shareability attributes of the shared memory area.
+This can only appear when B<role> = owner.
+
+=item Supported values
+
+Currently, only the following policy is supported:
+
+=over 4
+
+=item B<ARM_normal>
+
+Only applicable to ARM guests. This would mean Inner and Outer Write-Back
+Cacheable, and Inner Shareable.
+
+=back
+
+=item Default value
+
+ARM_normal
+
+=back
+
+=back
+
+=head1 TYPICAL USAGE
+
+A typical procedure of setting up a shared mem region among several VMs is:
+
+=over 4
+
+1. Add a static_shm option to the owner domain's xl config file, assign an
+B<ID> to it and mark it's B<role> as owner, and set up the boundaries, prot
+flag, and B<cache_policy> appropriately.
+
+2. Add a static_shm option to every borrower domain's xl config file, set
+their B<ID> to the same value as the owner's, and set up the B<offset>,
+boundaries and prot flag appropriately.
+
+3. Create the owner domain.
+
+4. Create the borrowers.
+
+=back
+
+Remember that the owner domain must be created before any borrower domains 
could
+be created, for the borrowers depend on the memory pages backed by their owner.
+
+=head1 Example
+
+Suppose that we have 3 domains: vm1~vm3. And we want to setup two shared
+regions, say, ID1 and ID2, among the three domains, with the following address
+mapping:
+
+   ID1: (vm1 : 0x100000~0x200000) <=====> (vm2 : 0x500000~0x600000)
+   ID2: (vm1 : 0x310000~0x400000) <=====> (vm3 : 0x690000~0x800000)
+
+According to the syntax defined above, the xl config files of the three domains
+should contains the following content:
+
+In xl config file of vm1:
+  static_shm = [ "id=ID1, begin=0x100000, size=0x100000, role=owner,
+cache_policy=x86_normal, prot=rw",
+"id=ID2, begin=0x300000, size=0x100000, role=owner" ]
+
+In xl config file of vm2:
+  static_shm = [ "id=ID1, offset=0, begin=0x500000, size=0x100000,
+role=borrower, prot=rw" ]
+
+In xl config file of vm3:
+  static_shm = [ "id=ID2, offset=0x10000, begin=0x690000,
+size=0x110000, role=borrower" ]
+
+After that, just create vm1 first, and then create vm2 and vm3 in any order.
diff --git a/docs/man/xl.cfg.pod.5.in b/docs/man/xl.cfg.pod.5.in
index b1c0be1..360237f 100644
--- a/docs/man/xl.cfg.pod.5.in
+++ b/docs/man/xl.cfg.pod.5.in
@@ -278,6 +278,14 @@ memory=8096 will report significantly less memory 
available for use
 than a system with maxmem=8096 memory=8096 due to the memory overhead
 of having to track the unused pages.
 
+=item B<static_shm=[ "SSHM_SPEC", "SSHM_SPEC", ... ]>
+
+Specifies the static shared memory regions of this guest. Static shared
+memory regions enables guests to communicate with each other through
+one or more shared memory regions, even without grant table support.
+Currently, this only works on ARM guests.
+See L<xl-static-shm-configuration(5)> for more details.
+
 =back
 
 =head3 Guest Virtual NUMA Configuration
diff --git a/docs/misc/xenstore-paths.markdown 
b/docs/misc/xenstore-paths.markdown
index 33d2819..b66fae4 100644
--- a/docs/misc/xenstore-paths.markdown
+++ b/docs/misc/xenstore-paths.markdown
@@ -174,6 +174,14 @@ than this amount of RAM.
 
 The size of the video RAM this domain is configured with.
 
+#### ~/static_shm/[_a-zA-Z0-9]+/role = ("owner"|"borrower") []
+
+(Note: Currently, this will only appear on ARM guests.)
+
+The role of this domain in the static shared memory region whose id matches
+the `[_a-zA-Z0-9]+` part in the path. (Described in the manpage
+**xl-static-shm-configuration(5)**).
+
 #### ~/device/suspend/event-channel = ""|EVTCHN [w]
 
 The domain's suspend event channel. The toolstack will create this
@@ -556,6 +564,45 @@ type. The name of each backend directory is the same as 
the backend type
 
 Contains the PIDs of the device models running on the domain.
 
+#### /libxl/static_shm/[_a-zA-Z0-9]+/* []
+
+(Note: Currently, this will only appear on ARM guests.)
+
+The following paths contain backing memory parameters of a static shared memory
+whose id matches the `[_a-zA-Z0-9]+` part in the path. Their formats and
+meanings are the same as those in an xl config file, described in the manpage
+**xl-static-shm-configuration(5)**.
+
+* begin/size: the boundary of the backing memory region.
+* prot: the largest set of stage-2 permission flags that can be granted to
+  the borrower domains.
+* cache_policy: the stage-2 cacheability/shareability attributes of the backing
+  memory region.
+
+The following paths contain run-time information about the static shared memory
+region.
+
+* owner: the domid of the backing domain.
+* borrowers: information about the borrowers that are sharing the region, see
+  ** /libxl/static_shm/[_a-zA-Z0-9]+/borrowers/$DOMID/* ** below.
+* usercnt: An integer. This is the reference count of the backing memory 
region,
+  including the owner domain itself. When this value reachies 0, the backing
+  memory region will be freed.
+
+#### /libxl/staitc_shm/[_a-zA-Z0-9]+/borrowers/$DOMID/* []
+
+(Note: Currently, this will only appear on ARM guests.)
+
+The following paths contain static shared memory region parameters of a 
borrower
+domain. Their formats and meanings are the same as those in xl config files,
+described in the manpage **xl-static-shm-configuration(5)**.
+
+* begin/size: the boundary of the shared memory region.
+* prot: the stage-2 permission flags of the shared memory area.
+* offset: when mapping the backing memory region to the borrower's memory 
space,
+  the mapping will start from offset bytes after the beginning of the backing
+  memory region.
+
 ## Virtual Machine Paths
 
 The /vm/$UUID namespace is used by toolstacks to store various
-- 
1.9.1


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel