Re: [Xen-devel] [DOC v1] Xen transport for 9pfs

[Stefano Stabellini <stefano@xxxxxxxxxxx>]

On Thu, 1 Dec 2016, Dario Faggioli wrote:
> On Tue, 2016-11-29 at 15:34 -0800, Stefano Stabellini wrote:
> > ### Frontend XenBus Nodes
> > 
> >     num-rings
> >          Values:         <uint32_t>
> >     
> >          Number of rings. It needs to be lower or equal to max-rings.
> >     
> >     port-<num> (port-0, port-1, etc)
> >          Values:         <uint32_t>
> >     
> >          The identifier of the Xen event channel used to signal
> > activity
> >          in the ring buffer. One for each ring.
> >     
> >     ring-ref-<num> (ring-ref-0, ring-ref-1, etc)
> >
> blkif uses ring-ref%u, rather than ring-ref-%u (i.e., no dash before
> the index). Not a big deal, I guess, but I thought it could be nice to
> be a bit more uniform.

Sure, but in this case each ring-ref-%u is used to map a different ring.
That said, I can make the change.


> >          Values:         <uint32_t>
> >     
> >          The Xen grant reference granting permission for the backend
> > to
> >          map a page with information to setup a share ring. One for
> > each
> >          ring.
> > 
> So, this means there can be multiple rings (up to max-rings advertised
> by backend), _each_of_which_ has its own event channel, and
> _each_of_which_ can be a multi-page (in case ring_order > 0) ring... Is
> this correct?

That's right, and the size of each ring can be up to max-ring-page-order.


> If it is, what's the typical envisioned use of these multiple rings, if
> I can ask?

They are used to handle multiple read/write requests in parallel. Let's
assume that we configure the rings to be 8K each. Given that the data is
transmitted over the ring, each ring can hold only one outstanding 4K
write request (there is an header for each write request).

With two 8K rings, we can have two outstanding 4K write requests, each
of them processed in parallel on a different vcpu.

The system is completely configurable in terms of number and size of
rings, so a user can configure it to only export one 4K ring for
example or go as far as several 2MB rings.


> (Maybe it's obvious, I'm still not so familiar with these protocols..
> but I'm trying to improve, and that's why I'm here, so bear with me if
> possible. :-D)
> 
> > ## Ring Setup
> > 
> > The shared page has the following layout:
> > 
> >     typedef uint32_t XEN_9PFS_RING_IDX;
> > 
> >     struct xen_9pfs_intf {
> >             XEN_9PFS_RING_IDX in_cons, in_prod;
> >             XEN_9PFS_RING_IDX out_cons, out_prod;
> >     
> >             uint32_t ring_order;
> >             grant_ref_t ref[];
> >     };
> > 
> >     /* not actually C compliant (ring_order changes from socket to
> > socket) */
> >     struct ring_data {
> >         char in[((1 << ring_order) << PAGE_SHIFT) / 2];
> >         char out[((1 << ring_order) << PAGE_SHIFT) / 2];
> >     };
> >
> Sorry, what does "ring_order changes from socket to socket" mean?

Woops, copy/paste error from PVCalls. I meant "ring_order changes from
ring to ring".


> > The binary layout of `struct xen_9pfs_intf` follows:
> > 
> >     0         4         8         12        16        20
> >     +---------+---------+---------+---------+---------+
> >     | in_cons | in_prod |out_cons |out_prod |ring_orde|
> >     +---------+---------+---------+---------+---------+
> > 
> >     20        24        26      4092      4096
> >     +---------+---------+----//---+---------+
> >     |  ref[0] |  ref[1] |         |  ref[N] |
> >     +---------+---------+----//---+---------+
> > 
> > **N.B** For one page, N is maximum 1019 ((4096-20)/4), but given that
> > N
> > needs to be a power of two, actually max N is 512.
> >
> It may again be me being still too naive, but I'd quickly add at least
> another example, with the value of N computed for a multiple pages
> ring. Something like: "For 4 pages (i.e., ring_orfer=2), N is..."

For 4 pages, N is 4. N is the number of pages that make up the ring.

Maybe there is a misunderstanding, let me try to explain it again: each
page shared via xenstore contains information to handle one new ring,
including grant references for the pages making up the multipage ring
itself. I'll repeat: pages shared via xenstore are not used as a ring,
they are used to setup the rings, each page has the info to setup a new
ring.

The structure of these "setup pages" is `struct xen_9pfs_intf`. Each
page is completely separate and independent from the others. Given that
one page is just 4096 bytes, it can contain max 512 grant refs (see
calculation above). So the max size of one multipage ring is 512 pages =
2MB.

Does it make sense?

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel