| Increasing the ring size requires patches to Xen's dom0 as well, and may not be entirely trivial due to the increase from a single page to multiple pages. I know that these changes did happen for the blkback (and Thomas Leonard added indirect block descriptor support to our blkfront to take advantage), but I don't know if they are supported on dom0's netback (which is much more latency sensitive).
Dave, any idea of the status here?
-anil
Thanks.
What is the right way of increasing the size of allocated ring in
netif? I found that in high packet rate, that would be a limiting
factor.
On 20/12/14 17:31, Anil Madhavapeddy wrote:
This is an excellent point -- applying backpressure going back to
dom0 would help balance out load once multiple netfronts come into
play.
However, the issue with the ring handler being
Lwt-aware is that it's not clear what to do if the function
blocks. All subsequent acks would also be blocked since the
ring has to be processed in order (although the responses on the
ring themselves may be out of order).
I think we should take a look at the Ring API itself
once Dave's refactoring has landed and is released. A more
concurrent API could push the backpressure to the layer
immediately after the Ring ack.
-anil
I am following up to share
some experience from when I was experimenting with
the original mirage switch. I also have a few design
suggestions, but maybe they are invalid.
I think the problem mentioned in the
original mail stems from the way packets are
handled by a xen unikernel. if you check https://github.com/mirage/mirage-net-xen/blob/master/lib/netif.ml
on line 300, for each new packet arriving on a
VIF, the code will start a thread and ignore the
result. As a result, if you start sending lots of
packets to a unikernel and the main processing
pipeline cannot keep up with the rate, then new
packets are allocated and delegated to an handling
thread, but these threads will never return and
they will create a huge backlog, which at some
point will exhaust memory.
From a design point of view the best
approach to solve this problem, I think, is to
create a pool of threads with a fixed size. If
the pool doesn't have a free thread, then the
packet is dropped. Alternatively, the driver can
be redesigned to create backpressure to the
netback and force packets to drop in the Dom0,
instead of the unikernel, and thus reduce a bit
the cpu load. In the past I tried to rewrite the
rx_poll method, But the problem is that the
function that is passed as a handler to ack
new pages from the netback is not lwt aware,
thus you cannot easily integrate some of the lwt
asyncronicity in the processing pipeline.
This message and any attachment are intended solely for the addressee
and may contain confidential information. If you have received this
message in error, please send it back to me, and immediately delete it.
Please do not use, copy or disclose the information contained in this
message or in any attachment. Any views or opinions expressed by the
author of this email do not necessarily reflect the views of the
University of Nottingham.
This message has been checked for viruses but the contents of an
attachment may still contain software viruses which could damage your
computer system, you are advised to perform your own checks. Email
communications with the University of Nottingham may be monitored as
permitted by UK legislation.
|
