Re: [Xen-devel] Interesting observation with network event notification and batching

[annie li <annie.li@xxxxxxxxxx>]

On 2013-6-17 18:35, Wei Liu wrote:
> On Mon, Jun 17, 2013 at 10:38:33AM +0100, Ian Campbell wrote:
> > On Sun, 2013-06-16 at 10:54 +0100, Wei Liu wrote:
> > > > > Konrad, IIRC you once mentioned you discovered something with event
> > > > > notification, what's that?
> > > > They were bizzare. I naively expected some form of # of physical NIC
> > > > interrupts to be around the same as the VIF or less. And I figured
> > > > that the amount of interrupts would be constant irregardless of the
> > > > size of the packets. In other words #packets == #interrupts.
> > > It could be that the frontend notifies the backend for every packet it
> > > sends. This is not desirable and I don't expect the ring to behave that
> > > way.
> > It is probably worth checking that things are working how we think they
> > should. i.e. that netback's calls to RING_FINAL_CHECK_FOR_.. and
> > netfront's calls to RING_PUSH_..._AND_CHECK_NOTIFY are placed at
> > suitable points to maximise batching.
> >
> > Is the RING_FINAL_CHECK_FOR_REQUESTS inside the xen_netbk_tx_build_gops
> > loop right? This would push the req_event pointer to just after the last
> > request, meaning the net request enqueued by the frontend would cause a
> > notification -- even though the backend is actually still continuing to
> > process requests and would have picked up that packet without further
> > notification. n this case there is a fair bit of work left in the
> > backend for this iteration i.e. plenty of opportunity for the frontend
> > to queue more requests.
> >
> > The comments in ring.h say:
> >   *  These macros will set the req_event/rsp_event field to trigger a
> >   *  notification on the very next message that is enqueued. If you want to
> >   *  create batches of work (i.e., only receive a notification after several
> >   *  messages have been enqueued) then you will need to create a customised
> >   *  version of the FINAL_CHECK macro in your own code, which sets the event
> >   *  field appropriately.
> >
> > Perhaps we want to just use RING_HAS_UNCONSUMED_REQUESTS in that loop
> > (and other similar loops) and add a FINAL check at the very end?
> >
> > > > But it was odd and I didn't go deeper in it to figure out what
> > > > was happening. And also to figure out if for the VIF we could
> > > > do something of #packets != #interrupts.  And hopefully some
> > > > mechanism to adjust so that the amount of interrupts would
> > > > be lesser per packets (hand waving here).
> > > I'm trying to do this now.
> > What scheme do you have in mind?
> Basically the one you mentioned above.
>
> Playing with various event pointers now.
Did you collect data of how much requests netback processes when 
req_event is updated in RING_FINAL_CHECK_FOR_REQUESTS? I assume this 
value is pretty small from your test result. How about not updating 
req_event every time when there is no unconsumed request in 
RING_FINAL_CHECK_FOR_REQUESTS?
Thanks
Annie
> Wei.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel