Re: one-byte TCP writes wedging

[Anil Madhavapeddy <anil@xxxxxxxxxx>]

The long chain of 36-byte frags is consistent with the backend dropping it.  Does it work better if you restrict the total fragment chains size to just 10 or 11?

The first unexplained packet loss is a real alarm bell though.  The entire TCP retransmit code on our stack is just a canary that spots latent bugs elsewhere in the device stack :-)

-anil

On 25 May 2013, at 22:25, Balraj Singh <balraj.singh@xxxxxxxxxxxx> wrote:

In the particular test I am using I write 36 bytes of payload and use the Mirage equivalent of TCP_NODELAY.  This works for a bit but then suffers some packet loss (why? TBD) and triggers a rexmit.  The retransmitted packet is 1400+ bytes and is made up of a long chain of 36 byte io_pages.  I thought that it may be that the ring did not have enough slots to take all the chunks of the pkt.  Making the retransmitted pkt be the size of the original write improved it very significantly but it would still fail in the same way, tho less frequently.  I'm working on it - I see available txring slots vary, but I havent yet found a case where the slots are fully depleted or down to fewer than chunks that need to be written.  I'm still narrowing it down.

This test originally was with 1-byte writes, but that seemed to wedge even before the 1st data packet made it to the wire.  This may be because of the limitation Steven mentioned.  I think I'm getting close on the 36 byte write test, once this is figured out I'll try it with 1 byte writes again.

Balraj

On Sat, May 25, 2013 at 11:11 AM, Anil Madhavapeddy <anil@xxxxxxxxxx> wrote:

Balraj noticed that a stream of 1-byte writes on a TCP connection would cause netfront/netback to wedge.  This is obviously quite unrealistic, but a good regression test.

A quick chat with Steven Smith pointed out that some Linux netbacks had a limit on the number of fragments allowed (based on the skbuff chain limit size).  So you might be ending up with a situation where the backend drops the entire set of fragments, and the frontend is retransmitting all the time.

So if you modify our frontend to limit the fragment size to ~10 or so for any given packet, that might help.  On the other hand, if you're doing writes with a TCP segment size of 1, but still only 3-4 fragments (for the Ethernet/IP/TCP headers), then we have some other bug.  What does the Netif request look like, Balraj?

-anil