The packet size setting is being recognized by the OAK. You can see it in the size of those packets. your message overhead is 65 or 66 bits, and in fact, when you set it to 9000, it's probably just setting a toggle for "jumbo frames" or "standard frames" because you're pushing closer to 15,000 bits per packet. Even when you have it set to standard, you're pushing 2114 bits per packet - 66 bits for overhead = exactly 2kb per packet.
What's happening after that, as a result of the higher packet size, I am not sure, but this DOES seem like a class case of Bill the Cat! I"m hoping I'm not the only one old enough to get that reference…but what I mean is, ACK!
This is quite typically the response that gets seen when using jumbo packets in a highly dynamic traffic environment, rather than a singularly purposed datacenter file transfer server. In fact, I would bet that everything slows down for you, and this form page gets especially annoying as you're waiting for responses to clear and elements to load.
Nagle, himself, expressed frustration that the delayed ack and jumbo packets were often combined, but usually with disasterous results…but don't trust anything I say…take it from the guy himself, as you can read from the quote below. But I would first try to disable the jumbo packets, back to 1500, and leave ackdelay on. If it doesn't get better, switch them. We might also have yackack or no_delay options if other elements on your network require the jumbo packets…but usually they don't.
The idea of the Nagle algorithm was to prevent more than one undersized packet from being in transit at a time. The idea of delayed ACKs (which came from Berkeley) was to avoid sending a lone ACK for each character received when typing over a Telnet connection with remote echo, by waiting a fixed period for traffic in the reverse direction upon which the ACK could be piggybacked.
The interaction of the two algorithms is awful. If you do big send, big send, big send, that works fine. If you do send, get reply, send, get reply, that works fine. If you do small send, small send, get reply, there will be a brief stall. This is because the second small send is delayed by the Nagle algorithm until an ACK comes back, and the delayed ACK algorithm adds 0.5 second or so before that happens.
A delayed ACK is a bet. The TCP implementation is betting that data will be sent shortly and will make it unnecessary to send a lone ACK. Every time a delayed ACK is actually sent, that bet was lost. The TCP spec allows an implementation to lose that bet every time without turning off delayed ACKs. Properly, delayed ACKs should only turn on when a few unnecessary ACKs that could have been piggybacked have been sent in a row, and any time a delayed ACK is actually sent, delayed ACKs should be turned off again. There should have been a counter for this.
Unfortunately, delayed ACKs went in after I got out of networking in 1986, and this was never fixed. Now it's too late.
John Nagle
Share
Improve this answer
Follow
answered May 21, 2013 at 6:06
John Nagle79566 silver badges5
