Slow http downloads-client rcv window never fills up
Hello all I'm troubleshooting a fault, and this one has me stumped. Client A is requesting a HTTP download (duration approx 10 seconds, 3 parallel threads) from Server B. During the 3 way handshake, both are advertising window size scaling (256 for the server and 16 for the client)-so all good on that front. I'm expecting a speed of circa 550Mb/sec-the tests are automated, and run every hour,24/7. Sometimes I get full speed, sometimes its half! Looking at the TCP Stream Graphs/Windows scaling on WS on the "bad" downloads, there is a very slow ramp up on the RCV window (it takes a full 3 seconds to reach the full capability (604,000))-and as a result, the throughput is very poor-about 240Mb/sec. On the good downloads, the RCV window (again approx 604,000) is met within 0.2 second-and stays there for the full 10 seconds of the test. Looking at the captures, I can see a stream of packets from the server, then an ACK is sent-so normal so far. But when I look at the bytes in flight from the server, the client "bucket" only ever gets to about 230,000, and then the client ACK's-but the advertised window size is still 604,000? Also, the bytes in flight starts off from a really low value, about 10,000-and then climbs to about the 230,000, ACK is sent-and then the process repeats itself. On the good trace, the client bucket get up to just shy of the 600,000 mark,ACK is sent, and the stream continues (from about 400,000, straight out of the gate). There are no to none TCP retransmissions, in fact very little tcp.analysis.flags pop up (a few DUP_ACK's) on the bad trace.The only thing I can see on the bad trace compared to the good trace is a rise in RTT (but max is around 40ms, so not huge?). It might be an internal buffer/CPU issue on the client side but looking for ideas or other items in the trace, which would give me a clue :-)
Edit-on Graham's advice, anom file example of the "bad" trace below
Cheers
While screenshots help, they are no substitute for an actual capture that can be investigated using our favourite tool Wireshark. Pictures and capture files can be uploaded to a public share, e.g. Google Drive, DropBox etc. and a link to the posted back here.
Thanks for the comment Graham- unfortunately, the trace is from a customer so I'm a bit hesitant to put it up on a public forum :-(
cabs
Then TraceWrangler may help to anonymise the file. TraceWrangler is written by @Jasper.
Can you make a capture on the client end?
Would be interesting to see how the large packets (some up to 27580 bytes) are arriving at the client.
The client and server both picked a MSS of 1460 but the server is expecting the NIC or something in between to fragment the packets.
Is it possible to get a capture on something other than the client or server?
Hi Bubba-thanks for the reply. Unfortunately, the client device is locked down, and remote from me, so I cant get a capture from that end. Anything else I could do?
cabs