Hello, I have an application with a custom protocol running on a customer's wireless network that connects via TCP to AWS. The client randomly disconnects with an RST. The customer tells me their wireless network is sketchy. Can any more information be gathered from the Wireshark trace? Before the disconnect I see a series of ACKs from the client for the last server communication, followed by a 70 second delay, then an RST from the client. In the cases I have traced, there is always about a 70 second delay between the last client ACK and the RST. Any thoughts on what is going on here? The Wireshark trace is below. Thanks in advance for your help! Regards, Cameron asked 18 Jul '14, 12:14  dcdeheer |
One Answer:
There is a "WatchGuard" proxy next to the client. Looking at the SYN packets they clearly show they have been modified from a windows fingerprint to a linux-ish one (TTL=64). Also, the ip.id of the packets are different. So it is probably an idle-timer that pops after having not seen any traffic within 60 seconds kicking the session out by sending a RST to each end. answered 21 Jul '14, 14:06  mrEEde |
Your analysis looks right to me. That "series of ACKs" from the client is in response to a 65236-byte segment immediately followed by a 29809 segment from the server. That looks unusually large to me, but that might be normal in other people's networks.
In any case if you look at the sequence numbers, you'll see that the client finished successfully acknowledging all of the data sent by the server in packet #511. It looks to me like both the client and the server are satisified with what they received, and the 70 seconds is simply a idle timeout trigger on the client.
Was there an application-layer problem captured in this trace? I don't see anything wrong, outside some minor packet loss.
Hello, That is what I thought, thanks very much for your feedback. Nothing out of the ordinary is going on at the application level, and there is no application-level timeout for inactivity. I had them install another client on their primary network (no wireless) and we're seeing the same disconnect, after about 67 seconds. I am leaning toward a switch on their site with a 60 second inactivity timeout, or something along these lines. Thanks again for your help. Regards, Cameron
TCP idle timeouts can vary widely, and so can the mechanisms to control them. You may not see timeout properties from an application perspective, but they're more likely somewhere in the TCP/IP properties on the client. These properties are not always easy to find - if they are even accessible. Some closed systems don't even provide a mechanism to adjust them.
I doubt it's a switch since that's a L2 device. But you're right, it could be a firewall, or any number of proxy-type devices sitting in the middle of the conversation. If there's no proxy intermediary, I'd suspect the idle timer is in the client TCP/IP stack somewhere.
Hello smp, Thanks so much for your help. Both client and server are Windows, I could not find any documentation on the Windows TCP/IP stack that indicated a configuration for idle timeout. The client is our software, and it does not have any idle timeout in its protocol. It sends a heartbeat message every 5 minutes to the server to keep the connection alive, the server stops listening if the heartbeat doesn't show up for a while (30 minutes). The disconnect we are seeing happens at one minute. I had the customer get me a wireshark capture of the client at the same time i was capturing the server:
client: https://www.cloudshark.org/captures/b5fce301d9e1 server: https://www.cloudshark.org/captures/bbfa28dd02ee
Both sides are receiving the RST. The only thing I noticed out of the ordinary on the client was a lot of 'reassembly error' messages. Could these have something to do with the RST? Thanks again for your help! Regards, Cameron