Why no data flow after TCP 3 way handshake?

No-data

asked 2019-05-16 14:46:42 +0000

batteryworm
3 ●3 ●5 ●5

updated 2019-05-16 14:51:00 +0000

grahamb

flag of United Kingdom of Great Britain and Northern Ireland

23835 ●4 ●973 ●227 https://www.wireshark.org

I have an application that uses API to connect to server separated by a WAN. The problem is that it can establish a 3 way TCP handshake, but after that could not connect at the API level and after some time (20 seconds), the session disconnects. A tcpdump is taken at the client end who initiates the connection; at the same time a tcpdump is taken at the server end.

At the client end, we can see client sending SYN, then receiving SYN-ACK from the server, and client sends ACK. Subsequent to this TCP session establishment, client then sends data (32 bytes) with PUSH, ACK. But there is no response from server. After 300 ms, client retransmit the data packet (same packet with 32 bytes) ie. retransmit. Again no response from server. After 600ms, client retransmit same data packet. No response from server. After 1.2 seconds, client retransmit; no response from server. This goes on for another few times, after about 11 seconds, client sends a RST to server; no response from server. Again client sends RST to server and no response. This repeats several time. Until finally at about 20 seconds, client receives RST from server.
At the server end, we can see the SYN packet from client. Server response with SYN-ACK. Then server receives ACK from client so the TCP 3 way handshake is established. Subsequent to this server does not receive any more packets from client. After 20 seconds, server sends RST to client. What causes this behaviour?

edit retag flag offensive close merge delete

add a comment

2 Answers

Sort by » oldest newest most voted

answered 2019-05-16 15:08:33 +0000

SYN-bit

18528 ●9 ●339 ●255 https://SYN-b.it

Something in the path between the client and the server is blocking the HTTP request. As the same packet is dropped over and over again, this is not random packet-loss. It could be an IPS (Intrusion Prevention System) or maybe a WAF (Web Application Firewall). Or it could be a Loadbalancer that does initiate the 3-way handshake but has to process the HTTP request first and somehow can't. It could also be a NGFW (Next Generation Firewall) that tries to understand the request, but is failing.

Can you describe the infrastructure between the client and the server?
Can you show the 32 bytes of the HTTP request? 32 bytes are a bit less than expected, so maybe the request itself is not fully compliant and that is the cause the intermediate device is blocking it.

edit flag offensive delete link

Comments

To clarify, this is not a HTTP protocol. It is a proprietary protocol using port 23027. I will attempt to describe the infrastructure between the server and the client but because it crosses companies, I only have details on my company A end which is the server end. I don't have visibility on the other company B end.

Server at Company A - edge switches - core switches - distribution switches - firewall - external facing switches - telco mux - MetroE point to point telco network (2Mbps) - Company B site telco mux - cisco router (belonging to company A) - RJ45 LAN cable handoff to company B network infrastructure - company B's network infrastructure - company B's client.

Presumably company B's network infrastructure have some sort of firewall as well. Not sure if it makes sense, but here are the 32 bytes in hexadecimal:

0C 00 00 00 00 00 00 00
40 00 00 00 ...

(more)

batteryworm ( 2019-05-16 15:23:45 +0000 )edit

Just to add. I took a closer look at company B pcap. Noticed at the MAC layer the server seems to be communicating to two gateways:

TCP SYN - MAC address of company B server ---> gateway MAC address (manufacturer Palo Alto)
TCP SYN/ACK from Company A - MAC addr of another gateway (manufacturer Netscreen) --> MAC addr of company B server
TCP ACK - MAC address of company B server ---> gateway MAC address (manufacturer Palo Alto).

So potentially the communications path from company B server to company A goes via different path from the return communications from company A to company B server?

batteryworm ( 2019-05-16 15:47:32 +0000 )edit

Good catch on the mac-addresses. If the traffic is not flowing back-and-forth through the same Firewall, you will most certainly run into problems. Have Company B check it's routing for this traffic flow.

Then: As Palo Alto is a NGFW, it needs to be setup to not do any ApplicationIdentification for this proprietary protocol (or create a custom App-ID)

SYN-bit ( 2019-05-16 17:43:36 +0000 )edit

And sorry for assuming HTTP, I read API and since 90% of API's are HTTP(S) these days, I made the wrong assumption... :-)

SYN-bit ( 2019-05-16 17:44:39 +0000 )edit

Thanks for your suggestions. A call to Company B network and server admin resolved the issue. It was due to a wrong routing config at their server end. After fixing the routing issue, all traffic passes through their Netscreen and the data traffic was able to flow subsequently.

batteryworm ( 2019-05-18 11:32:41 +0000 )edit

see more comments

answered 2022-12-18 22:26:51 +0000

cartesius
1

Hello,

I happen to have the same problem on a customer's network.

Client machines and Server are separated by a VPN S2S link. The communication happens over 2 TCP ports, of them being 80 and one non standard value, let's say (under 1000).

I saw that from time to time, the TCP handshakes pass through, but not the data afterwards, just like the case here. Also the TTL is changed on the way, from 128 - x as expected value to 255 - y.

I suggested to check the routing and the Application Id. I keep you posted with the evolution.

edit flag offensive delete link

add a comment