Connections from one location being blocked (RST, ACK)

asked 2019-03-18 20:52:34 +0000

updated 2019-03-19 08:43:44 +0000

grahamb gravatar image

We have a remote client attempting to connect to one of our web servers and they are unable to connect from their primary data center. Their backup data center is able to connect successfully. From what I can tell a RST packet is being sent but we are unable determine why.

Our side Public 3.88.123.54 Private 10.201.48.153 Remote side Public 69.25.46.11 Private 172.25.19.35

This is the sequence we see on our side

1538    56.948946   69.25.46.11 10.201.48.153   TCP 74  48694 → 443 [SYN] Seq=0 Win=29200 Len=0 MSS=1380 SACK_PERM=1 TSval=1094375775 TSecr=0 WS=128
1539    56.948996   10.201.48.153   69.25.46.11 TCP 74  443 → 48694 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=8961 WS=256 SACK_PERM=1 TSval=44542844 TSecr=1094375775
1540    56.951208   69.25.46.11 10.201.48.153   TCP 66  48694 → 443 [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSval=1094375778 TSecr=44542844
1541    56.994988   10.101.2.174    10.201.48.153   UDP 54  51486 → 3389 Len=12
1542    57.030570   69.25.46.11 10.201.48.153   TCP 54  48694 → 443 [RST, ACK] Seq=1 Ack=1 Win=29312 Len=0

This is the sequence on the remote side

2   66.273162   172.25.19.35    3.88.123.54 TCP 162 48694 → 443 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=1094375775 TSecr=0 WS=128
3   66.273197   172.25.19.35    3.88.123.54 TCP 179 [TCP Out-Of-Order] 48694 → 443 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=1094375775 TSecr=0 WS=128
4   66.275413   3.88.123.54 172.25.19.35    TCP 179 443 → 48694 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1380 WS=256 SACK_PERM=1 TSval=44542844 TSecr=1094375775
5   66.275420   3.88.123.54 172.25.19.35    TCP 179 [TCP Out-Of-Order] 443 → 48694 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1380 WS=256 SACK_PERM=1 TSval=44542844 TSecr=1094375775
6   66.275698   172.25.19.35    3.88.123.54 TCP 171 48694 → 443 [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSval=1094375778 TSecr=44542844
7   66.275703   172.25.19.35    3.88.123.54 TCP 171 [TCP Dup ACK 6#1] 48694 → 443 [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSval=1094375778 TSecr=44542844
8   66.353152   172.25.19.35    3.88.123.54 TLSv1   382 Client Hello
9   66.353166   172.25.19.35    3.88.123.54 TCP 382 [TCP Retransmission] 48694 → 443 [PSH, ACK] Seq=1 Ack=1 Win=29312 Len=211 TSval=1094375855 TSecr=44542844
10  66.354012   3.88.123.54 172.25.19.35    TCP 159 443 → 48694 [RST, ACK] Seq=1 Ack=212 Win=58624 Len=0
11 ...
(more)
edit retag flag offensive close merge delete

Comments

Hi, which version(s) of TLS is(are) supported by the web server and which version is the client trying to use exactly?

SomeRando gravatar imageSomeRando ( 2019-03-20 15:49:08 +0000 )edit

Hello, the server supports only TLSversion 1.2 and the client is trying to use 1.2. Below is an output of their open ssl connection that has the ssl protocol specified

vsl-wapp0001a ~]# openssl s_client -connect 3.88.123.54:443 CONNECTED(00000003)

write:errno=104

no peer certificate available

No client certificate CA names sent

SSL handshake has read 0 bytes and written 289 bytes

New, (NONE), Cipher is (NONE) Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE No ALPN negotiated SSL-Session: Protocol : TLSv1.2 Cipher : 0000 Session-ID: Session-ID-ctx: Master-Key: Key-Arg : None Krb5 Principal: None PSK identity: None PSK identity hint: None Start Time: 1553178233 Timeout : 300 (sec) Verify return code: 0 (ok)

Jubaru125 gravatar imageJubaru125 ( 2019-03-22 13:59:15 +0000 )edit

Okay, thanks. I am not really an expert with SSL/TLS, but i don't think the issue lies here, if there is no version mismatch. What I see is, that something along the path is decreasing the MSS for both sides to 1380. Do you have visibility into what it could be? The MSS decrease is not an issue by itself but that device could contribute to the problem (adding gurther modifications/drop packets) Another thing I noticed is that the client hello never reaches the destination - something is dropping this packet. It si possible the a device cannot handle it for some reason. The third observation, that could be lead, is that both ends think that the other side sent the TCP RST. Again, just a theroy, but some smart device may be sending the reset on behalf of the endpoints.I admit this is a somewaht curious ...(more)

SomeRando gravatar imageSomeRando ( 2019-04-13 18:19:56 +0000 )edit