Ask Your Question
0

Help analyzing TCP connection sequence

asked 2019-05-08 06:49:03 +0000

EricSnijders gravatar image

Hi all, I'm trying to understand the complete flow of a TCP sequence and i think our application is sometimes not closing TCP sessions right but i'm not 100% sure.

Please see the following: image description

My concerns are with the closing stage of the TCP connection.

  • Packet #2262 is a [FIN,ACK] but shouldn't there be a [FIN] from the other side first?
  • Packet #2265 shows the same but the other way around
  • I can't place those random [ACK]'s at #2264 and #2267.

Why am i asking? This is a capture from our firewall. Apparently, something in this connection is triggering a "new connection" on our firewall. I'm suspecting it has something to do with this TCP connection not being closed gracefully and [ACK]'s coming after the [FIN,ACK]'s but i'm not 100% sure.

Would be great to get some help!

Thanks in advance,

Eric

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
4

answered 2019-05-08 09:12:16 +0000

SYN-bit gravatar image

Hi Eric,

Packet #2262 is a [FIN,ACK] but shouldn't there be a [FIN] from the other side first?

Either side may close the connection by sending a FIN/ACK. This tells the other side that it won't send any more data.

Packet #2265 shows the same but the other way around

This is where the other side sends it's own FIN/ACK to indicate that it too will not send any more data. This means the connection is (almost) closed.

I can't place those random [ACK]'s at #2264 and #2267.

Both FIN/ACK packets need to be acknowledged, as both sides need to know their FIN/ACK was correctly received. This is accomplished by adding a "phantom" byte to the FIN/ACK packets. So even that there is no data in the FIN/ACK packets, it gets treated as if there was 1 byte of data. That's why you see the ACK's in frame #2264 and #2267

The only thing weird in this case is that 172.24.9.13 does not wait for the ACK to be received, but sends a RST straight after the FIN/ACK. This means the session will be removed from the TCP session table immediately (both on the endpoints as well as session based devices in the path between them). My bet is that the ACK in frame #2267 is now seen as a new session by the firewall, because it already removed the session from its sessiontable when it received the RST of frame #2266.

What kind of OS and Application is running on 172.24.9.13?

edit flag offensive delete link more

Comments

The only thing weird in this case is that 172.24.9.13 does not wait for the ACK to be received, but sends a RST straight after the FIN/ACK

I've seen this a lot of times, for example for Google Chrome

Packet_vlad gravatar imagePacket_vlad ( 2019-05-08 09:22:46 +0000 )edit

Hi SYN-Bit, First of all: Wow, thank you so very much for this! Lately i've been going deeper and deeper in the nitty-gritty stuff about (TCP) connections so this is really helpful. I really thought ending TCP connections was started with a [FIN] but now i understand that it could be a [FIN,ACK] right away.

And exactly how you explain it: chances are pretty big our firewall (Cisco ASA) see's the [RST] and removes it from his connection table as well. If that's the case, it makes sense that #2267 is seen as "new" (which in this case is allowed by our firewall since there is a permit rule to investigate this).

About 172.24.9.13. In this case it is our customer, so the OS and/or Application might differ. I can only control 172.24.0.43 which is running RHEL and running ...(more)

EricSnijders gravatar imageEricSnijders ( 2019-05-08 09:30:32 +0000 )edit

You are very welcome. One additional note on the use of ACK. When the ACK bit is set, it means the acknowledgement number field is significant. That is true for all packets except:

  1. the initial SYN (in which case there was no sequence number to acknowledge)
  2. some RST packets also do not have the ACK bit set (although I can't remember in which cases)
SYN-bit gravatar imageSYN-bit ( 2019-05-08 13:33:43 +0000 )edit

@Packet_vlad : OK, I need to capture some Chrome traffic. Thanks!

SYN-bit gravatar imageSYN-bit ( 2019-05-08 13:34:21 +0000 )edit
  • RST must have ACK set if it's intended to close an established TCP session. The second endpoint will check this ACK # for validity (a form of 'connection close attack' protection).
  • RST may be clear (no ACK flag) if this is a reply to SYN (service refusal).

As for the question this is kinda strange ASA treats pure ACK packet as a new connection. Not an expert in ASAs, but as I know to treat connection as 'new' a firewall has to observe SYN flag whereas ACK packets match 'established' rule.

Packet_vlad gravatar imagePacket_vlad ( 2019-05-08 13:50:53 +0000 )edit

Alright, thanks once again. I need to focus a bit on the ACK's since it's not yet 100% clear to me to see what packet(s) are beeing "ACK"-ed but i believe there are some tutorials on the internet explaining the Seq and ACK numbers and so.

@Packet_vlad You are right, a ASA would normally only allow TCP traffic if he see's the initial [SYN] from the right way. Except, in my case: we have a TCP-State Bypass policy for our source (172.24.0.43).

EricSnijders gravatar imageEricSnijders ( 2019-05-08 13:51:56 +0000 )edit

@EricSnijders Ah, ok, thanks for clarification!

Packet_vlad gravatar imagePacket_vlad ( 2019-05-08 13:56:38 +0000 )edit

@EricSnijders As for packet #2267: this is an ACK emitted by server for [FIN,ACK] packet #2265.

It has ACK# 5733 (FIN has Seq 5732, FIN packet length is 1 Byte, FIN packet has to be ACK'ed). The question is - why the server sent it if there was [RST,ACK] #2266? My opinion is: the server sent it BEFORE it got and interpreted [RST,ACK] #2266.

But at the time the client has already marked connection as closed, so when it receives this ACK #2267, it says 'go away, my port 17240 is closed now'.

Packet_vlad gravatar imagePacket_vlad ( 2019-05-08 14:06:30 +0000 )edit

In general you'll always see FIN and ACK together, because it signals the end from one side of the conversation and all packets after the first SYN are always ACKing opposite packets. At least I can't remember ever seeing a FIN on its own except when packets have been manipulated by scan/attack tools

Jasper gravatar imageJasper ( 2019-05-08 14:16:00 +0000 )edit

@Packet_vlad Alright, that makes sense. But could you also say that in this case it's pretty harsh for 172.24.9.13 to send the [RST] right after he sends it's own [FIN,ACK]? If i'm right, it doesn't allow the server (172.24.0.43) with much time to send it's [ACK] on the [FIN,ACK]? What would've happened if packets #2266 and #2267 were flipped in order? Would the TCP connection be closed "gracefully" in that case?

EricSnijders gravatar imageEricSnijders ( 2019-05-08 15:14:37 +0000 )edit

@EricSnijders I think I was too hasty in my analysis. Would it be possible for you to share this tracefile somewhere and paste the link here? I'd like to look at the exact sequence of events which is not possible from a screenshot unfortunately.

SYN-bit gravatar imageSYN-bit ( 2019-05-08 15:16:38 +0000 )edit

@SYN-bit No problem. Unfortunately i've already deleted this trace but since this behaviour is happening regularly, i could create a new capture. I'll try to do it asap!

EricSnijders gravatar imageEricSnijders ( 2019-05-08 15:20:21 +0000 )edit

How can you delete such an interesting trace? ;-)

BTW Looking at the timing of the packets I believe it is the ASA sending the RST packets and I would like to verify that theory by looking at the ip.id and ip.ttl fields

SYN-bit gravatar imageSYN-bit ( 2019-05-08 15:25:00 +0000 )edit

@SYN-bit Capture is already running again. Now i just need to hope the customer is still working and generating a packet like this. Will get back as soon as i have something!

EricSnijders gravatar imageEricSnijders ( 2019-05-08 15:27:39 +0000 )edit

Sake: this is very good assumption. Check also Win size in RSTs - it is different (0 and 14336).

I cannot understand one more thing: server in #2262 has Seq = 3105, but the client in #2264 ACKs...3020. What is the reason for that? Then: #2265, client's [FIN,ACK]: 3020, but instantly [RST,ACK]: 3105.

Looks like some factor we don't take into account.

What do you think - maybe #2264 is not for #2262 [FIN,ACK] but for some packet before? Whereas #2266 [RST,ACK] is a response to #2262 [FIN,ACK].

Apparently this trace is more complex than it seems to be.)

Packet_vlad gravatar imagePacket_vlad ( 2019-05-08 16:09:18 +0000 )edit

@Packet_vlad & @SYN-bit The capture is still running but as i was afraid: the customer is already off so i don't think i'll capture any traffic today. I'll keep the capture running. I'm expecting this traffic somewhere tomorrow morning. I'll try to capture a bit more than just 1 connection. Hopefully the behaviour will be the same in every connection. I'll make sure to provide you with the .pcap.

I really appreciate both your help. Unfortunately i don't understand the Seq, Ack and Win numbers yet but i would definity like to learn. Seems like i got some interesting stuff here :)! Thanks again guys, you'll hear back from me tomorrow morning.

EricSnijders gravatar imageEricSnijders ( 2019-05-08 16:29:39 +0000 )edit

Yes, the ACK #3020 in frame #2264 triggered me. Analyzing the 3-way-handshake made me realize we are capturing somewhere in the middle (but closer to the server) as there is 2,3 ms delta between SYN and SYN/ACK and 11,2 ms delta between SYN/ACK and ACK. So anything that responds within 2 ms must be from a system closer to the capture point. Also the FIN from the client seems to not ack the "Encrypted Alert", but only the (assumingly) last byte of the HTTPS response. So, looking at the full IP and TCP headers should make this more clear...

SYN-bit gravatar imageSYN-bit ( 2019-05-08 16:32:35 +0000 )edit

@SYN-bit &@Packet_vlad Morning guys! Took me some time but i captured a example. You can download the .pcap here: https://packettotal.com/cache/a8859d7... If it's any helpfull, i can also provide a .pcap where the "reverse" rule on our Firewall is NOT triggered (from the same client/server IP's). In other words: a connection that seems to be going as it should. Just let me know if that would be usefull. I'm looking forward to your findings, hope i can keep up with you guys!

Btw: this capture is from our firewall, so indeed not directly on our "server".

EricSnijders gravatar imageEricSnijders ( 2019-05-09 07:05:10 +0000 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2019-05-08 06:49:03 +0000

Seen: 5,841 times

Last updated: May 09 '19