Ask Your Question
0

Syn and Immediate Out of Order

asked 2020-11-06 21:39:28 +0000

Have a customer who experiences random slowness and packet loss using a cloud application. The issues seems to be return traffic. Routes are good. They provided a capture which shows that the three way handshake never completes. A syn goes out from client, then there is an immediate out of order from client and then the client send an ack. TLS begins handshake begins, bytes in flight keeps climbing until it goes back to 0 and does the same process over again. I am puzzled.

There is zero traffic in this conversation that comes back from the server. It is all unidirectional from the client. I cannot upload a pic unfortunately.

https://ibb.co/HrF6TSb

edit retag flag offensive close merge delete

Comments

Can you anonymize the capture and share it?
How was the capture made? On the client? Span port? TAP?

Chuckc gravatar imageChuckc ( 2020-11-07 03:19:12 +0000 )edit

Sorry I didn't provide enough info. Yes I am told it was captured client side when the issues was happening but I know nothing about what type of device it is. I am wondering if the NIC is just getting bogged down. It is a little over 5 minute log capture and according to the capture properties the average bytes per second are below 1KB and pps are VERY low.

I anonymized and uploaded to my DropBox.

https://www.dropbox.com/s/91vbv6tej2a...

DreamTFK gravatar imageDreamTFK ( 2020-11-08 18:45:18 +0000 )edit

I think you will need to get the capture process resolved if you want to use it to diagnose the "random slowness and packet loss".
The frames marked as TCP Out-Of-Order are an exact copy of the SYN before it.
There is a Frame protocol preference to have Wireshark Generate an MD5 hash for each frame which can be used to confirm they are identical.
Here is a brief overview of Duplicate Packets and TCP Retransmissions . At about 2:52 is a description of layer 2 duplicate packets.

Chuckc gravatar imageChuckc ( 2020-11-08 21:31:25 +0000 )edit

I added a packet ID column and confirm each SYN and OUT-OF-ORDER are identical. I have never seen this kind of behavior before although I do not examine captures all that often. Any idea why the client is still sending out an ACK without receiving the SYN-ACK?

DreamTFK gravatar imageDreamTFK ( 2020-11-08 21:52:05 +0000 )edit

I'm a picture guy so at this point I would ask the customer for a diagram showing the client, devices between it and the Internet, and where the capture is being done. Just because the SYN-ACK isn't in the capture doesn't mean the client didn't see it.
The one sided capture could be asymmetric routing as @Jaap mentioned and @Jasper describes here or it could be how the capture is configured. Double binary copies of the SYN seems like a SPAN/TAP/switch config issue.

Add on to comment above about enabling MD5: wiki Frame page now with screen shots.

Chuckc gravatar imageChuckc ( 2020-11-08 22:09:48 +0000 )edit

1 Answer

Sort by ยป oldest newest most voted
0

answered 2020-11-07 10:58:25 +0000

Jaap gravatar image

A picture gives very limited options to analyse this situation. Pulling this capture file through Trace Wrangler and putting it on a public accessible file share site gets you further.

Also without knowing where the capture was made makes analysis even more limited. So what can we tell? From the looks of it it seems that we're only seeing traffic from client (port 51824) to server (443), but never the other way round. While from the reactions from the client there must be something coming back. That suggest interface bonding, where the capture is made on the standby interface. It might be that the other side of the bonded network link is some wonky equipment spewing this stuff on this link? Hard to tell without further details.

edit flag offensive delete link more

Comments

Sorry I didn't provide enough info. Yes I am told it was captured client side when the issues was happening but I know nothing about what type of device it is. I am wondering if the NIC is just getting bogged down. It is a little over 5 minute log capture and according to the capture properties the average bytes per second are below 1KB and pps are VERY low.

I anonymized and uploaded to my DropBox.

https://www.dropbox.com/s/91vbv6tej2a...

DreamTFK gravatar imageDreamTFK ( 2020-11-08 18:45:28 +0000 )edit

Since we're seeing only one direction of traffic I wonder how this capture was made. Either there's interference of some firewall / VPN or other network stack SW, or some sort of asymmetric routing?

Jaap gravatar imageJaap ( 2020-11-08 20:18:01 +0000 )edit

I do agree. No idea if the packets are actually getting put onto the wire from the client and making it out of the net work. If that is happening I cannot see yet if they are coming back. It is odd that it is sending out identical packet ID's. I instructed the customer to grab a fresh capture from the client the firewall and the router the next time it happens so I can see the full path.

DreamTFK gravatar imageDreamTFK ( 2020-11-08 21:43:55 +0000 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2020-11-06 21:39:28 +0000

Seen: 556 times

Last updated: Nov 07 '20