TLS Wireshark find failed TLS Handshakes

asked 2021-11-14 15:14:40 +0000

fly_agaric gravatar image

Hello Guys,

I have a Problem with an application "Battle Net" not working in an Deep Packet Inspection enabled environment since 3 days. Before everything was working. Since 3 days the whole internet connection nearly breaks while the Application is running. I waited 5 min for example for to come up while the application was running. When I was looking at task manager I saw that around 100Kbit/s of traffic was flowing not much compared to my 4 Mbit Download Max speed. I looked on the firewalls traffic monitor and there I saw that my whole Bandwidth was used but it was not shown in task manager or resource monitor. The strange thing is that my computer was the only device attached to the firewall at this time and the issues disappeared 10 seconds after I have closed the Application. It was reproduceable. After I disabled Deep Packet Inspection on my Firewall the issue has gone. However I don't want to disable DPI for the whole Internet only for Battle Net. So I wanted to find out which connections failed and started a trace capture.

Then I captured a Trace with Microsoft Network Monitor and saved it as cap file so that wireshark can open it. At first I only exported everything which was in connection with Agent.exe, Battlenet.exe and all the *.exe the battle net launcher used. After I had the data in wireshark I suspected that some TLS handshakes failed and I that I simply need to exempt them from DPI Profil on Firewall. So I set a Display Filter "tls" in Wireshark and there I saw every TLS Packet I guess. Then I thought to get "defective" handshakes I look at the TCP Stream ID and if there is "Application Data" in the Info field. There were many TLS Streams so I exported the data into a mysql database where I did the following statement on all the exported data:

select stream from data where stream not in (SELECT DISTINCT(Stream) FROM `data` WHERE (Info = "Application Data"))

My Problem was that MySQL didn't returned any data then i checked every tcp stream which was returned in wireshark manually and every stream contained at least a valid Application Data field. So for me this means wireshark is not able to decode the handshake pakets or that there are no invalid tls handshakes right?

Am I right when I'am saying when a tls stream in wireshark does contain at least one application data packet that the tls handshake was successful or am I missing something here?

I have attached 2 trace files one already prefiltered with netmon and the other one the unfiltered one from the same trace.

Battle Net filtered Battle Net Trace unfiltered

edit retag flag offensive close merge delete


I vaguely understand your issue, but all the other detail about your approach to resolving the issue obscures what you're trying to do.

I think you're trying to find TLS connections that don't complete the handshake, is that correct? There are a number of reasons that a handshake may not complete, what do you think the behaviour caused by your DPI is\will be?

Looking at the unfiltered capture there are some TLS conversations which aren't automatically dissected as TLS by Wireshark, e.g. port 8080, 27032 so they may be missing from your analysis. You have to use Decode As ... to set those ports as TLS.

grahamb gravatar imagegrahamb ( 2021-11-14 19:55:29 +0000 )edit

Hello grahamb,

yes I'am trying to find TLS Handshakes which wont complete. My Firewall does a Man in the Middle to read what user is doing. So I need to exempt the failed TLS connections from TLS Inspection.

TCP Port 8080 only occurs with one ip 10.0.x.x which is the local ip of my firewall in the other vlan. During the trace i was connected to the webinterface so port 8080 is not important in this case.

I think that "Battle Net" is using some form of certificate pinning or does not use the windows certificate store so some tls connections may fail and i need them to exclude them from dpi.

In my copy of wireshark tcp port 27032 is detected as tls. When I click Analyse -> Decode As -> it shows Field: TLS Port, Value: 27032, Typ: Integer, bas10, Standard: (none), and current ...(more)

fly_agaric gravatar imagefly_agaric ( 2021-11-14 21:01:14 +0000 )edit

I used the Analyze->Endpoints dialog, looking at the TCP tab to see what iP\hosts and ports were in the capture. I sorted by port number and discounted all those from the ephemeral port range as likely to be the client side and then filtered by the port tcp.port == xxxx to see what it was dissected as. TLS is a heuristic dissector so should pick up non-standard ports as long as another dissector doesn't get there first. I suspect the HTTP dissector grabbed it first but didn't investigate.

So what does a failed handshake look like, a missing server hello, or something later in the exchange?

grahamb gravatar imagegrahamb ( 2021-11-15 08:43:50 +0000 )edit