Filter out tls 1.2/1.3 app_data using tshark

asked 2019-09-02 08:08:53 +0000

Schiltech
3 ●1 ●1 ●3

updated 2019-09-05 14:14:17 +0000

Hello,

I use Wireshark 3.1.0 and tshark 3.0.3 in my dailywork and often have to analyse pcaps with huge amount of encrypted tls application data which I do not need.

I usually simply filter out those packets with the filter "not tls.app_data" on wireshark GUI which works fine but I would like to directly remove those packets from the source pcaps via an automated script using tshark.

I have tried :

tshark -F pcap -r source.pcap -w filtered.pcap -Y "not tls.app_data"

tshark -F pcap -r source.pcap -w filtered.pcap -Y "not (tls.record.content_type == 23 or tls.record.opaque_type ==23)"

In both commands only TLS 1 seems to be filtered out and the TLS 1.2 and TLS 1.3 packets are still there

tshark seems to not understand that tls 1.2 and tls 1.3 are to be considered as TLS packets.

I would like to remove only the application data, I still need to keep the other type of tls record.

Any help is welcome.

EDIT : Extra info my tshark is using : libpcap version 1.7.4, with GnuTLS 3.4.10, with Gcrypt 1.6.5,with zlib 1.2.8.

I'm starting to suspect that those lib version don't support TLS 1.2 or 1.3.

Comments

Not sure if it's relevant, but Wireshark always performs 2-pass analysis of the capture, whereas tshark will default to a single pass unless the -2 parameter is supplied.

As you're reading and writing, you may also need to use a -R read filter to accomplish your task.

I'm not sure if the library versions have anything to do with the issue as it's down to the libwireshark dissection engine, but you could check the versions of those libraries used by Wireshark in the Help -> About Wireshark dialog to see if there are any differences.

grahamb ( 2019-09-05 14:41:21 +0000 )edit

add a comment

answered 2019-09-06 09:24:36 +0000

SYN-bit

18528 ●9 ●339 ●255 https://SYN-b.it

Works fine with me, using tshark 3.0.2

Are you using the same settings? Tshark will use your default profile if you don't add -C <profilename>, so if for instance your default profile does not have TCP reassembly enabled (but the profile you use with Wireshark does), you might not get the results you want.

You can either create a profile with all the right settings and use that with tshark, or you can specify the important preference settings in your command line with -o options:

tshark -F pcap -r <file> -w <new> \
       -Y not (tls.record.content_type == 23 or tls.record.opaque_type ==23)" \
       -o tcp.check_checksum:FALSE \
       -o tcp.desegment_tcp_streams:TRUE \
       -o tcp.reassemble_out_of_order:TRUE \
       -o tcp.no_subdissector_on_error:TRUE \
       -o tls.desegment_ssl_records:TRUE \
       -o tls.desegment_ssl_application_data:TRUE

edit flag offensive delete link

Comments

Thanks. It seems to partially solve the problem. Now I have some sites in tls 1.2 and tls 1.3 filtered out too. It appears some sites still aren't filtered out but the filter seems to be working for most. Will have to dig a bit more to understand why since the TLS version doesn't seem to be the problem anymore.

Schiltech ( 2019-09-09 08:13:58 +0000 )edit

You're welcome, I'm glad you are one step closer to your goal. Are you seeing some TLS sessions for which all the application data packets are still in the trace? Or do you see sessions where some application data packets are still in the trace.

If the latter, then that might be caused by packet-loss, out-of-order packets, retransmissions, etc. The TLS dissector might loose track in those conditions and is not able to detect the TLS record layer. What you could do is remove all the packets for port 443 altogether except the three-way handshake, the ACK's and the TLS handshake, change-cipherspec and alert packets. You could use something like:

tshark -F pcap -r <file> -w <new> \
       -Y "not (tcp.port==443 and not (tcp.flags&7 or tcp.len==0 or tls.record.content_type in {20 21 22}))" \
       -o tcp.check_checksum:FALSE \
       -o tcp.desegment_tcp_streams:TRUE \
       -o ...

(more)

SYN-bit ( 2019-09-09 13:39:48 +0000 )edit

add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Filter out tls 1.2/1.3 app_data using tshark

Comments

1 Answer

Comments

Your Answer

Question Tools

Stats

Related questions

Filter out tls 1.2/1.3 app_data using tshark edit

Comments

1 Answer

Comments

Your Answer

Question Tools

Stats

Related questions

Filter out tls 1.2/1.3 app_data using tshark