I have some PCAP files from which I'm trying to extract metadata. I am doing this using tshark, opening the file, extracting a couple dozen fields, then writing the table to disk. I've noticed that this process can be pretty time consuming, sometimes up to 60 minutes for a single PCAP file. I am performing reverse DNS on the data, using the default settings (-N dmN) and I have the same reverse DNS settings in Wireshark. I understand that reverse DNS is a fairly time-consuming process relative to other processes that tshark/Wireshark is performing. However, when opening the same file in Wireshark and in tshark, Wireshark loads the file in a matter of seconds, while tshark will take minutes. My tshark command is:
tshark -r my_pcap_file.pcap \
-2 \
-T fields \
-E separator=/t \
-E header=y \
-E quote=d \
-e frame.time_epoch \
-e frame.len \
-e frame.protocols \
-e _ws.malformed \
-e _ws.col.Protocol \
-e _ws.col.Length \
-e ip.rec_rt \
-e ip.src \
-e ip.dst \
-e ip.src_host \
-e ip.dst_host > my_pcap_file.tsv
I've timed the processing of a few files using /usr/bin/time followed by the tshark command shown above. To measure the fastest possible time, neglecting writing the output to disk, I directed the output to /dev/null instead of my_pcap_file.tsv. The resulting file sizes and timing outputs are:
20.7 MB: 6.16user 3.51system 11:36.95elapsed 1%CPU
10.2 MB: 2.18user 3.28system 10:45.22elapsed 0%CPU
42.1 MB: 6.70user 5.13system 44:07.60elapsed 0%CPU
Is there a known reason for this speed difference? More importantly, is there a way I can speed up the tshark processing?
