Ask Your Question
0

How to handle memory growth in tshark while reading from a captured file.

asked 2019-05-22 10:43:28 +0000

dishadaniel gravatar image

updated 2019-05-22 16:10:16 +0000

Jaap gravatar image

Regarding https://blog.wireshark.org/2014/07/to... , I see that the "-b" flag "ring buffer" only applies to the live capture mode with "-i". How to handle the memory growth while reading a very large capture file using "-r" option. e.g. ./tshark -r example.pcap ??

Tried following changes to discard states : In process_cap_file(), I added following code after each process_packet_single_pass() call :

if (perform_stateless_analysis) {
          epan_dissect_free(edt);
          epan_free(cf->epan);
          cf->epan = tshark_epan_new(cf);
          edt = epan_dissect_new(cf->epan, create_proto_tree, print_packet_info && print_details);
 }

After these changes the %CPU and %MEM still seems to grow as observed by the 'top' command. The processing time also have increased drastically. If we do not want to maintain the states, is there anything else we can do to reduce the memory consumed and the processing time??

edit retag flag offensive close merge delete

Comments

Split the capture file into smaller pieces? Slice the packets in the capture file to remove layers of no interest?

Unfortunately processing big captures takes lots of CPU and memory.

grahamb gravatar imagegrahamb ( 2019-05-22 10:48:45 +0000 )edit

Can you explain a bit more on what could be the reason for the memory consumption even after freeing up the memory by using epan_dissect_free(edt); epan_free(cf->epan) calls after each packet is processed ??

dishadaniel gravatar imagedishadaniel ( 2019-05-22 12:34:51 +0000 )edit

What about using "-M" option, does that help? (not able to look into the details now, the documentation is quite sparse).

Jaap gravatar imageJaap ( 2019-05-22 16:24:12 +0000 )edit

There is some improvement with the -M option but not much. But observed that if I run the same tshark command with -V option (add output of packet tree), the %MEM as monitored by 'top' is very very low. Any particular reason for the same ?? e.g. ./tshark_orig -M 1000 -Vr ../tigo.pcap

dishadaniel gravatar imagedishadaniel ( 2019-05-23 08:40:49 +0000 )edit

1 Answer

Sort by ยป oldest newest most voted
0

answered 2019-05-22 13:32:43 +0000

JeffMorriss gravatar image

See some suggestions on the Wiki's OutOfMemory page.

edit flag offensive delete link more

Comments

Thanks for your reply.

From the shared link, the two main reason for memory usage is to store : 1. information in the packet list 2. information that is kept in order to support more advanced protocol analysis.

I am not that familiar with the wireshark code, so wanted to understand the actual data structures that are used to store the above mentioned data.

If I do not want to store data for advanced protocol analysis like reassembly etc., is freeing below two structures enough ??: epan_dissect_free(edt); epan_free(cf->epan);

Can you please provide some pointers in the code which could help us free memory related to data that is used for further analysis which we might not need in our application ?

dishadaniel gravatar imagedishadaniel ( 2019-05-22 13:53:16 +0000 )edit

It's not so easy some structures are handled by the protocol dissectors them selfs and may or may not be possible to turn off by preferences. One possibility is to dissable all protocols and then just enable the ones you need. Anything on TCP will be useless without reasembly for example.

Anders gravatar imageAnders ( 2019-05-22 18:47:13 +0000 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2019-05-22 10:43:28 +0000

Seen: 48 times

Last updated: May 22