How to log to .pcap files and also allow real time logging

asked 2020-07-04 10:13:06 +0000

le_top
1 ●3 ●5 ●4

updated 2020-07-10 18:17:20 +0000

My specific context is related to debugging misbehavior on Zigbee networks that happens when one is not looking, and allow at the same time to do real time debugging (remotely). The capturing hardware would be located on a small linux system like a Raspberry Pi, Beagleboard, Olimex or other Linux based "embedded" system.

More generally, the question is about how to log captured packets to a file while allowing a remote connection at the same time. Logging packets to files by itself is also a well-known methodology. To limit the log sizes, history and make it easier to get the appropriate log, the log files would be "rotated" every hour for example.

Doing both at the same time requires a "trick" that I did not identify yet.

This is what I observed/deducted with as less than 100% level of certainty: - "sshdump" allows to connect to a "FIFO" on a remote system. Data read from the FIFO is gone forever once it is read. - Only one connection can be made to a "FIFO" at a time. - As far as I could see the external capturing tool sends a header identifying the nature of the stream which also has to be added to the log file.

Any hint?

edit retag flag offensive close merge delete

add a comment

2 Answers

Sort by » oldest newest most voted

answered 2020-07-10 17:34:47 +0000

le_top
1 ●3 ●5 ●4

updated 2020-07-10 18:28:08 +0000

I did not find a solution matching my needs, so I implemented my own in perl.

It:

Reads 1 to many input pcap fifos (pipes);
Writes 0 to many output pcap fifos (pipes);
Writes 0 or 1 full file log;
Writes 0 or 1 "rotating" filelog (it creates a new log after the configured delay);
Allows restarting wireshark (or other tool) on the output fifo (pipe);
Allows reconnecting the input pipe (without restarting wireshark or other tool).

The script gets the header from the input(s). The headers must be compatible (only one of the headers is used). The header is written to each output file. The header is writtent to each pipe when a slave connects on the pipe.

This allows logging on a small linux system such as a raspberry pi, beaglebone or olimex board. At the same time any local or remote tool can connect to the stream and be used for real time debugging.

Code and dense instructions - should work for any type of capture: https://gist.github.com/mdeweerd/4bf3...

edit flag offensive delete link

add a comment

answered 2020-07-05 21:45:19 +0000

Bob Jones
1496 ●2 ●179 ●22 Boston, MA

updated 2020-07-05 21:46:54 +0000

I solve this problem with local storage for the historical data collection and then use ssh with dumpcap to bring across 802.11 data from a Linux capture host to other PCs (Windows/Mac/Linux) for realtime analysis. So I have the same requirement as you, but I don't need this particular solution. I much prefer the local storage option as there are fewer moving parts - when the Linux capture system boots, it automatically starts the ring buffer with no human intervention, or even a network that is up, let alone other hosts that need to collect the traffic.

If you feel you need to bring across the data for remote storage and realtime analysis, you can try the pee command in Linux/MacOS (part of moreutils package). Something like this works to send the stream from a Linux capture system to a Macbook:

ssh sniff@remotesniffer "/usr/bin/dumpcap -i wlan1 -P -w - "| pee  "wireshark -k -i -" "tshark -i -"

I don't know what will happen if, for instance, you shutdown Wireshark - you may not be able to restart it without bringing down the whole chain, which will defeat the purpose.

edit flag offensive delete link

Comments

I was not familiar with pee on Linux. Pretty good discussion here.

Chuckc ( 2020-07-05 22:46:45 +0000 )edit

Apparently I was not entirely clear. Storing the historical data on the Linux capture host if fine for me. If I need the historical data I can copy them using scp or a file share if I want. From what I understand, it look like I can not use dumpcap here as I am capturing Zigbee packets through an "extcap" interface. I'll check pee - if pee can help me dump to two pipes open or not, writes the header when the pipe is (re)opened then I could setup some kind of logrotation and allow realtime sniffing at the same time.

le_top ( 2020-07-05 23:21:28 +0000 )edit

add a comment