Ask Your Question
0

can tshark rotate logs when using json output (-T ek)?

asked 2019-12-23 07:41:03 +0000

JoDT gravatar image

I'm trying to ingest packet captures into Elasticsearch, using filebeat. Seems like the ringbuffer doesn'twork when using the option "-T ek". I'm using wireshark 3.0.2 on CentOS 7

edit retag flag offensive close merge delete

Comments

What is the full command line you're using for tshark?
Do you get them same warning as in this question:
https://discuss.elastic.co/t/tshark-a...

Chuckc gravatar imageChuckc ( 2019-12-23 14:32:48 +0000 )edit

It's an extra step but can you write the files in raw mode with ring buffer then process back through tshark to feed JSON to filebeat?

Chuckc gravatar imageChuckc ( 2019-12-23 14:43:22 +0000 )edit

The ring buffer mechanism is for raw capture files (pcap or pcapng files), not for dissected output such as text, JSON, PSML/PDML, or ElasticSearch; the dissected output just gets written to the standard output in TShark, and that inherently has no mechanism for rotating files, so the ElasticSearch output has no ring buffer. Are you saying that you're capturing to a raw capture file and using "-T ek" to get dissected output for the same capture, and the raw capture files aren't being treated as a ring buffer?

Guy Harris gravatar imageGuy Harris ( 2019-12-24 03:58:58 +0000 )edit

I was only using the -T ek not using the raw capture output. Would piping it to rotatelogs from Apache be an option to get the ringbuffer behaviour? Or would I need to look into the 2-step process? I wanted to get a near real-time feed of tshark into Elastic, so not sure if that's possible with tshark -T ek. Not sure if vichargrave/espcap would be better suited for this

Thanks for the help

JoDT gravatar imageJoDT ( 2019-12-31 11:13:27 +0000 )edit

1 Answer

Sort by ยป oldest newest most voted
0

answered 2019-12-31 20:34:59 +0000

Guy Harris gravatar image

updated 2020-01-01 04:19:19 +0000

The ring buffer is a mechanism in the raw binary capture code path (in dumpcap).

There is no ring buffer mechanism for any -T text formats. Note that those mechanisms write to the standard output stream, not to files created by TShark, so a ring buffer mechanism for text output would require that TShark create the text files itself.

If rotatelogs reads its standard input and writes it to a sequence of log files, switching to a new file based on time or file size (as the documentation implies but does not explicitly state, perhaps assuming it's obvious), then piping the output of TShark to rotatelogs should work.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2019-12-23 07:41:03 +0000

Seen: 647 times

Last updated: Jan 01 '20