Ask Your Question
0

using tshark with huge display filters

asked 2024-03-21 13:22:20 +0000

echo gravatar image

I have a large capture file which I want to filter, and have written a PowerShell which selects the frames in the capture file to keep. The produced display filter looks like this 'frame.number in {1, 6, 8,... }' but is large (saved to a text file it exceeds 700kB). When running tshark with this filter (tshark -r infile -Y "frame.number in ...." -w outfile) I get an error because the command is simply too large for the (powershell) command prompt to handle.

When I copy the filter to my clip board, open the GUI, paste the filter into Wireshark, it actually works (but very very slow), so the command shell (powershell) is the bottleneck, not tshark itself.

I've considered selecting frames in smaller batches, going over the capture file again and again, and then glueing the resulting capture files together with mergecap, but I consider that a messy (and most likely extremely slow) solution.

So the main question: Is there a more elegant solution, for example to read the display filter from a text file, and if so, how?

edit retag flag offensive close merge delete

2 Answers

Sort by » oldest newest most voted
0

answered 2024-03-21 19:51:31 +0000

André gravatar image

updated 2024-03-21 19:57:22 +0000

There is no option to read the display filter from a file instead.

The Windows command line length is limited to 32,767 (wide) characters. (Limited by the CreateProcess system call.)

Option 1: Make sure the total length does not exceed 32k:

  • for in operator use ranges where possible. Not in { 1, 2, 3, 4 } but in{1..4}, etc.
  • remove all unnecessary spaces (in{1..4}).

Option 2: Use an OS that allows a larger command line length.
On Mac OS it is about 256 kbyte.
On Linux it is about 2 Mbyte.
To get the exact value use the command: echo $(( $(getconf ARG_MAX) - $(env | wc -c) - $(env | wc -l) * 8 - 8 )) (= ARG_MAX – environment array size).
More info: man execve

Option 3: Is there a better way to filter out what you want, resulting in a smaller filter? E.g tcp.stream in { ... }

And splitting up in batches is also a possibility. That will be a slow process.

edit flag offensive delete link more
0

answered 2024-03-23 02:47:37 +0000

Chuckc gravatar image

updated 2024-03-23 11:20:25 +0000

What about a Lua script that reads a text file with packet numbers to keep and dumps those packets to a new file?

C:\>type keeplist.txt
1
3
5
37916

C:\>capinfos -c bvlc.pcap
File name:           bvlc.pcap
Number of packets:   37 k

C:\>tshark -q -X lua_script:.\keeplist.lua -r bvlc.pcap --disable-all-protocols
Tap draw
Tap reset

C:\>capinfos -c keeplist.pcap
File name:           keeplist.pcap
Number of packets:   4
-- keeplist.lua
-- https://ask.wireshark.org/question/34089/using-tshark-with-huge-display-filters/
-- "(tshark -r infile -Y "frame.number in ...." -w outfile)"
-- Create new capture file containing frames which are specified in a text list

-- Step 1 - document as you go. See header above and set_plugin_info().
local keeplist_info =
{
    version = "1.0.0",
    author = "Good Coders",
    description = "Copy/pasted together by Chuck Craft",
    repository = "Floppy in top drawer"
}

set_plugin_info(keeplist_info)


do
    keeplist = {};
    file, err = io.open("keeplist.txt","r")

    if not file then
        -- Opening the packet list failed, return the error
        print("Cannot load keeplist file: " .. err)
        return
    end

    while true do
        local line = file:read("*number")
        if not line then break end  -- break on EOF

        keeplist[line] = true
--        print(line)
    end
    io.close(file)

    local function init_listener()
        local keep_dmp

        local tap = Listener.new()
        function tap.reset()
            print("Tap reset")
            keep_dmp:close()
        end
        function tap.packet(pinfo,tvb,tapinfo)
--            print("pinfo.number = " .. pinfo.number)
            if pinfo.number == 1 then
                keep_dmp = Dumper.new_for_current( "./keeplist.pcap" )
            end
            if keeplist[pinfo.number] ~= nil then
--                print("Dump packet = " .. pinfo.number)
                keep_dmp:dump_current()
            end
        end
        function tap.draw()
            print("Tap draw")
            keep_dmp:flush()
        end
    end
    init_listener()   
end


Credits to:

Ask question Filter udp packets using lua script
Wiki Dump VoIP calls into separate files
Github wireguard-dissector /wg.lua

edit flag offensive delete link more

Comments

Thanks @Chuckc ! This is exactly the kind of solution I needed. It also helped me to finally look into Lua which appears much more accessible than I believed it to be. Unfortunately I am not allowed to upvote your comment, due to lack of 'points' so I leave that to others :)

echo gravatar imageecho ( 2024-04-08 19:06:33 +0000 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2024-03-21 13:22:20 +0000

Seen: 114 times

Last updated: Mar 23