Tshark -d option to format date doesn't work with -T fields

asked 2020-05-21 16:37:15 +0000

PaulELong
1 ●1 ●2 ●2

updated 2020-05-22 15:17:33 +0000

flag of United Kingdom of Great Britain and Northern Ireland

23835 ●4 ●973 ●227 https://www.wireshark.org

Tshark's driving me mad! I want to parse a trace and output as csv with headers and I want the date format in UTC. Instead I always get the long format like "May 20, 2020 17:34:23.241938000 Eastern Daylight Time".

My current attempt is using the following, which according to link:this post can be done with -t ud, but it doesn't affect the output. For instance:

tshark -r in.pcap -Y frame.number==1 -E header=y  -E separator=',' -t ud -T fields -e frame.time

I also explored the -o gui.column.format option (which is tricky to get working in powershell, but I did). I was able to format the date properly using that method, but I didn't see a way to add comma separators or headers.

My long term goal is to dump TCP parameters so that I can import into Splunk and design charts to understand slow uploads and TCP congestion. Wireshark takes to long to chart and crashes frequently. And some a previously mentioned tool like TCP trace is archaic stating that maybe it'll work on Win2000 :) Splunk integration would also be nice because I could correlate with other log data that is already imported.

Doesn't anybody know why -d option doesn't work?

Thanks -Paul

edit retag flag offensive close merge delete

Comments

Have you tried with the field _ws.col.Time ?

    $ tshark -r ./stream-0.pcap -T fields -e _ws.col.Time  -e frame.number -e frame.time -Eheader=y -Eseparator="/t" -Equote=d -c 2 -t ad
    _ws.col.Time    frame.number    frame.time
    "2014-01-21 05:28:16.588384"    "1"     "Jan 21, 2014 05:28:16.588384000 Central Standard Time"
    "2014-01-21 05:28:16.603260"    "2"     "Jan 21, 2014 05:28:16.603260000 Central Standard Time"

$ tshark -r ./stream-0.pcap -T fields -e _ws.col.Time  -e frame.number -e frame.time -Eheader=y -Eseparator="/t" -Equote=d -c 2 -t ud
_ws.col.Time    frame.number    frame.time
"2014-01-21 11:28:16.588384"    "1"     "Jan 21, 2014 05:28:16.588384000 Central Standard Time"
"2014-01-21 11:28:16.603260"    "2"     "Jan 21, 2014 05:28:16.603260000 Central Standard Time"

Chuckc ( 2020-05-22 20:42:15 +0000 )edit

Yes, that does work! How is that different than frame.time?

PaulELong ( 2020-05-22 21:27:49 +0000 )edit

If you look in the Wireshark preferences file there is a section that defines the GUI columns:

# Packet list column format
# Each pair of strings consists of a column title and its format
gui.column.format:
        "No.", "%m",
        "Time", "%t",
        "Source", "%s",
        "Destination", "%d",
        "Protocol", "%p",
        "Length", "%L",
        "Stream index", "%Cus:udp.stream:0:R",
        "Info", "%i"

_ws.col.Time is displaying the Time column.

Format strings for columns here or you can look in the Gui.
Right click the column header for Time and select Edit Column.
The default is Time (format as specified) which the -t sets.

Chuckc ( 2020-05-22 21:48:37 +0000 )edit

gui.time_format is stored in recent which tshark doesn't use.

# Timestamp display format.
# One of: RELATIVE, ABSOLUTE, ABSOLUTE_WITH_YMD, ABSOLUTE_WITH_YDOY, ABSOLUTE_WITH_DATE, DELTA, DELTA_DIS, EPOCH, UTC, UTC_WITH_YMD, UTC_WITH_YDOY, UTC_WITH_DATE
gui.time_format: ABSOLUTE_WITH_YMD

$ tshark -G currentprefs | grep -i gui.time
$ tshark -G defaultprefs | grep -i gui.time

(No need to look at source for column types - $ tshark -G column-formats )

Chuckc ( 2020-05-22 22:02:06 +0000 )edit

add a comment

2 Answers

Sort by » oldest newest most voted

answered 2020-05-22 17:43:06 +0000

PaulELong
1 ●1 ●2 ●2

It turns out the simple solution was to separate by tabs and then Splunk was able to import the date format natively. Thanks for your help.

edit flag offensive delete link

add a comment

answered 2020-05-21 16:54:02 +0000

grahamb

23835 ●4 ●973 ●227 https://www.wireshark.org

updated 2020-05-22 14:59:58 +0000

The -t option only works on "normal" tshark output. When you use -T fields and select a particular time field, i.e. frame.time, then you get the format for your locale.

If you haven't set a TZ env variable, then tshark will use your "system" locale, so set TZ then run tshark, e.g. for PowerShell:

$env:TZ="UTC"
tshark -r ... -T fields -e frame.time ...

should give you output in UTC.

edit flag offensive delete link

Comments

I wasn't clear enough. I want the ISO UTC format, or at least a date without a comma in it like 2019-10-11T18:56:08.984Z. Or at least how %Aut works with the gui-column.format option which gives me 2020-05-20 21:34:23.241938. Is that possible?

PaulELong ( 2020-05-21 17:41:36 +0000 )edit

I believe, by using Google and looking at the docs for Splunk (I have never used Splunk), you can specify a time format for import, see Configure Timestamp Recognition and the TIME_FORMAT option. I'll leave the working out of that format as an exercise for the reader, but as a hint look at the examples.

grahamb ( 2020-05-22 11:32:38 +0000 )edit

Yes, I know I can work around the tshark issue. I think I can format as TSV rather than CSV to get around the extra comma. And then in splunk you can parse any kind of data manually. But having the tshark functionally that is documented and mentioned by @bubbasnmp would be cleaner. I was hoping I'm missing something, but maybe it's just broken?

PaulELong ( 2020-05-22 13:17:56 +0000 )edit

See the answer to this question which was so similar to yours I thought it was also for Splunk. TLDR; the format for frame.time is hard-coded.

grahamb ( 2020-05-22 13:34:56 +0000 )edit

Thanks! I guess this means I'll just need to work around the limitation which is helpful to know.

But to confirm, the -t ud option doesn't work with the -T options? You said -d above only works with the normal output, but you meant -t? And I think by normal you mean the output when you don't use the -T option?

If this is true, then maybe a better enhancement request would be to to all -t to work with -T?

PaulELong ( 2020-05-22 14:55:08 +0000 )edit

see more comments