TShark CSV export problem

tshark
csv

asked 2020-01-04 16:26:24 +0000

NIoSaT
3 ●1 ●2

Hello,
I am using tshark to export a pcap to a csv.
The command looks like this (I have many more fields that I want to export, I removed them here to make it shorter):

tshark -r recording.pcap -T fields -e ip.proto -e ipv6.nxt -e frame.number -e frame.time_relative -e ... -E header=y -E separator=, -E quote=d > out.csv

This works for the most part and I get output lines in my CSV as I would expect them

"6",,"2552","53.591729000","Jan 25, 2011 19:53:16.076138000 CET","936","936","00:1a:8c:15:f9:80","40:61:86:9a:f1:f5","66.235.139.121","192.168.3.131",,,"80","56126","3233954145",,

But some lines in my CSV are "borken" like this

"1,1",,"5583","2.171949000","Feb 26, 2013 23:02:38.125121000 CET","70","70","00:90:7f:3e:02:d0","00:90:fb:34:44:24","68.85.69.49,172.16.133.109","172.16.133.109,64.30.236.34",,,,,,,

As you can see the protocol number (1) is there twice and the ip addresses are also "doubled"...
When I look at this packet in wireshark I can see that it is an ICMP packet and from 68.85.69.49 to 172.16.133.109.
It seems that tshark has somehow "merged" two packets into on row. Sometimes its even worse with 5 or more.

I used this export technique before and it always worked as expected.
Is there anything I could do about this?
I used the smallFlows.pcap and bigFlows.pcap from here

edit retag flag offensive close merge delete

add a comment

answered 2020-01-04 17:11:32 +0000

grahamb

flag of United Kingdom of Great Britain and Northern Ireland

23835 ●4 ●973 ●227 https://www.wireshark.org

This is not a Wireshark or tshark issue, but due to the nature of the traffic.

Depending on the specific ICMP message, an ICMP packet can include portions of the original packet that caused the ICMP message to be transmitted. Looking at smallFlows.pcap there are ICMP Time-To-Live exceed packets that do include the original packet. As this original packet info is also dissected by Wireshark, the "duplicated" fields are included in the output.

To filter these out, either exclude ICMP as part of a filter expression, disable the ICMP dissector or limit the field output to the first occurrence with -E occurrence=f.

Comments

Oh I never noticed this before.
Wireshark handles it like I would expect: Source and Destination are a single IP and all other information are inside the payload part. So with -E occurrence=f tshark only outputs the first occurance and not others that might be inside the actual payload an not the header of the packet?

NIoSaT ( 2020-01-05 12:53:28 +0000 )edit

Yes, first occurrence limits the output to the first occurrence of a field in the packet. From the tshark man page:

occurrence=f|l|a Select which occurrence to use for fields that have multiple occurrences. If f the first occurrence will be used, if l the last occurrence will be used and if a all occurrences will be used (this is the default).

grahamb ( 2020-01-05 16:02:27 +0000 )edit

add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

TShark CSV export problem

1 Answer

Comments

Your Answer

Question Tools

Stats

Related questions

TShark CSV export problem edit

1 Answer

Comments

Your Answer

Question Tools

Stats

Related questions

TShark CSV export problem