Converting Pcap file to CSV file while defautly keeping all features/fields defined in pcap
Hi all, I have been searching for solutions that can transform PCAP files to CSV format while keeping all the details defined in the pcap file, without explicitly defining which features/fields that i want to include in CSV format.
With Wireshark&Tshark, i have tested the following two ways:
the first one is to use tshark commands such as:
tshark -r traffic.pcap > traffic.csv
tshark -r traffic.pcap -T fields -e ip.src -E separator=, -E occurrence=f > traffic.csv
In any case, the traffic.csv contains only general information (e.g., No., Time, Source, Destination, Length, Protocol, Info) of the traces without any packet detail(Packet Bytes).
I also tried with Wireshark by selecting Export Packet Dissections, however i got differet results by exporting the same pcap file to csv, json, and plain text format. In general, the exported csv file still contains only general information of packets, without packet detail (Packet Bytes), even i selected the field (Packet Bytes during export).
I wonder if there is a way i can also have the the details of each packet in the csv file? Thanks.
What would you expect a CSV of all fields to look like? A CSV file is a representation of data in a row\column format, with the rows as packets and the columns as fields. Depending on the data the number of columns could be enormous.
Maybe you could explain what you want to do with the data so alternatives could be suggested?
the dimension of the data is not an issue, since i would like to explore the different column combinations to support the machine learning work later. One thing i found wierd is that, if I choose the json or plain text as an export format, the packet byte is in the file, it does not work with csv file.
CSV needs to know what the columns are in order to put the value into the correct columns. Maybe look at the ElasticSearch output,
-T ek
.I don't know if i'm being naive thinking that the automatic conversion can leave fields as empty if certain features are not appicable to specific packets, I will check the ElasticSearch as suggested thanks.