tshark -E occurrence=a

Question

For my purposes, I record packets sent from radios stations that tell me their GPS position. The provider sometimes packages up 4 radio station reports into 1 packet. Basically, all the reports are exactly the same in terms of data, except for one or two fields where its ID# and position are different. All 4 reports are assembled and sent in one packet. I can easily decode and see the 4 distinct reports in that one packet while in the Wireshark GUI.

When I dump this to CSV format using tshark, I can only strip out the first report or the last report in the packet "correctly" using:

tshark -E occurrence =a or ....=l

I get one line of beautiful data showing all the values for each field for the first/last report. But if I say to give me ALL, it gives me each field from every report bunched together in a comma-separated format, all on one line as per the tshark documentation. So, it's working as stated; just not what I wanted.

If I can get the first report or last report to print correctly using the f or a switch, is there any way to get all the occurrences in that packet to print out like that? Each report pulled from the packet and printed on its own line?

I hope someone can follow along with what I am saying. I certainly appreciate any help. I'm assuming from how the documentation specifically states about printing multiple items at once for a single field that there is no workaround?

UPDATE: I uploaded a sample to cloudshark.org. My packet of concern is the first packet for port 59951 and is 412 bytes long. There are actually 8 separate reports in this packet from 8 different radio stations. Wireshark might/will think it's a PROFINET-TIME protocol, which it's not. I have my own dissector for this for it's not this protocol. I can see all 8 reports fine in the GUI.

My tshark command is quite long for there is a lot coming in each report. As is, it works great, and I can bring it into Excel to display nicely as long as I choose first or last report in a packet.

"c:\program files\wireshark\tshark.exe" -r filename.pcap -R "(adsb.type == 33)" -T fields -e frame.number -e frame.time_epoch -e adsb.svid.sac -e adsb.svid.domain -e adsb.svid.sic -e adsb.version.status -e adsb.version.number -e adsb.lti.unknown -e adsb.lti.version -e adsb.lti.1090 -e adsb.lti.uat -e adsb.lti.non_adsb -e adsb.toap.elapsed.whole -e adsb.toap.elapsed.frac -e adsb.toa.velocity.sign -e adsb.toav.elapsed.frac -e adsb.target_address.sv_type -e adsb.target_address.dup -e adsb.target_address.qualifier -e adsb.target_address.address -e adsb.iap.utc -e adsb.iap.nic -e adsb.iap.sil.sup -e adsb.iap.sil -e adsb.iap.nac_p -e adsb.iap.test -e adsb.iap.validation -e adsb.iap.nac_v.horiz -e adsb.iap.nic_baro -e adsb.gps_position.lat -e adsb.gps_position.lon -e adsb.altitude_pressure.resolution -e adsb.altitude_pressure.altitude -e adsb.velocity_air.vert_src -e adsb.velocity_air.scale -e adsb.velocity_air.ns_dir -e adsb.velocity_air.ns_vel -e adsb.velocity_air.ew_dir -e adsb.velocity_air.ew_vel -e adsb.velocity_air.vert_dir -e adsb.velocity_air.vert_vel -e adsb.mode3a.validity -e adsb.mode3a.code -e adsb.target_id -e adsb.emitter_category -e adsb.target_status.ident -e adsb.target_status.surveillance_status -e adsb.target_status.emergency_priority_code -e adsb.altitude_geometric -e adsb.op_mode_cap_code.capability_codes.tcas -e adsb.op_mode_cap_code.operational.ident -e adsb.op_mode_cap_code.operational.tcas.ra -e adsb.tomr.sign -e adsb.tomr.ns -e adsb.enh.val -e adsb.dqp.sda -e adsb.dsq.eq_type -e adsb.dsq.location_id -e adsb.dsq.instance -e adsb.target_state -E header=y -E separator=/t -E occurrence=f > filemane.csv

As I stated earlier, if I strip out the first report or last report of a packet, they come out great in tshark. If I ask for all the reports, I get one line of data with multiple used fields piled up together separated by commas. adsb.dsq.location.id is a field that is different in each report within the packet for this is the radio station ID. Rather than get 8 lines of data for the 8 reports in the packet, I get one line with any multiple-instance field (like this one) shown with comma separated values, such as: 15,16,24,356,etc.

I can list what the expected output is versus the actual output, if needed.

Here's the packet in Wireshark with my decoder clearly showing there are 8 reports within the packet. When I dump through tshark to csv, I can only choose to get first or last if I want one line per report. If I choose -E occurrence=a, I get ALL 8 reports on one line aggregated together. I can't figure out how to get them as 8 lines of output and not crammed together.

screenshot

Accepted Answer

O.K. now I understand what you are looking for:

With "-E occurrence=a" the output looks like this:

frame.number frame.time_epoch adsb.svid.sac adsb.svid.domain adsb.svid.sic
1 1.337.628.248.061.540.000 12,12,12,12,12,12,12,12 2,2,2,2,2,2,2,2 3,3,3,3,3,3,3,3
2 1.337.628.248.075.200.000 12,12,12,12,12 2,2,2,2,2 3,3,3,3,3

while you would like to have it in this format (with reduced number of lines)

frame.number frame.time_epoch adsb.svid.sac adsb.svid.domain adsb.svid.sic
1 1.337.628.248.061.540.000 12 2 3
1 1.337.628.248.061.540.000 12 2 3
1 1.337.628.248.061.540.000 12 2 3
2 1.337.628.248.075.200.000 12 2 3
2 1.337.628.248.075.200.000 12 2 3
2 1.337.628.248.075.200.000 12 2 3

Unfortunately, this is not possible with the current version of wireshark, as the field print function just adds several instances separated by the "aggregator" character.

So, to solve your problem, you would have to script something with perl (or a similar scripting language).

Here's a Perl solution:

Download TinyPerl (http://tinyperl.sourceforge.net/) and unzip it to c:\tinyperl\tinyperl.
Save the code below to a file called c:\tinyperl\adsb-split.pl. WARNING: no guarantee for correctness !!
run this command

type output.csv | c:\tinyperl\tinyperl c:\tinyperl\adsb-split.pl > splitted.csv

Code:

my $aggregator = ',';
my $separator  = ';';
while (<STDIN>) {
my $line = $_;
chomp($line);
my $f_index = 0;
my $max_subfields = 0;
my @fields;
my @results;
if ($line =~ /$aggregator/) {
my @fields = split($separator,$line);
foreach my $field (@fields) {
    $f_index++;

    if ($field =~ /$aggregator/) {
        @{$result[$f_index]-&gt;{multi}} = split($aggregator,$field);

        $n_subfields = @{$result[$f_index]-&gt;{multi}};
        if ($n_subfields &gt; $max_subfields) {
            $max_subfields = $n_subfields
        }
    } else {
        $result[$f_index]-&gt;{single} = $field;
    }
}

for my $n (1..$max_subfields) {

    foreach my $field (@result) {
        if (exists $field-&gt;{single}) {
            print $field-&gt;{single} . $separator;
        }
        if (exists $field-&gt;{multi}) {
            my $subfield = pop(@{$field-&gt;{multi}});
            print $subfield . $separator;
        }
    }
    print &quot;\n&quot;;
}

} else {
print "$line\n";
}
}

Output Sample:

frame.number;frame.time_epoch;adsb.svid.sac;adsb.svid.domain;adsb.svid.sic;adsb.version.status;adsb.version.number;adsb.lti.unknown;adsb.lti.version;adsb.lti.1090;adsb.lti.uat;adsb.lti.non_adsb;adsb.toap.elapsed.whole;adsb.toap.elapsed.frac;adsb.toa.velocity.sign;adsb.toav.elapsed.frac;adsb.target_address.sv_type;adsb.target_address.dup;adsb.target_address.qualifier;adsb.target_address.address;adsb.iap.utc;adsb.iap.nic;adsb.iap.sil.sup;adsb.iap.sil;adsb.iap.nac_p;adsb.iap.test;adsb.iap.validation;adsb.iap.nac_v.horiz;adsb.iap.nic_baro;adsb.gps_position.lat;adsb.gps_position.lon;adsb.altitude_pressure.resolution;adsb.altitude_pressure.altitude;adsb.velocity_air.vert_src;adsb.velocity_air.scale;adsb.velocity_air.ns_dir;adsb.velocity_air.ns_vel;adsb.velocity_air.ew_dir;adsb.velocity_air.ew_vel;adsb.velocity_air.vert_dir;adsb.velocity_air.vert_vel;adsb.mode3a.validity;adsb.mode3a.code;adsb.target_id;adsb.emitter_category;adsb.target_status.ident;adsb.target_status.surveillance_status;adsb.target_status.emergency_priority_code;adsb.altitude_geometric;adsb.op_mode_cap_code.capability_codes.tcas;adsb.op_mode_cap_code.operational.ident;adsb.op_mode_cap_code.operational.tcas.ra;adsb.tomr.sign;adsb.tomr.ns;adsb.enh.val;adsb.dqp.sda;adsb.dsq.eq_type;adsb.dsq.location_id;adsb.dsq.instance;adsb.target_state
1;1337628248.061542000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1378549;12590923;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;16;3;5443853;
1;1337628248.061542000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1378549;12590923;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;16;2;5443853; 1;1337628248.061542000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1378549;12590923;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;16;1;5443853; 1;1337628248.061542000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1378549;12590923;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;16;0;5443853; 1;1337628248.061542000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1399883;12471098;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;15;3;12480802; 1;1337628248.061542000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1399883;12471098;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;15;2;12480802; 1;1337628248.061542000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1399883;12471098;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;15;1;12480802; 1;1337628248.061542000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1399883;12471098;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;15;0;12480802; 2;1337628248.075203000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1415326;12608481;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;341;0;15150739; 2;1337628248.075203000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1402027;12555896;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;338;3;16390297; 2;1337628248.075203000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1402027;12555896;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;338;2;16390297; 2;1337628248.075203000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1402027;12555896;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;338;1;16390297; 2;1337628248.075203000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1402027;12555896;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;338;0;16390297; 3;1337628248.086516000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1436506;12557234;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;357;0;9940084; 3;1337628248.086516000;12;2;3;0;3;0;2;1;0;0;69701;57;0;127;0;0;1;0xfaafaa;1;10;0;3;10;0;3;9;0;1416410;12512643;0;8192;0;0;0;0;0;0;0;0;;;;;0;0;0;;;;;0;1918076928;0;2;13;356;0;5266782; 4;1337628248.204514000;12;2;3;0;3;0;1;1;0;0;69701;64;0;4;0;0;0;0xa9ea08;0;9;0;2;9;0;3;10;1;1333652;12560133;2;137;0;0;1;609;0;85;1;1;1;2279;40:83:70:cf:60:60;7;0;0;0;592;0;0;0;0;536973395;0;0;13;16;2;5443998

Regards
Kurt

Answer 2

You can use Wireshark's Lua API to print info similar to tshark's output.

test.lua:

local filter = 'ads-b' -- use display filter syntax here
local tap = Listener.new(nil, filter)
– declare field extractors for your named fields
local f_sac = Field.new('adsb.svid.sac')
local f_domain = Field.new('adsb.svid.domain')
local f_sic = Field.new('adsb.svid.sic')
– converts a Field into an array of FieldInfo instances
local function toarray(fld)
local arr = {}
for _,x in ipairs({ fld() }) do
arr[1 + #arr] = x
end
return arr
end
– function called for each packet that matches filter defined above
function tap.packet(pinfo)
local sac       = toarray(f_sac)
local domain    = toarray(f_domain)
local sic       = toarray(f_sic)
-- For each instance of &#39;adsb.svid.sac&#39;, print a line that contains:
--    {frame.number} {frame.time_epoch} {adsb.svid.sac} {adsb.svid.domain} {adsb.svid.sic}
for i=1,#sac do

    -- print() here prints each argument delimited by tabs
    print(
        pinfo.number, -- equivalent to frame.number
        pinfo.abs_ts, -- equivalent to frame.time_epoch
        sac[i],
        domain[i],
        sic[i]
    )

    -- You can also change the format to match tshark&#39;s output
    -- (space-delimited) by using string.format() (which is like 
    -- C&#39;s printf):
    --
    --print(string.format(&quot;%d %.8f %s %s %s&quot;,
    --      pinfo.number, -- equivalent to frame.number
    --      pinfo.abs_ts, -- equivalent to frame.time_epoch
    --      sac[i],
    --      domain[i],
    --      sic[i]
    --))

end

end

Usage:

 tshark -q -r file.pcap -Xlua_script:test.lua

This command assumes file.pcap and test.lua are in the current directory. Use absolute paths here if necessary.

Answer 3

Nope, there is no workaround for this (apart from changing the code). But you can always use 'tshark -V' to get the whole tree. Or (even more bloated) 'tshark -T pdml' to get the XML output, which will be easier to parse.