Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Lua dissector memory-efficient packet reassembly

I'd like to use Wireshark to analyse an RS-485 based protocol. I'd like to use a generic tool such as GitHub j123b567/SerialPCAP to save the data to a pcap file with a DLT_USER_0 data type, then analyse it with Wireshark, using a custom dissector written in Lua (ideally).

Due to the way the generic capture tool works, frames in the pcap file don't necessarily coincide with protocol packets. So, it is necessary to search for valid packets that might start on any byte within the pcap frame, or might span 2 or more frames. Valid packets can be determined by checking if the byte sequence starting at a particular byte has a valid length and CRC-16 value at the end. I have succeeded in writing a Lua dissector that does this.

However, I am not so easily able to handle protocol packets that span 2 or more pcap frames. I could do manual frame reassembly by storing frame data in a Lua list, however, this takes significant memory so doesn't scale up well to large captures. I've read about TCP packet reassembly in Lua, and tried it, but it doesn't seem to work for dissecting DLT_USER_0 data. Also I've read some information that suggests that Wireshark's TCP packet reassembly also takes a large amount of memory, so if that would be the case for DLT files also, that's not so good.

The maximum protocol packet size is 31 bytes, so ideally I'd like the dissector to simply be able to look back to the data of previous pcap frames, up to 31 bytes' worth, to potentially reassemble valid packets.

Is it possible to do memory-efficient packet reassembly, by simply looking back to a small number of previous pcap frames as needed to find valid protocol packets?

If that's not possible, I guess I'd need to make a smarter capture tool, which is able to detect protocol packets and save them into pcap frames.

Lua dissector memory-efficient packet reassembly

I'd like to use Wireshark to analyse an RS-485 based protocol. I'd like to use a generic tool such as GitHub j123b567/SerialPCAP to save the data to a pcap file with a DLT_USER_0 data type, then analyse it with Wireshark, Wireshark using a custom dissector dissector, written in Lua (ideally).

Due to the way the generic capture tool works, frames in the pcap file don't necessarily coincide with protocol packets. So, it is necessary to search for valid packets that might start on any byte within the pcap frame, or might span 2 or more frames. Valid packets can be determined by checking if the byte sequence starting at a particular byte has a valid length and CRC-16 value at the end. I have succeeded in writing a Lua dissector that does this.

However, I am not so easily able to handle protocol packets that span 2 or more pcap frames. I could do manual frame reassembly by storing frame data in a Lua list, however, this takes significant memory so doesn't scale up well to large captures. I've read about TCP packet reassembly in Lua, and tried it, but it doesn't seem to work for dissecting DLT_USER_0 data. Also I've read some information that suggests that Wireshark's TCP packet reassembly also takes a large amount of memory, so if that would be the case for DLT files also, that's not so good.

The maximum protocol packet size is 31 bytes, so ideally I'd like the dissector to simply be able to look back to the data of previous pcap frames, up to 31 bytes' worth, to potentially reassemble valid packets.

Is it possible to do memory-efficient packet reassembly, by simply looking back to a small number of previous pcap frames as needed to find valid protocol packets?

If that's not possible, I guess I'd need to make a smarter capture tool, which is able to detect protocol packets and save them into pcap frames.

Lua dissector memory-efficient packet reassembly

I'd like to use Wireshark to analyse an RS-485 based protocol. I'd like to use a generic tool such as GitHub j123b567/SerialPCAP to save the data to a pcap file with a DLT_USER_0 data type, then analyse it with Wireshark using a custom dissector, written in Lua (ideally).

Due to the way the generic capture tool works, frames in the pcap file don't necessarily coincide with protocol packets. So, it is necessary to search for valid packets that might start on any byte within the pcap frame, or might span 2 or more frames. Valid packets can be determined by checking if the byte sequence starting at a particular byte has a valid length and CRC-16 value at the end. I have succeeded in writing a Lua dissector that does this.

However, I am not so easily able to handle protocol packets that span 2 or more pcap frames. I could do manual frame reassembly by storing frame data in a Lua list, however, this takes significant memory so doesn't scale up well to large captures. I've read about TCP packet reassembly in Lua, and tried it, but it doesn't seem to work for dissecting DLT_USER_0 data. Also I've read some information that suggests that Wireshark's TCP packet reassembly also takes a large amount of memory, so if that would be the case for DLT files also, that's not so good.

The maximum protocol packet size is 31 bytes, so ideally I'd like the dissector to simply be able to look back to the data of previous pcap frames, up to 31 bytes' worth, to potentially reassemble valid packets.

Is it possible to do memory-efficient packet reassembly, by simply looking back to a small number of previous pcap frames as needed to find valid protocol packets?

If that's not possible, I guess I'd need to make a smarter capture tool, which is able to detect protocol packets and save them each packet into pcap frames.its own pcap frame.

Lua dissector memory-efficient packet reassembly

I'd like to use Wireshark to analyse an RS-485 based protocol. I'd like to use a generic tool such as GitHub j123b567/SerialPCAP to save the data to a pcap file with a DLT_USER_0 data type, then analyse it with Wireshark using a custom dissector, written in Lua (ideally).

Due to the way the generic capture tool works, frames in the pcap file don't necessarily coincide with protocol packets. So, it is necessary to search for valid packets that might start on any byte within the pcap frame, or might span 2 or more frames. Valid packets can be determined by checking if the byte sequence starting at a particular byte has a valid length and CRC-16 value at the end. I have succeeded in writing a Lua dissector that does this.

However, I am not so easily able to handle protocol packets that span 2 or more pcap frames. I could do manual frame reassembly by storing frame data in a Lua list, however, this takes significant memory so doesn't scale up well to large captures. I've read about TCP packet reassembly in Lua, and tried it, but it doesn't seem to work for dissecting DLT_USER_0 data. Also I've read some information that suggests that Wireshark's TCP packet reassembly also takes a large amount of memory, so if that would be the case for DLT files also, that's not so good.

The maximum protocol packet size is 31 bytes, so ideally I'd like the dissector to simply be able to look back to the data of previous pcap frames, up to 31 bytes' worth, to potentially reassemble valid packets.

Is it possible to do memory-efficient packet reassembly, by simply looking back to a small number of previous pcap frames as needed to find valid protocol packets?

If that's not possible, I guess I'd need to make a smarter capture tool, which is able to detect packetise protocol packets from the serial stream and save each packet into its own pcap frame.