Ignoring false bytes in tvb

edit

Yeah I have a field called offset and when I first found this I placed a check in the location I noticed the behaviour then exactly as you said went offset+=1.

The problem is, I have since learned this escape character can exist anywhere within the tvb passed from the TCP dissector. As such before every field grab, (which includes 32bit numbers and strings), I would have to scan that region and skip over duplicates. This becomes especially difficult in the case where I have a IEEE 32bit float passed as. 0x00: 0x10: 0x10: 0x80: 0x34. The second 0x10 here should not exist and obviously creates bad data. Since it's in the middle of my float I cannot simple increment offset (at least using the proto_tree_add_item function that I have been working with based on README.Dissector).

Ok, in that case you have to un-byte stuff the whole tvb to create a new tvb and then pass that into the actual dissector function. Imagine the un-byte stuffing as another layer in the protocol stack so you have

...
tcp
byte stuffed data
un-byte stuffed data

Where the byte-stuffed data layer produces the new tvb with the byte stuffing removed, so when you select the un-byte stuffed part of the protocol tree the hex bytes will only show the un-byte stuffed data as it's a new tvb.

I don't think you can have dissection of the byte-stuffed data because of the issues you mention.

I've had to do something similar for DNP3 where the data link layer adds a 16 bit CRC for every 16 bytes of application data. The data link layer function removes those CRC's (after checking them) and creates a new tvb to hand to the application layer function. Both functions are in the same dissector file.

I would expect you to do something similar, at some point you would need to call the un-stuff function and that would either return the new tvb to be dissected in subsequent code, or it could directly call the function to dissect the tvb contents. Which arrangement you use is up to you.

AH I GOT IT! THANK YOU FOR THE HELP!

My original idea to create a second tvb was correct I was just missing one small step (I would prefer to give you credit for the answer though so I'll just paste what I ended up doing).

new_data_len = bsap_duplicate_removal(tvb, &new_data);

trimmed_tvb = tvb_new_real_data(new_data, new_data_len, new_data_len);

tvb_set_child_real_data_tvbuff(tvb, trimmed_tvb);

add_new_data_source(pinfo, trimmed_tvb, "trimmed data");

tvb = trimmed_tvb;

By adding the next to last command I injected the created tvb into the hex debugger section. I did not realize there was a separate command for this I assumed the tvb was handling that somewhere. Thanks again (funnily enough I've been using the DNP3 dissector as reference for a lot of what I've been working on).