This is a static archive of our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

number bigger than 64 bits

0

Hi,

Just wondering if anyone has to write a dissector in Lua that contains values bigger than 64 bit. How do you do it.

Thanks.

asked 03 Mar '14, 18:23

YXI's gravatar image

YXI
21182023
accept rate: 0%

retagged 04 Mar '14, 04:24

Hadriel's gravatar image

Hadriel
2.7k2939


2 Answers:

0

To do what with? Just to show it in the display tree? Have it be filterable by value? Or to actually perform some arithmetic on it?

If it's just to display in the tree, I suppose you can treat it as a byte string. You can even show it's numerical value by processing each byte in Lua and building/concatenating a string representing the numeric value, that you add to the tree item using Lua. (it would take some careful Lua coding to make such a bignum string, but it's possible)

But to have it be filterable by numeric value (i.e., have a display filter like 'myproto.bugnum = 42446744073709551615') isn't possible as far as I know, since there's no filter type for numbers bigger than 64-bit.

There's no way of performing Lua arithmetic (ie, add, subtract, divide) on numbers that big directly as a Lua number, because Lua itself only handles numbers as doubles; and though a double can certainly represent massive numbers, it loses precision above 53-bits (or 52 bits... I always forget which). As of Wireshark 1.11.3 we now have Lua arithmetic for 64-bit numbers, as Int64/UInt64 userdata objects, but nothing bigger. So if you need to handle something bigger, you'll have to write the Lua code to do it based on smaller number chunks of the bigger number.

answered 04 Mar '14, 04:23

Hadriel's gravatar image

Hadriel
2.7k2939
accept rate: 18%

Thanks Hadriel. I do need to do a lot of stuff on the big numbers (bit operations such as shifting, masking), change endianess, and also type conversion, e.g. interpreting an unsigned long long number as a double. I think this is definitely beyond the capability of Wireshark Lua even for the newest development version. I know Lua has libraries that handle numbers to an arbitrary size, but I haven't seen anyone using it writing Wireshark dissectors. In reality, most of my values are going to be capped at 64 bits, so I will probably ignore the really big numbers for now. I'm still using wireshark 1.10.2. I think I will have to move to 1.11.2 to do all the 64 bit stuff. However, since it's not a stable release, I'm hesitating to do it. You can probably tell I'm still a novice to the whole thing. But how stable is the development release? Any serious risks?

Thanks so much, YXI

(04 Mar '14, 08:14) YXI

Well... it's fairly stable, if you keep to the automated nightly builds instead of just pulling down the latest source code and compiling yourself. But the 64-bit stuff is only in 1.11.3, not 1.11.2, so you can only get it from the nightly builds here: http://www.wireshark.org/download/automated/

Info on the Int64/UInt64 is here: http://wiki.wireshark.org/LuaAPI/Int64

If you need some new Lua function/class/whatever exposed into Wireshark, just submit a bugzilla enhancement request, or post it in a new question here with the tag 'lua'. I've been on a kick lately exposing more stuff into Lua. The changes merged into 1.11.3 so far are documented here: http://wiki.wireshark.org/Lua/ApiChanges

Which bignum library for Lua would you like to see in Wireshark?

(04 Mar '14, 08:44) Hadriel

Hi Hadriel,

I haven't decided which one I like (I looked at both lbc and lmapm). So you may need to wait for more input from the community on this one.

I saw that you committed changes very recently to add Lua struct (pack, unpack, etc) functionalities to Wireshark. Are they ready to be used? will they work on 64 bit numbers? I do need to do endianness swaps, converting signed to unsigned number, long to double, etc. I think using the struct library may be the easiest way to go. Is that true?

Thanks, YXI

(04 Mar '14, 09:47) YXI

I looked into lbc and lmapm (and lbn and lqd and lgmp), and the easiest to include in Wireshark would be lbc, by far. Unfortunately it's also probably the slowest. :(

I assume though that raw speed isn't that important?

(04 Mar '14, 11:01) Hadriel

Speed is not important to me. Neither of these two have bit operation APIs though.

(04 Mar '14, 16:49) YXI

No, but I'd add that. I added it for Int64/UInt64, for example.

(04 Mar '14, 19:22) Hadriel
showing 5 of 6 show 1 more comments

0

Yes, Struct is in the latest nightly builds, and has Int64/UInt64 support by using 'e'/'E' in the pattern. Note that it's "Struct" not "struct", however - I didn't want to collide with someone's use of the real struct library if they were already using it. Also, our auto-generated docs for it are here, but it's not very helpful. I've been hoping to update the doc generator for the API, because we're very limited in the type of documentation that can be generated for the API. (it's auto-generated by a Perl script from source code)

So here are some rough notes from the source file:

** Some changes are based on a patch to struct.h from
** Flemming Madsen, from here:
** http://lua-users.org/lists/lua-l/2009-10/msg00572.html
** In particular, these changes from him:
** -Can handle 'long long' integers (i8 / I8); though they're converted to doubles
** -Can insert/specify padding anywhere in a struct. ('X' eg. when a string is following a union)
** -Can report current offset in both pack and unpack ('=')
** -Can mask out return values when you only want to calculate sizes or unmarshal pascal-style strings. '(' & ')'
**
** Changes I made:
** -Added support for Int64/UIn64 being packed/unpacked, using 'e'/'E'
** -Made it follow Wireshark's conventions so we could get API docs
** =======================================================
*/
/*
** Valid formats:
** > - big endian
** < - little endian
** ![num] - alignment
** x[num]   - pad num bytes, default 1
** X[num]   - pad to num align, default MAXALIGN
**
** Following are system-dependent sizes:
** b/B - signed/unsigned byte
** h/H - signed/unsigned short
** i/I - signed/unsigned int
** l/L - signed/unsigned long
** f - float
** d - double
** T   - size_t
**
** Following are system-independent sizes:
** in/In - signed/unsigned integer of size `n' bytes
          Note: unpack of i/I is done to a Lua_number, typically a double,
          so unpacking a 64-bit field (i8/I8) will lose precision.
          Use e/E to unpack into a Wireshark Int64/UInt64 object/userdata instead.
** e/E - signed/unsigned eight-byte Integer (64bits, long long), to/from Int64/UInt64 object
** cn - sequence of `n' chars (from/to a string); when packing, n==0 means
        the whole string; when unpacking, n==0 means use the previous
        read number as the string length
** s - zero-terminated string
** ' ' - ignored
** '(' ')'  - stop assigning items. ')' start assigning (padding when packing)
** '='      - return current position / offset
*/

answered 04 Mar '14, 09:58

Hadriel's gravatar image

Hadriel
2.7k2939
accept rate: 18%

Oh, and I added a tohex()/fromhex() function, as documented in the API docs.

(04 Mar '14, 10:01) Hadriel

This is great. I really need it. Will let you know how it goes.

(04 Mar '14, 16:48) YXI

Hadriel and others,

If I simply have these two lines of code:

a = UInt64.new("ff000000")

b = a.rshift(2)

I get: Lua: Error During execution of dialog callback: [string "a = UInt64.new("ff000000")..."]:2: bad argument #2 to 'rshift' (number expected, got no value)

What did I do wrong?

Thanks.

(05 Mar '14, 14:59) YXI

You called 'rshift()' with a single argument - a number 2 - instead of with 2 arguments: the 'a' object, and the number 2. You need to use a : colon instead of . period, because a colon is basically Lua shorthand for a.rshift(a,2).

So you should do this:

a = UInt64.new(0xff000000)
b = a:rshift(2)

Note also I didn't put the ff000000 in quotes, as that would make it a string, and although UInt64/Int64 new() accepts a string, it decodes it in base 10, which means it has to be a string of digits, not hex. Maybe I should change it to decode in base 0, so you could use hex inside (if you prepend with 0x).

Also, you don't need to use the 'new' method - you can just call it like this:

a = UInt64(0xff000000)
b = a:rshift(2)
(05 Mar '14, 15:16) Hadriel

BTW, I think your post and mine were just changed to comments, because this ask.wireshark.org site isn't really a forum model of topic threads, but instead a question+answer model site. So each topic is a single question and one or more answers to that question. So you should ask future questions as new, separate topics, not in this one.

(everyone makes that mistake, but the folks behind ask.wireshark.org are refusing to yield to how everyone else communicates naturally, under the hope that by keeping it a question+answer model then people will search for their question being already answered previously. It's a nice dream. :)

(05 Mar '14, 15:36) Hadriel

Thanks Hadriel. Sorry for my mistake. You can probably tell I'm still new to Lua. The funny thing was I did not get any error messages using "." instead of ":" on the band() and bor() functions, with only one argument, like b=a.band(0xff). That's why I didn't even suspect my syntax was wrong for the shift operations.
Anyway, thanks for the explanations. I will start a new question next time I change the topic.

(05 Mar '14, 18:20) YXI

Hmmm.. well technically that's because they're not errors - you just created a new UInt64 of value 0xff and didn't and/or it with anything. The and/or/xor functions all take any number of arguments, including only one argument. So when you do b=a.band(0xff), you're really calling band() with just the 0xff argument, and thus creating a new UInt64 of value 0xff, and assigning it to variable b. The problem you had before was the rshift() function, which must have two arguments, but you were only giving it one. For example, you could have done this: b=UInt64.rshift(0xff000000, 2) and it would have worked as well.

I know that's a bit confusing, not being consistent in different functions, but I was trying to mimic exactly what the Lua bitop library does for all of these functions, since Wireshark has the bitop library available in Lua but it only handles up to 32bit integers... and the bitop library does exactly the same thing Int64/UIn64 does for and/or/xor vs. rshift and the others.

(05 Mar '14, 19:04) Hadriel

@Hadriel, @YXI

Yep, the mods do try to keep things as a Q&A. User assistance in keeping it that way would be appreciated. StackOverflow etc. seem to do OK with that model. I think we need to have a start page that consists of nothing but a search box, then users might actually use it. :-)

It's sometime difficult to determine if a supplementary question should be promoted to its own question, but that's a judgement call.

(06 Mar '14, 01:42) grahamb ♦
showing 5 of 8 show 3 more comments