This is a static archive of our old Q&A Site. Please post any new questions and answers at ask.wireshark.org.

UInt64 bitwise and operation wrong in Windows

0

UInt64 is supposed be able to bitwise AND with another 64 bit integer. The API says:

uint64:band(arg1 [, arg2 [, ...]])
arg1 : number|UInt64|UUInt64|string
The number/UInt64/UIn64/hex-string to bitwise 'and' the UInt64 with.

However, it doesn't seems to work correctly on the Windows platform.

See the code below:

str = "3f91df0b2b89dd1e"
num = UInt64.fromhex(str)
print(num)
newNum = num:band(0xffffffff00000000)
print(newNum)

In Mac, my tshark result is:

4580687535080594718
4580687534350139392  -- correct

In Windows, the same code using tshark gets:

4580687535080594718
730455326 -- wrong

It seems windows can only handle 32 bit mask, not 64.

asked 21 Mar '14, 08:31

YXI's gravatar image

YXI
21●18●20●23
accept rate: 0%

edited 21 Mar '14, 09:06

Hadriel's gravatar image

Hadriel
2.7k●2●9●39

Please post the "About Wireshark" output for your windows wireshark.

(21 Mar '14, 09:11) Hadriel

Version 1.11.3-1851-gb2689ab (wireshark-1.11.3-rc1-1851-gb2689ab-dirty from master)

Copyright 1998-2014 Gerald Combs [email protected] and contributors. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiled (32-bit) with GTK+ 3.4.4, with Cairo 1.10.2, with Pango 1.30.1, with GLib 2.32.4, with WinPcap (4_1_3), with libz 1.2.5, with SMI 0.4.8, with c-ares 1.9.1, with Lua 5.1, without Python, with GnuTLS 2.12.18, with Gcrypt 1.4.6, with MIT Kerberos, with GeoIP, with PortAudio V19-devel (built Mar 4 2014), with AirPcap.

Running on 32-bit Windows 7 Service Pack 1, build 7601, with WinPcap version 4.1.3 (packet.dll version 4.1.0.2980), based on libpcap version 1.0 branch 1_0_rel0b (20091008), GnuTLS 2.12.18, Gcrypt 1.4.6, without AirPcap. Intel(R) Core(TM) i7-3615QM CPU @ 2.30GHz, with 2815MB of physical memory.

Built using Microsoft Visual C++ 10.0 build 40219

Wireshark is Open Source Software released under the GNU General Public License.

Check the man page and http://www.wireshark.org for more information.

(21 Mar '14, 09:23) YXI

Then it's probably because it's a 32-bit build you're using, as opposed to Mac/Linux vs. Windows (maybe). If the windows system you're running is 64-bit capable, can you try a 64-bit build of wireshark?

Still, it should have worked, so I'll look into it as well.

(21 Mar '14, 09:41) Hadriel

BTW, you shouldn't really do:

newNum = num:band(0xffffffff00000000)

because a raw 0xffffffff00000000 is a Lua number, and bigger than 53 bits of precision. (this is similar to a previous example you gave in a previous question thread)

(21 Mar '14, 09:45) Hadriel

One Answer:

1

I've reproduced this on a 32-bit Windows XP system. The problem is Lua itself is treating the 0xffffffff00000000 as a 32-bit number, even though it should be a double even on 32-bit machines.

For example, try this:

print(0xffffffff00000000)

That prints out 4294967295 instead of 1.8446744069415e+19.

As a work-around (and in fact the right way to do band with such a large mask number), do one of these:

newNum = num:band(UInt64.fromhex("ffffffff00000000"))
newNum = num:band(UInt64(0, 0xffffffff))

answered 21 Mar '14, 10:22

Hadriel's gravatar image

Hadriel
2.7k●2●9●39
accept rate: 18%

Interesting - it's a limitation in Lua 5.1 itself... I'd even call it a bug in Lua 5.1, which are super-rare these days, but apparently it's been discussed on the mailing list and isn't considered a bug. But basically on a 32-bit system, literal hex numbers (such as 0xffffffff00000000) are truncated to 32-bits big.

So yet another reason not to use a number that big as a Lua number, but instead create a UInt64/Int64 out of two smaller numbers or out of a string.

(21 Mar '14, 10:48) Hadriel

Your suggestion made sense, but it didn't work for some reason. I ran this code in tshark in the Mac (didn't try Windows):

str = "3f91df0b2b89dd1e"

value = UInt64.fromhex(str)

myMask = 0xffffffff00000000

num = value:band(myMask)

print(num)

numNew = value:band(UInt64.fromhex(myMask))

print(numNew)

The result:

4580687534350139392

0

So directly using the big number worked, but passing in the 64bit type returned 0.

(21 Mar '14, 12:58) YXI

Because you're doing: UInt64.fromhex(myMask), and myMask is not a hex string in your example, but rather a Lua number. (and it's a number that won't be the same number on 32-bit machines apparently)

Do this instead:

-- the value
local value = UInt64.fromhex("3f91df0b2b89dd1e")

– myMask is a UInt64, not a hex string
local myMask = UInt64.fromhex("ffffffff00000000")


– so this
local num = value:band(myMask)
print(num)


– or this
local numNew = value:band(UInt64.fromhex("ffffffff00000000"))
print(numNew)


– or this
local otherNum = value:band(UInt64(0, 0xffffffff))
print(otherNum)
(21 Mar ‘14, 13:19) Hadriel

Yes! Feel so stupid today. Sorry for wasting your time. Good news is now it works in Windows. Finally got Windows to give me the right numbers. Thank you so much!!!

(21 Mar ‘14, 14:02) YXI

Sure no problem.

(21 Mar ‘14, 14:06) Hadriel