How do I compile Wireshark with Lua and libsmi in Redhat?

• is there a file called '/home/OPENSOURCE/LUA/5.1/Linux/RHEL4_2.6/lib/liblua.a'?
• What is its size?
• what is the output of this command: nm /home/OPENSOURCE/LUA/5.1/Linux/RHEL4_2.6/lib/liblua.a
• can you please add a few lines of make output before that last error message.

Regards
Kurt

This is probably the issue you're seeing:

libtool: link:  gcc -shared  -fPIC -DPIC  .libs/libwireshark_la-addr_and_mask.o .libs/libwireshark_la-addr_resolv.o 
[...]
-pthread -Wl,-soname -Wl,libwireshark.so.0 -Wl,-version-script -Wl,.libs/libwireshark.ver -o .libs/libwireshark.so.0.0.0
/usr/bin/ld: /home/tony/src/lua-5.1.5/src/build/lib/liblua.a(lapi.o): relocation R_X86_64_32 against `luaO_nilobject_' can not be used when making a shared object; recompile with -fPIC
/home/tony/src/lua-5.1.5/src/build/lib/liblua.a: could not read symbols: Bad value
[...]
make: *** [install] Error 2

The error message was actually quite helpful. It was telling you to recompile liblua.a with the -fPIC compiler flag. Try this:

1. Open ${lua-5.1}/src/Makefile
2. Edit CFLAGS (it should be on/near line 11) so that it includes -fPIC. (e.g., CFLAGS=-fPIC -O2 -Wall $(MYCFLAGS))
3. Save the Makefile.
4. Recompile liblua (and install it).

You now should be able to finish the linker process in the Wireshark build.

It's easier to use the RHEL package manager.

Before I even tried the above, I played around with installing a Wireshark dev environment on CentOS (a derivative of RHEL). It's simple and painless. I didn't need to build liblua from source as it's already available as a package. The same is true for libsmi. The following commands worked from a fresh install of a CentOS VM:

#!/bin/sh
########################################################################
# This script installs the tools necessary for Wireshark development.
# If the tools are already installed, the package manager is smart
# enough to not re-install.
########################################################################

# Install build tools (gcc, gdb, flex, etc.)
sudo yum install 'Development Tools'

# Required packages (any dependencies get pulled in automatically)
sudo yum install gtk2-devel.x86_64 libpcap-devel.x86_64

# Optional packages
sudo yum install lua-devel.x86_64 libsmi-devel.x86_64

Note that the package manager only has Lua 5.1.4 available. If you need any other version (such as 5.1.5), you'll have to build from source as you were doing earlier.