LD_DEBUG

The dynamic library loader used in linux (part of glibc) has some neat tricks. One of these is that you can set an environment variable called

LD_DEBUG

to show how symbols (variables and functions, for example) are resolved for a dynamic executable. This can sometimes help resolve obscure bugs where your application isn’t doing what you expect (assuming it is caused by symbols being resolved differently to what you were expecting).

This is very useful if you get segmentation violations or aborts for a program - this can sometimes be caused by linking against the wrong version of a library. This is also a really good way to understand what happens when you run any program! It has some self-documentation - for the impatient, you can do

$ LD_DEBUG=help /path/to/some/dynamic/executable

eg

$ LD_DEBUG=help ls

prints out:

Valid options for the LD_DEBUG environment variable are:

libs display library search paths
reloc display relocation processing
files display progress for input file
symbols display symbol table processing
bindings display information about symbol binding
versions display version dependencies
all all previous options combined
statistics display relocation statistics
help display this help message and exit

To direct the debugging output into a file instead of standard output a
filename can be specified using the LD_DEBUG_OUTPUT environment variable.

As a quick example of what it does:

$ LD_DEBUG=all ls 2>&1 > /dev/null | less

13442:
13442: file=librt.so.1; needed by ls
13442: find library=librt.so.1; searching
13442: search cache=/etc/ld.so.cache
13442: trying file=/lib/librt.so.1
13442:
13442: file=librt.so.1; generating link map
13442: dynamic: 0×400263ec base: 0×40020000 size: 0×00010d14
13442: entry: 0×400219c0 phdr: 0×40020034 phnum: 6
13442:
13442:
13442: file=libc.so.6; needed by ls
13442: find library=libc.so.6; searching
13442: search cache=/etc/ld.so.cache
13442: trying file=/lib/libc.so.6
13442:
13442: file=libc.so.6; generating link map
13442: dynamic: 0×40146ce4 base: 0×40031000 size: 0×0011ab00
13442: entry: 0×4004a184 phdr: 0×40031034 phnum: 6
13442:
…
13442: checking for version `GLIBC_2.2′ in file /lib/librt.so.1 required by file ls
13442: checking for version `GLIBC_2.1′ in file /lib/libc.so.6 required by file ls
13442: checking for version `GLIBC_2.2.3′ in file /lib/libc.so.6 required by file ls
…
13442: relocation processing: /lib/libpthread.so.0 (lazy)
13442: symbol=_errno; lookup in file=ls
13442: symbol=_errno; lookup in file=/lib/librt.so.1
13442: symbol=_errno; lookup in file=/lib/libc.so.6
13442: symbol=_errno; lookup in file=/lib/libpthread.so.0
13442: symbol=_errno; lookup in file=/lib/ld-linux.so.2
13442: binding file /lib/libpthread.so.0 to /lib/libc.so.6: normal symbol `_errno’ [GLIBC_2.0]
13442: symbol=_h_errno; lookup in file=ls
13442: symbol=_h_errno; lookup in file=/lib/librt.so.1
13442: symbol=_h_errno; lookup in file=/lib/libc.so.6
13442: symbol=_h_errno; lookup in file=/lib/libpthread.so.0
13442: symbol=_h_errno; lookup in file=/lib/ld-linux.so.2
13442: binding file /lib/libpthread.so.0 to /lib/libc.so.6: normal symbol `_h_errno’ [GLIBC_2.0]
…

In other words, every single function and external variable in the standard library that ls(1) uses must be located each time it is run (kind of obvious, really).

$ ldd /bin/ls
librt.so.1 => /lib/librt.so.1 (0×40020000)
libc.so.6 => /lib/libc.so.6 (0×40031000)
libpthread.so.0 => /lib/libpthread.so.0 (0×4014c000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0×40000000)