when did the BAD ORDER stop working and why? It seems that not supporting it breaks legacy packages.

When?

Not dead sure but I think pre-GCC 4.5. Long ago. Subsequently, the --as-needed option is operative for shared libraries by default, so like static libraries, they must occur in the linkage sequence later than the objects for which they provide definitions. This is a change in the default options that the gcc/g++/gfortran etc. tool-driver passes to ld.

Why?

It was considered confusing to inexpert users that static libraries by default has to appear later that the objects to which they provided definitions while shared libraries by default did not - the difference between the two typically being concealed by the -l<name> convention for linking either libname.a or libname.so.

It was perhaps an unforeseen consequence that inexpert users who had formerly had a lot of luck with the mistaken belief that a GCC [compile and] link command conforms to the normal Unix pattern:

command [OPTION...] FILE [FILE...]

e.g.

gcc -lthis -lthat -o prog foo.o bar.o

now fare much worse with it.

Answer from Mike Kinghan on Stack Overflow
🌐
GNU
gcc.gnu.org › onlinedocs › gcc › Link-Options.html
Link Options (Using the GNU Compiler Collection (GCC))
For predictable results, you must also specify the same set of options used for compilation (-fpie, -fPIE, or model suboptions) when you specify this linker option. ... Link with the POSIX threads library. This option is supported on GNU/Linux targets, most other Unix derivatives, and also on x86 Cygwin and MinGW targets. On some targets this option also sets flags for the preprocessor, so it should be used consistently for both compilation and linking.
🌐
Red Hat
developers.redhat.com › blog › 2018 › 03 › 21 › compiler-and-linker-flags-gcc
Recommended compiler and linker flags for GCC | Red Hat Developer
July 2, 2024 - -Wl,-z,now (also referred to as BIND_NOW) is not recommended for use on Red Hat Enterprise Linux 6 because the dynamic linker processes non-lazy relocations in the wrong order (bug 1398716), causing IFUNC resolvers to fail. IFUNC resolver interactions remain an open issue even for later versions, but -Wl,-z,defs will catch the problematic cases involving underlinking. In RPM builds, some of these flags are injected using -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 and -specs=/usr/lib/rpm/redhat/redhat-hardened-ld because the option selection mechanism in GCC specs allows one to automatically drop the PIE-related flags (for static linking) for PIC builds (for dynamic linking).
🌐
Medium
medium.com › @abhishekjainindore24 › linker-flags-its-options-1119ff6fa7f9
Linker flags, its options. A linker flag is an option passed to… | by Abhishek Jain | Medium
May 27, 2024 - Example: -Wl,-Map,output.map tells the linker to generate a map file named output.map ... # Makefile CFLAGS := $(shell pkg-config --cflags mylib) LDFLAGS := $(shell pkg-config --libs mylib)myprogram: myprogram.o gcc -o myprogram myprogram.o $(LDFLAGS)myprogram.o: myprogram.c gcc -c myprogram.c $(CFLAGS) CFLAGS: Compiler flags, retrieved from pkg-config.
🌐
Linux Man Pages
linux.die.net › man › 1 › ld
ld(1): GNU linker - Linux man page
So for example -omagic sets the output file name to magic whereas --omagic sets the NMAGIC flag on the output. Arguments to multiple-letter options must either be separated from the option name by an equals sign, or be given as separate arguments immediately following the option that requires them. For example, --trace-symbol foo and --trace-symbol=foo are equivalent. Unique abbreviations of the names of multiple-letter options are accepted. Note---if the linker is being invoked indirectly, via a compiler driver (e.g. gcc) then all the linker command line options should be prefixed by -Wl, (or whatever is appropriate for the particular compiler driver) like this:
🌐
RapidTables
rapidtables.com › code › linux › gcc › gcc-l.html
gcc -l -L option flags for library link
gcc -l links with a static library file. gcc -L looks in a directory for library files.
🌐
Stack Exchange
unix.stackexchange.com › questions › 452187 › difference-between-the-linker-flags
linux - Difference between the linker flags - Unix & Linux Stack Exchange
These flags are defined in GCC’s spec files, so the best way to determine the differences between them is to look there: ... The relevant part is the link_command definition. This shows that -nostdlib, -nodefaultlibs and -nostartfiles have the following impact: %{!nostdlib:%{!nodefaultlibs:%:pass-through-libs(%(link_gcc_c_sequence))}} — this adds libgcc, libpthread, libc, libieee as necessary, using a macro and the lib and libgcc spec strings;
Find elsewhere
🌐
GitHub
github.com › ldc-developers › ldc › issues › 1700
Pass through linker flags to GCC · Issue #1700 · ldc-developers/ldc
July 1, 2016 - Changing all the Cflags/Libs entries in pkg-config files will not be possible (may break things, even if I would trick upstream into making the change), and filtering the flags out (which I suppose is what dub does) is only a workaround and only works if the buildsystem supports it.
Published   Aug 17, 2016
Top answer
1 of 3
7

The question is how to determine what linker flag to use from inspection of the source file. The example below will work for Debian. The header files are the relevant items to note here.

So, suppose one has a C source file containing the header

#include <X11/extensions/XInput.h>.

We can do a search for XInput.h using, say apt-file. If you know this header file is contained in an installed package, dpkg -S or dlocate will also work. E.g.

apt-file search XInput.h
libxi-dev: /usr/include/X11/extensions/XInput.h

That tells you that this header file belongs to the development package for libxi (for C libraries, the development packages (normally of the form libname-dev or libname-devel) contain the header files), and therefore you should use the -lxi linker flag.

Similar methods should work for any distribution with a package management system.

2 of 3
4

Core of the Question: Header File vs Library Name

The core of the question was posted in the comments below the question itself:

For XListInputDevices, the appropriate include-line is #include <X11/extensions/XInput.h>. I'm wondering how I can tell based on that what -lSomething flag I need

To answer the question appropriately we need to understand the distinction between -lXi ( or rather libXi.so ) and XInput.h. The XInput.h is a header file and -l looks for object files. According to Red Hat documentation on gcc usage, section 16.1,

A special file name convention is used for libraries: A library known as foo is expected to exist as file libfoo.so or libfoo.a. This convention is automatically understood by the linking input options of GCC, but not by the output options

Both headers and libraries are needed and both are related, but not the same(ref):

... if you do #include the header the compiler does not insert the code for the function you call into program.o. It merely inserts a reference to them.

As mentioned, the headers and linkers are somewhat different, since headers store declarations (or in layman terms - descriptions) of functions and other stuff, and libraries - store actual objects (reference) or compiled version of those functions. As this answer puts it:

  • The header is a phone number you can call, while...
  • ...the library is the actual person you can reach there!

How do the headers and libraries play together ? They are both necessary steps in building an application. First, the description of functions and interfaces is gathered ( aka compiled ) and then the code you write is linked to a library where the actual interfaces and functions are implemented.

The library itself should be compiled with the header included (ref) into the library. In fact you can take a look at an example of how simple static library is created.

So the answer to your question, is as follows: there is no guaranteed relationship between the header and the library name. ( or at least I haven't found any so far ; see also this ref) . And, finding out how to match them is a matter of knowing what's in the package or in the source code.

In fact you can do this to prove the point:

$ echo '#include<errno.h> int main(){return 0;}' | gcc -S -lc

which shows that we're compiling against libc, but the header that is part of libc is different in name.

Another example to prove the point is #include <math.h> , which is part of libc but actual implementation is in libm.so.6 , so you will see code compiled with gcc -lm.

$ apt-file find  /lib/x86_64-linux-gnu/libm.so.6 
libc6: /lib/x86_64-linux-gnu/libm.so.6

Finding the related .so or .a library file

A lot of mysteries can be solved with documentation. Let's refer to gcc documentation for link options (note , emphasis in bold is mine):

-llibrary

-l library

Search the library named library when linking. (The second alternative with the library as a separate argument is only for POSIX compliance and is not recommended.)

The -l option is passed directly to the linker by GCC. Refer to your linker documentation for exact details. The general description below applies to the GNU linker.

OK, so the gcc itself isn't actually looking up the library. It is the linker's job. But let's read on.

The linker searches a standard list of directories for the library. The directories searched include several standard system directories plus any that you specify with -L.

So there is a standard list of directories. Great, this narrows down. How do we get that list ? One way among many is to use ldconfig -v which was generously explained by the user telotorium

But the gcc documentation goes further to explain:

Static libraries are archives of object files, and have file names like liblibrary.a. Some targets also support shared libraries, which typically have names like liblibrary.so. If both static and shared libraries are found, the linker gives preference to linking with the shared library unless the -static option is used.

It makes a difference where in the command you write this option; the linker searches and processes libraries and object files in the order they are specified. Thus, ‘foo.o -lz bar.o’ searches library ‘z’ after file foo.o but before bar.o. If bar.o refers to functions in ‘z’, those functions may not be loaded.

In other words, what you're really instructing gcc and subsequently ld to do, is to look for a file libxi.so (so being a shared object extension) or libxi.a in the list of standard paths. If say we store something in /opt and the ld linker doesn't know about it, passing anything to -l won't work.

The accepted answer seems to hint at this, by virtue of most debian package names for libraries corresponding to that libwhatever.so name. But I cannot confirm or deny that bit, as I know enough about debian packaging for my humble needs only, and not these minute details. What I do know is that apt may trigger ld cache to be rebuilt.

Now, what is also important to realize is that ld actually has all these directories in cache /etc/ld.so.cache. So if the directory is removed from cache, linker won't know of library's existence.


But of course if we want to test all the above theory based on the example of libxi-devel , we can do this:


    $ dpkg-query -L libxi-dev | grep -i '.so'
    /usr/share/man/man3/XIDefineCursor.3.gz
    /usr/lib/x86_64-linux-gnu/libXi.so
    /usr/share/man/man3/XIUndefineCursor.3.gz

and then check ldconfig to confirm that /usr/lib/x86_64-linux-gnu is on the list of searched directories . . . which it is:

$ ldconfig -p | grep libXi.so
    libXi.so.6 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libXi.so.6
    libXi.so.6 (libc6) => /usr/lib/i386-linux-gnu/libXi.so.6
    libXi.so (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libXi.so

Now this also raises a question, is the .h file useless ? No, apparently gcc still needs it to exist at compilation time ( because gcc does check the include-paths and because some libraries can be header-only )

$ mv /usr/include/X11/extensions/XInput.h /tmp
$ echo '#include <X11/extensions/XInput.h> void main(){return 0;}' | gcc -lXi -E -
# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command-line>"
# 31 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 32 "<command-line>" 2
# 1 "<stdin>"
<stdin>:1:36: warning: extra tokens at end of #include directive
<stdin>:1:10: fatal error: X11/extensions/XInput.h: No such file or directory
compilation terminated.
🌐
GNU
sourceware.org › binutils › docs › ld › Options.html
Options (LD)
Instruct the executable or shared library that the all PT_LOAD segments should be sealed to avoid further manipulation (such as changing the protection flags, the segment size, or remove the mapping). This is a security hardening that requires system support. This generates GNU_PROPERTY_MEMORY_SEAL in .note.gnu.property section ... When generating an executable or shared library, mark it to tell the dynamic linker to resolve all symbols when the program is started, or when the shared library is loaded by dlopen, instead of deferring function call resolution to the point when the function is first called.
Top answer
1 of 2
33

As the linker processes each module (be it a library or a object file), it attempts to resolve each undefined symbol while potentially adding to its list of undefined symbols. When it gets to the end of the list of modules, it either has resolved all undefined symbols and is successful or it reports undefined symbols.

In your case, when it processed librt, it had no undefined symbols. Processing proc resulted in clock_gettime being an undefined symbol. gcc will not go back and look in librt for the undefined symbols.

For that reason, you should always have your code first, followed by your libraries, followed by platform provided libraries.

Hope this helps.

2 of 2
19

From the ld (the GNU linker) documentation (http://sourceware.org/binutils/docs/ld/Options.html#Options):

The linker will search an archive only once, at the location where it is specified on the command line. If the archive defines a symbol which was undefined in some object which appeared before the archive on the command line, the linker will include the appropriate file(s) from the archive. However, an undefined symbol in an object appearing later on the command line will not cause the linker to search the archive again.

So if you specify the library too early, the linker will scan it, but not find anything of interest. Then the linker moves on to the object file produced by the compiler and finds references that need to be resolved, but it has already scanned the library and won't bother looking there again.

🌐
Rene Nyffenegger
renenyffenegger.ch › notes › development › languages › C-C-plus-plus › GCC › options › Wl › index
gcc -Wl,linker-option
When I link the program, we want to pass the -t flag to the linker so that it traces the names of the input files that the linker processes. ... /usr/bin/ld: mode elf_x86_64 /usr/lib/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../lib/Scrt1.o /usr/lib/gcc/x86_64-pc-linux-gnu/8.1.1/../../../../lib/crti.o ...
🌐
Gentoo Wiki
wiki.gentoo.org › wiki › Project:Quality_Assurance › As-needed
Project:Quality Assurance/As-needed - Gentoo wiki
If you want to use the --as-needed ... add the flag back; check emerge --info | grep LDFLAGS to ensure that it is there, if not, you'll want to use the below code example to add it. Note LDFLAGS are generally passed not directly to ld but to gcc, so you have to use the -Wl, prefix to pass them back to the linker...
🌐
Foss-for-synopsys-dwc-arc-processors
foss-for-synopsys-dwc-arc-processors.github.io › man-pages › gcc › Link-Options.html
Link Options - Using the GNU Compiler Collection (GCC)
For predictable results, you must also specify the same set of options used for compilation (-fpie, -fPIE, or model suboptions) when you specify this linker option. ... Link with the POSIX threads library. This option is supported on GNU/Linux targets, most other Unix derivatives, and also on x86 Cygwin and MinGW targets. On some targets this option also sets flags for the preprocessor, so it should be used consistently for both compilation and linking.