GNU bug report logs - #7621
add a way to pass link dependencies

Previous Next

To reply to this bug, email your comments to 7621 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-automake <at> gnu.org
Subject: add a way to pass link dependencies
Date: Sun, 12 Dec 2010 15:43:00 +0100

Hi,

When creating an executable or library, sometimes other libraries
have to be mentioned on the link command line as dependencies.

For dependencies inside the build tree, Automake handles this fine,
through the _LDADD variable for executables and the _LIBADD variable
for libraries. See e.g. the node "Linking" in the Automake manual.

But for dependencies on libraries installed on the system, Automake
does not provide a way to specify these link dependencies in a complete,
reliable, and simple way. Please consider adding support for this
to a future version of Automake.

Let's take an example: You want to link with the libiconv library,
which may be installed in /usr/lib, in /usr/local/lib, in /opt/gnu/lib,
or similar. GNU gettext has a macro AM_ICONV which finds the library
and defines Autoconf variables LIBICONV (for use without libtool) and
LTLIBICONV (for use with libtool) that contain the needed linker options
or libtool options, respectively. Similarly for other libraries.
The problem is that is no way to use these with Automake in a reliable
and simple way.

Let me clarify the requirements:

1) I want to use the value of LIBICONV or LTLIBICONV. Typical values
of LIBICONV are

    -liconv
    -Wl,-rpath,/usr/local/lib -L/usr/local/lib -liconv
    /usr/local/lib/libiconv.a
    -Wl,-rpath,/usr/local/lib /usr/local/lib/libiconv.so

Typical values of LTLIBICONV are

    -liconv
    -rpath /usr/local/lib -L/usr/local/lib -liconv
    /usr/local/lib/libiconv.a
    -rpath /usr/local/lib /usr/local/lib/libiconv.so
    /usr/local/lib/libiconv.la

In particular, I can't get away without -rpath options, otherwise
the created programs just won't run.

2) Linking with a static library should work in the same way as
linking with a shared library.

3) I want to be able to specify the link dependencies for every
program and library separately. I don't want to put them into $(LIBS),
because dependencies on unused shared libraries increase the startup
time for programs, and the --as-needed option of the GNU linker is not
portable and carries its own set of problems.[1]

4) I want to use the Automake-provided rules for linking. I don't want
to override them, because that would be a maintenance hassle as Automake
evolves.

Now in which Automake provided variable could I stuff these linker options
or libtool options?


How to reproduce:
In a package that already has a configure.ac that uses Automake but not
Libtool, do "mkdir foo" and save this text as foo/Makefile.am:
=============================== foo/Makefile.am ========================
bin_PROGRAMS = maude
lib_LIBRARIES = library.a

maude_CFLAGS = $(AM_CFLAGS)
maude_LDFLAGS = $(AM_LDFLAGS)
maude_LDADD = $(LDADD)

#library_a_CFLAGS = $(AM_CFLAGS) # not used in linking
#library_a_LDFLAGS = $(AM_LDFLAGS) # yields warning
library_a_LIBADD =
========================================================================
then do "automake foo/Makefile" and look at the generated Makefile.in file.

In a package that already has a configure.ac that uses Automake and Libtool,
do "mkdir foolt" and save this text as foolt/Makefile.am:
============================== foolt/Makefile.am =======================
bin_PROGRAMS = maudelt
lib_LTLIBRARIES = librarylt.la

maudelt_CFLAGS = $(AM_CFLAGS)
maudelt_LDFLAGS = $(AM_LDFLAGS)
maudelt_LDADD = $(LDADD)

librarylt_la_CFLAGS = $(AM_CFLAGS)
librarylt_la_LDFLAGS = $(AM_LDFLAGS)
librarylt_la_LIBADD =
========================================================================
then do "automake foolt/Makefile" and look at the generated Makefile.in file.

The generated link commands are (with expanded *_LINK and *_AR variables):

For maude:
  $(CCLD) $(maude_CFLAGS) $(CFLAGS) \
          $(maude_LDFLAGS) $(LDFLAGS) \
          -o $@ \
          $(maude_OBJECTS) \
          $(maude_LDADD) $(LIBS)

For library.a:
  $(AR) $(ARFLAGS) library.a $(library_a_OBJECTS) $(library_a_LIBADD)
  $(RANLIB) library.a

For maudelt:
  $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link \
  $(CCLD) $(maudelt_CFLAGS) $(CFLAGS) \
          $(maudelt_LDFLAGS) $(LDFLAGS) \
          -o $@ \
          $(maudelt_OBJECTS) \
          $(maudelt_LDADD) $(LIBS)

For librarylt.la:
  $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link \
  $(CCLD) $(librarylt_la_CFLAGS) $(CFLAGS) \
          $(librarylt_la_LDFLAGS) $(LDFLAGS) \
          -o $@ \
          -rpath $(libdir) \
          $(librarylt_la_OBJECTS) \
          $(librarylt_la_LIBADD) $(LIBS)

When creating library.a, one can ignore dependencies to libraries not in the
build tree. So we need consider only maude, maudelt, and librarylt.la.
Where could I stuff the linker options or libtool options?

- Can I put them in the *_LDFLAGS variable?

  No, it does not work, because if it's a static library (/somewhere/libiconv.a),
  the linker will not pick any object files from it, because it precedes
  the *_OBJECTS on the link command line. Ralf noticed that in [2].

- Can I put them in the *_LDADD variable for programs or *_LIBADD variable
  for libraries?

  No, because the options contain other things than -l and -L options, and
  these can't go in *_LDADD or *_LIBADD variables, says the Automake doc.
  [3][4][5]
  I understand that the true reason for this requirement is that Automake
  wants to compute a *_DEPENDENCIES variable from *_LDADD or *_LIBADD variables:

    "If `PROG_DEPENDENCIES' is not supplied, it is computed by Automake.
     The automatically-assigned value is the contents of `PROG_LDADD', with
     most configure substitutions, `-l', `-L', `-dlopen' and `-dlpreopen'
     options removed.  The configure substitutions that are left in are only
     `$(LIBOBJS)' and `$(ALLOCA)'; these are left because it is known that
     they will not cause an invalid value for `PROG_DEPENDENCIES' to be
     generated.

    "If `_DEPENDENCIES' is not supplied, it is computed by Automake.
     The automatically-assigned value is the contents of `_LDADD' or
     `_LIBADD', with most configure substitutions, `-l', `-L',
     `-dlopen' and `-dlpreopen' options removed.  The configure
     substitutions that are left in are only `$(LIBOBJS)' and
     `$(ALLOCA)'; these are left because it is known that they will not
     cause an invalid value for `_DEPENDENCIES' to be generated."

  Additionally, for libraries, the description of LIBADD is strict
  about the fact that it "should be used to list extra libtool objects
  (.lo files) or libtool libraries (.la) to add to library."

- Can I put them in LIBS?

  No, I explained above why I don't want this.

Please offer a simple way to use these linker options.

I can see two choices:

 A) Either allow -Wl,-rpath or -rpath options also in *_LDADD variables,
    and allow -Wl,-rpath, -rpath, -l, -L options also in *_LIBADD variables
    of libraries built with libtool.

 B) Or introduce a new set of variables *_LIBS that can contain link options
    for dependencies, like $(LIBICONV) and $(LTLIBICONV) above. Modify the
    link commands to look like this:

    For maude:
      $(CCLD) $(maude_CFLAGS) $(CFLAGS) \
              $(maude_LDFLAGS) $(LDFLAGS) \
              -o $@ \
              $(maude_OBJECTS) \
              $(maude_LDADD) $(maude_LIBS) $(LIBS)

    For maudelt:
      $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link \
      $(CCLD) $(maudelt_CFLAGS) $(CFLAGS) \
              $(maudelt_LDFLAGS) $(LDFLAGS) \
              -o $@ \
              $(maudelt_OBJECTS) \
              $(maudelt_LDADD) $(maudelt_LIBS) $(LIBS)

    For librarylt.la:
      $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link \
      $(CCLD) $(librarylt_la_CFLAGS) $(CFLAGS) \
              $(librarylt_la_LDFLAGS) $(LDFLAGS) \
              -o $@ \
              -rpath $(libdir) \
              $(librarylt_la_OBJECTS) \
              $(librarylt_la_LIBADD) $(librarylt_la_LIBS) $(LIBS)

Note that solution A) would have the drawback of an inconsistency between
library_a_LIBADD and librarylt_la_LIBADD: library_a_LIBADD must not contain
link dependencies (see [6]).

Bruno


[1] http://www.gentoo.org/proj/en/qa/asneeded.xml
[2] http://lists.gnu.org/archive/html/bug-gnulib/2006-09/msg00146.html
[3] http://lists.gnu.org/archive/html/bug-gnulib/2006-09/msg00141.html
[4] http://www.gnu.org/software/automake/manual/html_node/Linking.html
[5] http://www.gnu.org/software/automake/manual/html_node/A-Library.html
[6] http://lists.gnu.org/archive/html/bug-gnulib/2006-09/msg00054.html

Message #8 received at 7621 <at> debbugs.gnu.org (full text, mbox):

From: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 7621 <at> debbugs.gnu.org
Subject: Re: bug#7621: add a way to pass link dependencies
Date: Sat, 22 Jan 2011 10:27:27 +0100

[ http://thread.gmane.org/gmane.comp.sysutils.automake.bugs/5122 ]

Hi Bruno,

first off, thanks for the very nice and detailed report, and sorry for
the delay.

* Bruno Haible wrote on Sun, Dec 12, 2010 at 03:43:00PM CET:
> When creating an executable or library, sometimes other libraries
> have to be mentioned on the link command line as dependencies.
[...]

> When creating library.a, one can ignore dependencies to libraries not in the
> build tree. So we need consider only maude, maudelt, and librarylt.la.
> Where could I stuff the linker options or libtool options?
> 
> - Can I put them in the *_LDFLAGS variable?
> 
>   No, it does not work, because if it's a static library (/somewhere/libiconv.a),
>   the linker will not pick any object files from it, because it precedes
>   the *_OBJECTS on the link command line. Ralf noticed that in [2].

Agreed.

> - Can I put them in the *_LDADD variable for programs or *_LIBADD variable
>   for libraries?

>   No, because the options contain other things than -l and -L options, and
>   these can't go in *_LDADD or *_LIBADD variables, says the Automake doc.
>   [3][4][5]

Here, I think we need to adjust both the documentation and the
implementation.  Even with portability as important aspect, autotools
should strive to support position-dependent linker flags better.
-Wl,-Bstatic and several others come to mind.  Of course supporting this
right will also require changes to libtool, but that is a next step.

>   I understand that the true reason for this requirement is that Automake
>   wants to compute a *_DEPENDENCIES variable from *_LDADD or *_LIBADD variables:

>     "If `PROG_DEPENDENCIES' is not supplied, it is computed by Automake.
[...]

Yes, I think that is the reason.  We can still do so however even with
more laxness, automake just has to skip the other flags.  And we are
limited in that at automake time, @substitutions@ cannot be analyzed
anyway.

>   Additionally, for libraries, the description of LIBADD is strict
>   about the fact that it "should be used to list extra libtool objects
>   (.lo files) or libtool libraries (.la) to add to library."
> 
> - Can I put them in LIBS?
> 
>   No, I explained above why I don't want this.

Agreed.

> Please offer a simple way to use these linker options.
> 
> I can see two choices:
> 
>  A) Either allow -Wl,-rpath or -rpath options also in *_LDADD variables,
>     and allow -Wl,-rpath, -rpath, -l, -L options also in *_LIBADD variables
>     of libraries built with libtool.
> 
>  B) Or introduce a new set of variables *_LIBS that can contain link options
>     for dependencies, like $(LIBICONV) and $(LTLIBICONV) above. Modify the
>     link commands to look like this:
> 
>     For maude:
>       $(CCLD) $(maude_CFLAGS) $(CFLAGS) \
>               $(maude_LDFLAGS) $(LDFLAGS) \
>               -o $@ \
>               $(maude_OBJECTS) \
>               $(maude_LDADD) $(maude_LIBS) $(LIBS)
> 
>     For maudelt:
>       $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link \
>       $(CCLD) $(maudelt_CFLAGS) $(CFLAGS) \
>               $(maudelt_LDFLAGS) $(LDFLAGS) \
>               -o $@ \
>               $(maudelt_OBJECTS) \
>               $(maudelt_LDADD) $(maudelt_LIBS) $(LIBS)
> 
>     For librarylt.la:
>       $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link \
>       $(CCLD) $(librarylt_la_CFLAGS) $(CFLAGS) \
>               $(librarylt_la_LDFLAGS) $(LDFLAGS) \
>               -o $@ \
>               -rpath $(libdir) \
>               $(librarylt_la_OBJECTS) \
>               $(librarylt_la_LIBADD) $(librarylt_la_LIBS) $(LIBS)
> 
> Note that solution A) would have the drawback of an inconsistency between
> library_a_LIBADD and librarylt_la_LIBADD: library_a_LIBADD must not contain
> link dependencies (see [6]).

Here's a third proposal: implement both (A) and (B), but in the latter
let $(prog_LIBS) override $(LIBS) instead of having both.  This would be
consistent in how prog_LDFLAGS and prog_LDADD work, and you could still
easily link with LIBS by using
  prog_LIBS = -lspecial $(LIBS)

This idea is listed in the TODO file, too.

Do you see any problem with the proposal?

Thanks,
Ralf

Message #11 received at 7621 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>
Cc: 7621 <at> debbugs.gnu.org
Subject: Re: bug#7621: add a way to pass link dependencies
Date: Sat, 22 Jan 2011 13:16:14 +0100

Hello Ralf,

Thank you for taking care of this long-standing problem.

> implement both (A) and (B), but in the latter
> let $(prog_LIBS) override $(LIBS) instead of having both.

For (A), the list of options to be ignored during dependency extraction is the
following (cf. gnulib/build-aux/config.rpath):
  -rpath /some/path
  -Wl,-rpath -Wl,/some/path
  -Wl,-rpath,/some/path
  -lopt=-rpath -lopt=/some/path
  -Qoption ld -rpath -Qoption ld /some/path
  -R/some/path
  -R /some/path
  -blibpath:/some/path
  -Wl,-blibpath:/some/path
  +b /some/path
  -Wl,+b -Wl,/some/path

A) Allow the above listed -rpath options also in *_LDADD variables,
   and allow the above listed -rpath options and -l, -L options also in
   *_LIBADD variables of libraries built with libtool.

B) Introduce a new set of variables *_LIBS that can contain link options
   for dependencies, like $(LIBICONV) and $(LTLIBICONV) above. Let them
   default to $(LIBS). Modify the link commands to look like this:

   For maude:
     $(CCLD) $(maude_CFLAGS) $(CFLAGS) \
             $(maude_LDFLAGS) $(LDFLAGS) \
             -o $@ \
             $(maude_OBJECTS) \
             $(maude_LDADD) $(maude_LIBS)

   For maudelt:
     $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link \
     $(CCLD) $(maudelt_CFLAGS) $(CFLAGS) \
             $(maudelt_LDFLAGS) $(LDFLAGS) \
             -o $@ \
             $(maudelt_OBJECTS) \
             $(maudelt_LDADD) $(maudelt_LIBS)

   For librarylt.la:
     $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link \
     $(CCLD) $(librarylt_la_CFLAGS) $(CFLAGS) \
             $(librarylt_la_LDFLAGS) $(LDFLAGS) \
             -o $@ \
             -rpath $(libdir) \
             $(librarylt_la_OBJECTS) \
             $(librarylt_la_LIBADD) $(librarylt_la_LIBS)

For the user, the benefit of using the *_LDADD and *_LIBADD variables will be
that he does not make a distinction between already installed libraries and
libraries in the build tree. But he has to be careful not to augment the
*_LIBADD variable of a .a library.

Whereas his benefit of using the *_LIBS variables will be that he can treat
programs, .a libraries, and .la libraries the same way. But he has to be
careful to handle dependencies to libraries in the build tree differently than
dependencies to already installed libraries.

The documentation could point this out.

But if you ask me, it is suboptimal to offer to the user two different ways of
doing the same thing, each with a different drawback. Ideally, there should be
one way of doing it, with none of the drawbacks.

So what I propose is

C) Introduce a new set of variables *_LIBS like in (B), and use them (in the
   case of a program or .la library) for the construction of the *_DEPENDENCIES
   variable.

   This computation of the *_DEPENDENCIES variable would then need to be
   enhanced to ignore all kinds of linker options and pick only the tokens
   that end in ".a" or ".la". This will be needed because people can write
   conditional code like

      if USING_GNU_LD
      maude_LIBS += -Wl,-E
      endif
      if ON_WIN32
      maude_LIBS += -Wl,--disable-auto-import
      endif
      if USE_EMBEDDED_LIBFOO
      maude_LIBS += ../foo/libfoo.a
      endif

   The documentation would make it clear that all link dependencies go in
   *_LIBS, for all kinds of programs and libraries.

   As a consequence, deprecate *_LDADD.

But wait, deprecating *_LDADD and letting *_LIBS take over their role is a
bit harsh. Why not keep the simplicity of proposal (C) without the deprecation?

D) Introduce a new set of variables *_LDADD for libraries that are built
   with libtool. Don't touch LIBS. Modify the link commands to look like this:

   For maude:
     $(CCLD) $(maude_CFLAGS) $(CFLAGS) \
             $(maude_LDFLAGS) $(LDFLAGS) \
             -o $@ \
             $(maude_OBJECTS) \
             $(maude_LDADD) $(LIBS)

   For maudelt:
     $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link \
     $(CCLD) $(maudelt_CFLAGS) $(CFLAGS) \
             $(maudelt_LDFLAGS) $(LDFLAGS) \
             -o $@ \
             $(maudelt_OBJECTS) \
             $(maudelt_LDADD) $(LIBS)

   For librarylt.la:
     $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link \
     $(CCLD) $(librarylt_la_CFLAGS) $(CFLAGS) \
             $(librarylt_la_LDFLAGS) $(LDFLAGS) \
             -o $@ \
             -rpath $(libdir) \
             $(librarylt_la_OBJECTS) \
             $(librarylt_la_LIBADD) $(librarylt_la_LDADD) $(LIBS)

   Compute the *_DEPENDENCIES not only from *_LDADD, but also from LIBS,
   ignoring all linker options like in proposal (C) above.

   The documentation would make it clear that all link dependencies go in
   *_LDADD and LIBS, for all kinds of programs and libraries, that LIBS
   is common to all programs and libraries being built by the same Makefile,
   and that the contents of *_LDADD and of LIBS are treated the same way.

   For libraries (both .la and .a libraries), the doc would continue to
   state that extra contents of the library goes in *_LIBADD whereas
   link dependencies go in *_LDADD (the latter being ignored for .a
   libraries, of course).

I think this proposal does not have any of the drawbacks, provides a
simple rule that the developer can easily remember, and is a straightforward
extension of what Automake offers now.

Bruno

Message #14 received at 7621 <at> debbugs.gnu.org (full text, mbox):

From: Jack Kelly <jack <at> jackkelly.name>
To: Bruno Haible <bruno <at> clisp.org>
Cc: Ralf Wildenhues <Ralf.Wildenhues <at> gmx.de>, 7621 <at> debbugs.gnu.org
Subject: Re: bug#7621: add a way to pass link dependencies
Date: Sun, 23 Jan 2011 07:52:10 +1100

On Sat, Jan 22, 2011 at 11:16 PM, Bruno Haible <bruno <at> clisp.org> wrote:
> For (A), the list of options to be ignored during dependency extraction is the
> following (cf. gnulib/build-aux/config.rpath):
>  -rpath /some/path
>  -Wl,-rpath -Wl,/some/path
>  -Wl,-rpath,/some/path
>  -lopt=-rpath -lopt=/some/path
>  -Qoption ld -rpath -Qoption ld /some/path
>  -R/some/path
>  -R /some/path
>  -blibpath:/some/path
>  -Wl,-blibpath:/some/path
>  +b /some/path
>  -Wl,+b -Wl,/some/path

Should there be an -Xlinker option in this list, too?

-- Jack

Message #19 received at 7621 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: 7621 <at> debbugs.gnu.org
Subject: Re: add a way to pass link dependencies
Date: Mon, 13 Feb 2023 13:13:06 +0100

To answer my own question:

In 1) I wrote:
> Typical values of LTLIBICONV are
>
>     -liconv
>     -rpath /usr/local/lib -L/usr/local/lib -liconv
>     /usr/local/lib/libiconv.a
>     -rpath /usr/local/lib /usr/local/lib/libiconv.so
>     /usr/local/lib/libiconv.la

But that is incorrect. /usr/local/lib/libiconv.a and /usr/local/lib/libiconv.la
cannot be among the results, because the Gnulib 'havelib' macros would put
"-L/usr/local/lib -liconv" into the LTLIBICONV variable instead.
See lib-link.m4:
LTLIB[]NAME="${LTLIB[]NAME}${LTLIB[]NAME:+ }-L$found_dir -l$name"

So, the value of LTLIBICONV consists of libtool options *only*. No file names.

I confirmed this, through some tests:
When libiconv is installed with --disable-shared, then (regardless
whether libiconv.la is deleted after installation):
  LTLIBICONV = -LPREFIX/lib -liconv
  LIBICONV = PREFIX/lib/libiconv.a

Then I asked:
> Now in which Automake provided variable could I stuff these linker options
> or libtool options?

The answer, as given by Ralf Wildenhues in
https://lists.gnu.org/archive/html/bug-gnulib/2006-09/msg00146.html :

  "this case is only a problem when Libtool is not used:
   libtool will reorder the command line so that `-l' flags appear after
   all encountered objects."

That is, we need to distinguish the case when libtool is used from the case
where libtool is not used.

* When libtool is used, we need to use $LTLIBICONV, not $LIBICONV. As documented
  in https://www.gnu.org/software/automake/manual/html_node/Libtool-Flags.html,
  these go into the library_LDFLAGS variable.
  In the libtool invocation, these *_LDFLAGS occur before the *.lo and *.la files,
  but this is not a problem since, as Ralf wrote,
    "libtool will reorder the command line so that `-l' flags appear after
     all encountered objects."

* When libtool is not used, i.e. the target is library.a, we just ignore both
  $LIBICONV and $LTLIBICONV. The documentation is at
  https://www.gnu.org/software/automake/manual/html_node/A-Library.html.

  When linking a program that uses library.a, $LIBICONV or $LTLIBICONV will
  need to be used, depending on whether this linking happens without or with
  libtool.

There is no need to unify these two cases (creating a .la library vs.
creating a .a library). It's just two different cases, two different Automake
rules.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.