Subject: AM_PROG_AR and LT_INIT don't work together when using lib.exe

From: Tim Ruffing <[email-masked]>
Date: Wed, 16 Mar 2022 15:24:37 +0100

Hello :)

Assume environment variable AR="lib.exe" and configure.ac has

```
AM_PROG_AR
LT_INIT
```

Then AM_PROG_AR checks for an archiver using AC_CHECK_TOOLS. Then it
detects the lib.exe interface and prefixes the AR variable with a call
to ar-lib. (There's a FIXME in the code that says that overwriting is
wrong). But so far so good, this works.

Now LT_INIT will again look for an archiver using AC_CHECK_TOOLS (in
subroutine _LT_PROG_AR). The check will discover that ac_cv_prog_AR has
"lib.exe" (unprefixed!) and overwrites AR with it.

So AR is correct (with ar-lib prefix) after AM_PROG_AR but LT_INIT
overwrites with the unprefixed value again, causing the build the fail.
I wonder why noone has reported this so far. I tend to think that
lib.exe support is only really useful in combination with libtool but
this then triggers the bug.

Possible workarounds include:
 * Calling AM_PROG_AR after LT_INIT. But this will raise a warning in
   autoreconf telling me that I should call AM_PROG_AR before LT_INIT.
 * Clearing the cache value. This is what I've done for now in our
   project [1]. Note that here I'm careful and clear two cache
   variables but in in the end it's enough to clear ac_cv_prog_AR.

The aforementioned FIXME mentions introducing am_AR as a cleaner way.
This sounds like it fix the problem but maybe I'm overlooking
something. A crude hack is to overwrite not only AR but also
ac_cv_prog_AR.

Best,
Tim

[1] https://github.com/bitcoin-core/secp256k1/pull/1084/commits/e6b8a1ca47c07b798cf45ea85e257546c4dbafc4