Subject: Solaris, lost mapfiles and silent failures

From: Jeffrey Walton <[email-masked]>
Date: Thu, 9 Nov 2017 00:19:49 -0500

I've been tracking an Autotools issue on Solaris 11.3 x86_64 (fully
patched). The project is a C++ library, and it uses a mapfile to set
capabilities on executables and shared objects.

The mapfile specifies SSE and SSE2 capabilities. The mapfile is added
with the linker option '-M mylib.mapfile'. There are no long forms of
the command. Confer,
https://docs.oracle.com/cd/E26502_01/html/E29030/ld-1.html.

When the Autotools project builds the library and test program,
libtool silently drops the mapfile when linking the shared object. It
just vanishes into the ether. The mapfile is present when building
test program. Its only missing when building the shared object.

Subsequently, trying to run the test program results in:

   $ ./cryptestcwd v
   ld.so.1: cryptestcwd: fatal:
   /export/home/cryptopp/.libs/libcryptopp.so.6: hardware capability
   (CA_SUNW_HW_1) unsupported: 0x4800000  [ AES SSE4.1 ]
   Killed

The failure is silent. libtool does not produce a warning or error
message stating its removing the option. The documentation does not
discuss libtool's behavior on Solaris. There's a warning for -M
options on Cygwin, and I believe that is the extent of it. Solaris is
certainly not Cygwin. Confer,
https://www.gnu.org/software/libtool/manual/libtool.html.

We spent the better part of a day trying to trouble shoot the issue.
Our attention was focused on Automake, and trying to untangle things
that result in missing AM_LDFLAGS options when linking the shared
object. The lack of a message probably wasted 6 to 10 man hours.

Please stop removing important options needed to link programs and
shared objects. The  options were specified for a reason and they are
necessary. If libtool cannot stop removing necessary options, then we
need (1) a clear message stating what is happening; (2) documentation
on the behavior; and (3) a way to override the default behavior.

An example of working around the problem would be nice since Autotools
does not appear to have a post-build or post-link hook. We cut-in an
awful hack, but it probably runs afoul of Autotools or libtool.
Confer, https://github.com/noloader/cryptopp-autotools/commit/61828068c6ab.

Thanks in advance.