Subject: libtool mishandles some compiler flags when used with MSVC tools
From: Kirill Makurin <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Sat, 2 Nov 2024 07:17:12 +0000
List-Id: <debbugs-submit.debbugs.gnu.org>

Hello,

I have encountered a few bugs with libtool when it is used with MSVC tools.

I have written a simple dummy project which should demonstrate the bugs. I uploaded the archive to Google Drive: https://drive.google.com/drive/folders/1DPJTid8PG6JjR7MHKKoz_i2QQTRGh8EO

The `configure`, `Makefile.in` etc. were generated from Msys2 environment with Autoconf 2.72, Automake 1.17 and libtool 2.5.3.

When built with MSVC tools, it will 1) produce a warning from the compiler (cl.exe) about unrecognized option and 2) fail with error from the linker (link.exe) about missing input file.

I also attached two files containing output from make for comparison. One when used with MSVC tools and another when used with GNU tools.

The encountered bugs are as follows:

1. When libtool is used with MSVC tools, libtool's link step mishandles compiler flags like`-Wl` and `-Xlinker`. For example, `-Wl,-def:FILENAME` becomes plain `-def:FILENAME` when libtool invokes the compiler*. (as seen in make-msvc.txt)

2. When producing a DLL, libtool passes `-Fe FILENAME` directly to the compiler*. The space between `-Fe` and `FILENAME` prevents cl.exe (and clang-cl.exe) from treating FILENAME as an output file, but rather as an input file.

* the compiler in question is `path/to/compile cl.exe` which should handle the usual `-o FILENAME` and flags like `-Wl` correctly with cl.exe

The second bug does not appear when libtool creates executables. It also seems to appear only in recent versions of libtool, since it does not appear in existing projects which used older versions of libtool. In older versions, libtool passes `-o FILENAME` which is handled by `compile` wrapper.

- Kirill Makurin