GNU bug report logs - #44462
Problem with get_multilibs on macOS

Previous Next

Full log

Message #23 received at 44462 <at> debbugs.gnu.org (full text, mbox):

From: Jacob Bachmeyer <jcb62281 <at> gmail.com>
To: Fred Wright <fw <at> fwright.net>
Cc: 44462 <at> debbugs.gnu.org
Subject: Re: bug#44462: Problem with get_multilibs on macOS
Date: Tue, 10 Nov 2020 18:48:54 -0600

Fred Wright wrote:
> On Sun, 8 Nov 2020, Jacob Bachmeyer wrote:
>> Fred Wright wrote:
>>> On Thu, 5 Nov 2020, Jacob Bachmeyer wrote:
>>>> Fred Wright wrote:
>>>>
>>>>> I traced the results of get_multilibs (as used by the libffi 
>>>>> tests) in many macOS versions, and even the "successful" cases 
>>>>> seem to have questionably useful results:
>>>>>
>>>>> [...]
>>>>>
>>>>> The "/usr/." result returned on 10.4, 10.5, and 32-bit 10.6 is the 
>>>>> same as what I see on Ubuntu 14.04, CentOS 7, and Fedora 25, 
>>>>> though it's not clear to me what it's supposed to represent.
>>>>
>>>> I have a suspicion that this feature is designed to support testing 
>>>> with an "experimental" compiler build that is not installed on the 
>>>> system and may be useless with system compilers generally, or with 
>>>> Apple's compilers specifically, if Apple does not use multilib.
>>>
>>> Apple supports the concept of multi-architecture binaries, but not 
>>> in the same way that multilib does it (AFAIK).  Macs can have 
>>> "universal" binaries, which are archives combining multiple 
>>> per-architecture slices. This is applicable to object files, shared 
>>> libraries, and executables.  If
>>> the build setup allows it, it can be as simple as including multiple 
>>> architecture options in the compile command.  E.g.:
>>>
>>>     cc -arch x86_64 -arch i386 -o hello hello.c
>>>
>>> Under the hood, the compiler driver runs a separate compile/assemble 
>>> for each architecture, and then combines the object files.  The 
>>> linker supports universal binaries directly.
>>>
>>> With this arrangement, architecture-related conditionals in the 
>>> source code work just fine, but what *doesn't* work is having 
>>> architecture-related parameters in a configure script, which is 
>>> unfortunately not as uncommon as it ought to be.
>>
>> No, Autoconf pushes programs to respond to features instead of using 
>> known architectures.  This approach has been very successful:  as I 
>> understand, most programs using Autoconf needed no changes at all to 
>> port to RISC-V even if they were written long before RISC-V existed.  
>> Many programs, if they had previously been ported to any 64-bit 
>> architecture, needed no changes at all for x86_64.  Autoconf has 
>> achieved something architecture #ifdefs cannot do: provide automatic 
>> portability to a new architecture that did not exist when the program 
>> was written.  I consider Autoconf's approach here completely vindicated.
>
> I'm not talking about checking for specific architectures; this is 
> about checking architecture-related *properties*.  For example, a 
> configure check for sizeof(long) is incompatible with 
> multi-architecture builds, while using LONG_BIT works just fine.  But 
> there are build procedures that do things like the former, for no good 
> reason.

As far as I can tell, LONG_BIT as a preprocessor symbol is relatively 
new; newer than many programs that have demonstrated this level of 
Autoconf-assisted portability.  On older systems, glibc provides 
CHAR_BIT and you can find the bit-length with CHAR_BIT*sizeof(long).

>>>>> Since get_multilibs already has code to return an empty string in 
>>>>> the "remote" case (where it assumes this function won't work), I 
>>>>> just added code to unix.exp to set multitop to "" for all "darwin" 
>>>>> targets, thereby short-circuiting almost all og get_multilibs.  
>>>>> That certainly fixes the problem with the libffi tests, and 
>>>>> doesn't change any non-Mac behavior, though I don't know if that's 
>>>>> the ideal fix.  The whole get_mutilibs function looks pretty ugly 
>>>>> anyway, and it's generally recognized that relying on -dumpspecs 
>>>>> is a bad idea.
>>>>
>>>> It is most certainly not ideal.  A better solution is probably to 
>>>> add a test to get_multilibs to return an empty string if the 
>>>> compiler is not GCC.  Of course, if another compiler pretends to be 
>>>> GCC enough to pass that check, but does not actually implement 
>>>> -dumpspecs, that is not our bug.
>>>
>>> [...]
>>> The libffi test suite comes up with a "compiler_vendor" variable 
>>> which seems to be able to distinguish clang from gcc, though I 
>>> haven't looked at the details.
>>>
>>> Fixing get_multilibs properly would probably mean making it both 
>>> highly platform-specific and optional.
>>
>> The get_multilibs procedure is *not* platform-specific; it is 
>> GCC-specific. I am still unsure how exactly it is used.
>
> Since it seems to involve file paths, it may be specific to 
> combinations of GCC and platforms.
>
> But since neither of us seems to know very much about what 
> get_multilibs is trying to do, it's hard to discuss it intelligently. :-)

This is the main reason that get_multilibs failing on MacOS X will be a 
known bug in 1.6.3.

>>>> This issue on Mac OS X will probably be a known bug in 1.6.3 and 
>>>> fixed in 1.6.4.
>>>>
>>>>> I primarily tested my patch against the 1.6.2 release, since the 
>>>>> current master won't install from a non-git directory, and also 
>>>>> has multiple failures in its own tests (even on Linux).  The patch 
>>>>> is nearly identical between the 1.6.2 and master cases, anyway.
>>>>
>>>> Are we looking at the same current master?  I have commit 
>>>> 3d62df24deedfb3c7c3e396a31b8ce431138eb49 here and all of the tests 
>>>> pass.
>>>>
>>>> ****These other problems are potential release blockers for 1.6.3.****
>>>> Can you file another bug report with the test failures and more 
>>>> information about these issues?
>>>
>>> I looked into this more closely and it's probably related to the 
>>> non-git issue.  When running from a non-git directory, the configure 
>>> script reports a "fatal" error, but then goes on to complete with a 
>>> zero exit status and a more or less buildable setup, so you have to 
>>> be paying close attention to the output to notice.
>
> Actually it now looks like the two things are unrelated; I filed two 
> separate bugs.

Thank you; PR44544 should now be fixed on master; I have responded 
accordingly at that report also.  I am still looking into PR44545.

>>>>> I can send the current patch, either as a bare email or as an 
>>>>> attachment. AFAIK, Savannah doesn't have the pull request / merge 
>>>>> request concept.
>>>>
>>>> This will need to be fixed in libgloss.exp, not unix.exp.  I am 
>>>> putting my foot down on fixing bugs in DejaGnu's own tree directly 
>>>> instead of hacking around them like that.
>>>
>>> Well, OK, but there seem to be other similar hacks in unix.exp, and 
>>> if the idea is that get_multilibs is completely useless on the Mac 
>>> (which appears to be the case with the current implemenation, 
>>> anyway), then disabling it in the target-related code doesn't seem 
>>> unreasonable.
>>
>> It is not completely useless even on MacOS X -- a user could install 
>> GCC and expect a testsuite to use it, particularly in the case of a 
>> cross-compiler for embedded development.
>
> If it's a cross compiler, then by definition it can't run the 
> resulting code on the host platform.  But since get_multilibs already 
> excludes all remote cases, it wouldn't be able to run it on a separate 
> target platform, either.
>
> Besides, aren't files like baseboards/unix.exp based on the *target* 
> platform, not the host?  If so, then it seems like disabling 
> get_multilibs for the Mac there is exactly the right thing, at least 
> until such time as get_multilibs can behave usefully for a Mac target.

Untangling cross-development configurations is an ongoing project.

GCC could (in theory) be installed on any platform.  A user may build a 
native GCC for MacOS X and want to test or use it.  In short, the 
solution needs to test the actual issue -- is $compiler GCC? -- instead 
of assuming that it is or is not based on the platform.


-- Jacob

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.