GNU bug report logs - #25024
build: adjust for ASAN/UBSAN

Previous Next

To reply to this bug, email your comments to 25024 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Marcel Böhme <boehme.marcel <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: Bug in Sort
Date: Fri, 25 Nov 2016 14:18:01 +0800

Dear all,

The following execution is flagged by UBSAN as undefined behaviour:

$ echo 0 > a; printf "%0.s0" {1..58} >> a
$ ./sort -R a

UBSAN says:
../lib/md5.c:371:7: runtime error: load of misaligned address 0x7ffdfd45a10d for type 'const uint32_t', which requires 4 byte alignment 

So, the root cause might actually be in GNULIB. The bug was found with AFLFast, a fork of AFL.

Best regards,
- Marcel

Message #8 received at 25024 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Marcel Böhme <boehme.marcel <at> gmail.com>,
 25024 <at> debbugs.gnu.org
Subject: Re: bug#25024: Bug in Sort
Date: Fri, 25 Nov 2016 13:06:47 +0000

On 25/11/16 06:18, Marcel Böhme wrote:
> Dear all,
> 
> The following execution is flagged by UBSAN as undefined behaviour:
> 
> $ echo 0 > a; printf "%0.s0" {1..58} >> a
> $ ./sort -R a
> 
> UBSAN says:
> ../lib/md5.c:371:7: runtime error: load of misaligned address 0x7ffdfd45a10d for type 'const uint32_t', which requires 4 byte alignment 
> 
> So, the root cause might actually be in GNULIB. The bug was found with AFLFast, a fork of AFL.

There is some support at least for catering
for architectures where unaligned access is significant.
So for UBSAN we should probably build with
_STRING_ARCH_unaligned defined globally
to avoid warning for the cases we already handle.

cheers,
Pádraig

Message #11 received at 25024 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>,
 Marcel Böhme <boehme.marcel <at> gmail.com>,
 25024 <at> debbugs.gnu.org
Subject: Re: bug#25024: Bug in Sort
Date: Fri, 25 Nov 2016 10:50:06 -0800

Pádraig Brady wrote:
> for UBSAN we should probably build with
> _STRING_ARCH_unaligned defined globally
> to avoid warning for the cases we already handle.

Yes. Translating this for non-experts: the problem here is a bug in the 
bug-finding procedure, not a bug in GNU coreutils or in Gnulib.

Recent glibc (since 2016-02-18) does not define _STRING_ARCH_unaligned, which 
means that this code in gnulib md5.c etc. is no longer exercised on recent 
platforms. So in some sense the originally-reported bug is already fixed (via an 
unexpected glibc change), though this does mean Gnulib md5 etc. is now slower on 
x86-64 etc., which is a performance bug on newer platforms. If we fix the 
performance bug I suppose we'll start getting false alarms from UBSAN again.

Message #14 received at 25024 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Marcel Böhme
 <boehme.marcel <at> gmail.com>, 25024 <at> debbugs.gnu.org
Subject: Re: bug#25024: Bug in Sort
Date: Fri, 25 Nov 2016 22:25:26 +0000

[Message part 1 (text/plain, inline)]

On 25/11/16 18:50, Paul Eggert wrote:
> Pádraig Brady wrote:
>> for UBSAN we should probably build with
>> _STRING_ARCH_unaligned defined globally
>> to avoid warning for the cases we already handle.
> 
> Yes. Translating this for non-experts: the problem here is a bug in the 
> bug-finding procedure, not a bug in GNU coreutils or in Gnulib.

Sorry I was a bit terse. coreutils/gnulib should currently be compiled with
  -D_STRING_ARCH_unaligned=0 -D_STRING_INLINE_unaligned=0
when using UBSAN, to use only alignment portable code.
Methods for avoiding false UBSAN warnings automatically are discussed below...

> Recent glibc (since 2016-02-18) does not define _STRING_ARCH_unaligned, which 
> means that this code in gnulib md5.c etc. is no longer exercised on recent 
> platforms.

Oh interesting. I see details in:
https://sourceware.org/bugzilla/show_bug.cgi?id=19462
There it suggests that _STRING_ARCH_unaligned is now internal
to glibc and _STRING_INLINE_unaligned is the newer stable equivalent.
Attached patch to do this for coreutils is attached.

> So in some sense the originally-reported bug is already fixed (via an 
> unexpected glibc change), though this does mean Gnulib md5 etc. is now slower on 
> x86-64 etc., which is a performance bug on newer platforms. If we fix the 
> performance bug I suppose we'll start getting false alarms from UBSAN again.

We can explicitly avoid the UBSAN warnings with something like:
http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=v8.23-80-g5760532
which might be acceptable given the few places it matters.
That's a bit of a big hammer though, defining away all of UBSAN for those routines.

Alternatively we might define the non-portable faster path away,
if we could detect we where compiling in UBSAN mode.
That's easy enough for -fsanitize=address, though it doesn't
look like there is currently a way to detect -fsanitize=undefined?
http://stackoverflow.com/q/39371798/4421

Another approach would be to support ../configure --with-asan --with-ubsan
which would define things appropriately.

cheers,
Pádraig.

[coreutils-unaligned.patch (text/x-patch, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.