Subject: 24.3.93; intermittent unexec failures when building on Mac OS X 10.10 beta, Xcode 6.0

From: David Caldwell <[email-masked]>
Date: Thu, 18 Sep 2014 21:13:41 -0700

Hello,

I tried to build the latest pretest on Mac OS X Yosemite Beta with the
new Xcode 6.0 (GM) tools and ran into this error during the unexec step:

unexec: not enough room for load commands for new __DATA segments

(Full unexec log is attached).

Poking around I discovered a couple discrepancies. The "headerpad_extra"
setting in configure.ac says entries are 56 bytes, but they are actually
78 bytes (probably because the comment was originally written in the 32
bit days). Also the number of load commands needed is variable now--it
depends on how many malloc segments were found and how many it could
merge (see find_emacs_zone_regions() and unexec_regions_merge() in
unexmacosx.c).

I printed num_unexec_regions in unexmacosx.c and observed anywhere
from about 14 to 34 regions needed. But given that it seems to be
different on every run and it's super annoying to debug when you run out
of space, I suggest bumping headerpad_extra up to 0x1000. This gives
about 1.5KB of headroom (or wasted space, depending on how you look at
it) and allows for 52 load commands, which should be future proof for
quite a while. I have attached a patch for configure.ac that does this
and changes the comment to reflect my understanding.

I've also provided a patch that makes the unexec error more helpful: it
computes the size needed for headerpad_extra given the number of load
commands that it needed for that particular unexec run.

I assume this affects the trunk code, too, though I didn't look at it.

-David
--- configure.ac.orig	2014-09-18 18:00:52.000000000 -0700
+++ configure.ac	2014-09-18 20:51:55.000000000 -0700
@@ -4856,17 +4856,19 @@
   darwin)
    ## The -headerpad option tells ld (see man page) to leave room at the
    ## end of the header for adding load commands.  Needed for dumping.
-   ## 0x690 is the total size of 30 segment load commands (at 56
-   ## each); under Cocoa 31 commands are required.
+   ## 0x1000 is enough for roughly 52 load commands on the x86_64
+   ## architecture (where they are 78 bytes each). The actual number of
+   ## load commands added is not consistent but normally ranges from
+   ## about 14 to about 34. Setting it high gets us plenty of slop and
+   ## only costs about 1.5K of wasted binary space.
+   headerpad_extra=1000
    if test "$HAVE_NS" = "yes"; then
      libs_nsgui="-framework AppKit"
      if test "$NS_IMPL_COCOA" = "yes"; then
         libs_nsgui="$libs_nsgui -framework IOKit"
      fi
-     headerpad_extra=6C8
    else
      libs_nsgui=
-     headerpad_extra=690
    fi
    LD_SWITCH_SYSTEM_TEMACS="-fno-pie -prebind $libs_nsgui -Xlinker -headerpad -Xlinker $headerpad_extra"
 
--- src/unexmacosx.c.orig	2014-09-18 20:53:21.000000000 -0700
+++ src/unexmacosx.c	2014-09-18 21:10:42.000000000 -0700
@@ -1302,7 +1306,9 @@
       }
 
   if (curr_header_offset > text_seg_lowest_offset)
-    unexec_error ("not enough room for load commands for new __DATA segments");
+    unexec_error ("not enough room for load commands for new __DATA segments"
+		  " (increase headerpad_extra in configure.in to at least %lX)",
+		  num_unexec_regions * sizeof (struct segment_command));
 
   printf ("%ld unused bytes follow Mach-O header\n",
 	  text_seg_lowest_offset - curr_header_offset);