GNU bug report logs - #64743
Speed up GNU make's internal processing

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Bruno Haible <bruno <at> clisp.org>
Subject: bug#64743: closed (Re: bug#64743: Speed up GNU make's internal
 processing)
Date: Fri, 21 Jul 2023 01:05:02 +0000

[Message part 1 (text/plain, inline)]

Your bug report

#64743: Speed up GNU make's internal processing

which was filed against the automake package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to 64743 <at> debbugs.gnu.org.

-- 
64743: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=64743
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Karl Berry <karl <at> freefriends.org>
To: bruno <at> clisp.org
Cc: 64743 <at> debbugs.gnu.org, psmith <at> gnu.org
Subject: Re: bug#64743: Speed up GNU make's internal processing
Date: Thu, 20 Jul 2023 19:03:52 -0600

Hi Bruno,

    GNU Automake already emits the '.SUFFIXES:' line. To optimize things
    for GNU make, it should also emit the remaining part.

Ok. I just hope those weird-looking %:: rules do not cause trouble with
other makes. I guess we'll find out.

The fnoc test also failed due to the new rules. I fixed that, added a
few words of documentation and NEWS, and pushed it.

Thanks Bruno. -k


2023-07-20  Bruno Haible  <bruno <at> clisp.org>

automake: disable GNU make's internal pattern rules, for speed.

From https://bugs.gnu.org/64743.

* lib/am/footer.am: Disable GNU make's internal pattern rules.
* lib/Automake/Rule.pm (_conditionals_for_rule): Add special handling
for these pattern rules from footer.am.
(define): Likewise.
* t/nodep.sh: Update test to avoid matching the new %:: rules.
* t/fnoc.sh: Update test to avoid matching the new %:: SCCS rule.

* NEWS: Mention this.
* doc/automake.texi (Suffixes): Likewise.
(Doc changes written by Karl.)
diff --git a/NEWS b/NEWS
index c64d4ef15..ced0fd640 100644
--- a/NEWS
+++ b/NEWS
@@ -21,6 +21,9 @@ New in 1.17:
   - Variables using escaped \# will trigger portability warnings, but be
     retained when appended.  GNU Make & BSD Makes are known to support it.

+  - GNU Make's default pattern rules are disabled, for speed and debugging.
+    (.SUFFIXES was already cleared.) (bug#64743)
+
   - For Texinfo documents, if a .texi.in file exists, but no .texi, the
     .texi.in will be read. Texinfo source files need not be present at
     all, and if present, need not contain @setfilename. Then the file name
diff --git a/doc/automake.texi b/doc/automake.texi
index 4561d1948..3b4f06e73 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -10977,6 +10977,14 @@ Automake generate the suffix list for @code{.SUFFIXES}.  Any given
 @code{SUFFIXES} go at the start of the generated suffixes list, followed
 by Automake generated suffixes not already in the list.

+@c https://bugs.gnu.org/64743.
+@cindex built-in suffix and pattern rules, disabled
+@cindex suffix rules of Make, disabled
+@cindex pattern rules of GNU Make, disabled
+Automake disables the Make program's built-in rules with a
+@code{.SUFFIXES:} rule, and then adds whatever suffixes are
+necessary.  Automake also disables GNU Make's built-in pattern rules.
+
 @node Include
 @chapter Include

diff --git a/lib/Automake/Rule.pm b/lib/Automake/Rule.pm
index 9f72d2728..52ee3be4a 100644
--- a/lib/Automake/Rule.pm
+++ b/lib/Automake/Rule.pm
@@ -695,6 +695,9 @@ sub _conditionals_for_rule ($$$$)

   return $cond if !$message; # No ambiguity.

+  # Don't coalesce the several pattern rules from footer.am into a single one.
+  return $cond if $target eq "%:" && $where->get =~ /\/am\/footer\.am$/;
+
   if ($owner == RULE_USER)
     {
       # For user rules, just diagnose the ambiguity.
@@ -764,23 +767,27 @@ sub define ($$$$$)

   my $tdef = _rule_defn_with_exeext_awareness ($target, $cond, $where);

-  # A GNU make-style pattern rule has a single "%" in the target name.
-  msg ('portability', $where,
-       "'%'-style pattern rules are a GNU make extension")
-    if $target =~ /^[^%]*%[^%]*$/;
-
-  # See whether this is a duplicated target declaration.
-  if ($tdef)
+  # The pattern rules in footer.am look like duplicates, but really aren't.
+  if ($source !~ /\/am\/footer\.am$/)
     {
-      # Diagnose invalid target redefinitions, if any.  Note that some
-      # target redefinitions are valid (e.g., for multiple-targets
-      # pattern rules).
-      _maybe_warn_about_duplicated_target ($target, $tdef, $source,
-                                           $owner, $cond, $where);
-      # Return so we don't redefine the rule in our tables, don't check
-      # for ambiguous condition, etc.  The rule will be output anyway
-      # because '&read_am_file' ignores the return code.
-      return ();
+      # A GNU make-style pattern rule has a single "%" in the target name.
+      msg ('portability', $where,
+           "'%'-style pattern rules are a GNU make extension")
+        if $target =~ /^[^%]*%[^%]*$/;
+
+      # See whether this is a duplicated target declaration.
+      if ($tdef)
+        {
+          # Diagnose invalid target redefinitions, if any.  Note that some
+          # target redefinitions are valid (e.g., for multiple-targets
+          # pattern rules).
+          _maybe_warn_about_duplicated_target ($target, $tdef, $source,
+                                               $owner, $cond, $where);
+          # Return so we don't redefine the rule in our tables, don't check
+          # for ambiguous condition, etc.  The rule will be output anyway
+          # because '&read_am_file' ignores the return code.
+          return ();
+        }
     }

   my $rule = _crule $target;
diff --git a/lib/am/footer.am b/lib/am/footer.am
index 9715c826c..388defb14 100644
--- a/lib/am/footer.am
+++ b/lib/am/footer.am
@@ -17,3 +17,14 @@
 # Tell versions [3.59,3.63) of GNU make to not export all variables.
 # Otherwise a system limit (for SysV at least) may be exceeded.
 .NOEXPORT:
+
+# Tell GNU make to disable its built-in pattern rules.
+## This reduces make's internal processing.
+## Recommended by Paul Smith in
+## <https://lists.gnu.org/archive/html/bug-make/2023-07/msg00063.html>.
+## These rules have no effect on non-GNU make implementations.
+%:: %,v
+%:: RCS/%,v
+%:: RCS/%
+%:: s.%
+%:: SCCS/s.%
diff --git a/t/fnoc.sh b/t/fnoc.sh
index 97b694c63..7d6b49096 100644
--- a/t/fnoc.sh
+++ b/t/fnoc.sh
@@ -35,5 +35,10 @@ END
 $ACLOCAL
 $AUTOMAKE

-grep '[^F]CC' Makefile.in | grep -v MKDEP && exit 1
+# In bug#64743, explicit rules were added to disable make's default
+# built-in rules, including one for SCCS:
+# %:: SCCS/s.%
+# So don't match that. Meanwhile, MKDEP does not occur in the output
+# any more, but leaving it in in case it comes back.
+grep '[^F]CC' Makefile.in | grep -v MKDEP | grep -v SCCS && exit 1
 exit 0
diff --git a/t/nodep.sh b/t/nodep.sh
index 4201c8ead..0174b838e 100644
--- a/t/nodep.sh
+++ b/t/nodep.sh
@@ -37,6 +37,6 @@ $ACLOCAL
 $AUTOMAKE

 sed 's/printf .*%s//' Makefile.in > Makefile.tmp
-grep '%' Makefile.tmp && exit 1
+grep -v '^%::' Makefile.tmp | grep '%' && exit 1

 :

compile finished at Thu Jul 20 18:00:35 2023

[Message part 3 (message/rfc822, inline)]

From: Bruno Haible <bruno <at> clisp.org>
To: bug-automake <at> gnu.org
Cc: Paul Smith <psmith <at> gnu.org>
Subject: Speed up GNU make's internal processing
Date: Thu, 20 Jul 2023 16:07:53 +0200

[Message part 4 (text/plain, inline)]

Hi,

Paul Smith, the GNU make maintainer, gives this recommendation how
to disable built-in rules and thus speed up make's processing [1]:

 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
My recommendation has been to disable the built-in rules directly, if
you don't need them.  For example you can use:

  .SUFFIXES:

to disable most of the built in rules (this is a POSIX standard
facility so it's helpful even for other versions of make).

Unfortunately that doesn't fix all problems because GNU Make also has a
few built-in rules that are defined using pattern rules (because suffix
rules are not powerful enough).  So a full list of "turn it all off"
would be this:

  .SUFFIXES:
  %:: %,v
  %:: RCS/%,v
  %:: RCS/%
  %:: s.%
  %:: SCCS/s.%
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

GNU Automake already emits the '.SUFFIXES:' line. To optimize things
for GNU make, it should also emit the remaining part.

It has no effect with non-GNU make [2].

The effect, for example in gettext's gettext-runtime/src directory, is
that
  - The output of "make -n -d" shrinks from 11028 lines to 4928 lines.
  - The number of 'stat()' calls made by "make -n" shrinks from 188 to 178.
    (make no longer tests whether various directories have an 'RCS' or 'SCCS'
    subdirectory.)

Patch is attached.

Bruno

[1] https://lists.gnu.org/archive/html/bug-make/2023-07/msg00063.html
[2] https://lists.gnu.org/archive/html/bug-make/2023-07/msg00067.html

[0001-Speed-up-GNU-make-s-internal-processing.patch (text/x-patch, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.