GNU bug report logs - #38043
Incorrect Python byte-compiling for Python 3.5+ and PyPy3

Previous Next

Full log

Message #38 received at 38043 <at> debbugs.gnu.org (full text, mbox):

From: Mike Frysinger <vapier <at> gentoo.org>
To: 38043 <at> debbugs.gnu.org
Subject: [PATCH 6/6] py-compile: fix optimized compiling for Python 3.5+
Date: Sun,  6 Feb 2022 01:57:14 -0500

Fixes automake bug https://bugs.gnu.org/38043.

Split the optimized compilation logic into a new section.  This avoids
trying to support multiple versions of major versions in a single script
as it gets harder to verify new changes don't break old versions as time
goes on.

Now for Python 3.5+, compile with -O2.

* THANKS: Add Michal Górny.
* lib/py-compile: Add new section for compiling Python 3.5+.
---
 THANKS         |  3 ++-
 lib/py-compile | 57 +++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 58 insertions(+), 2 deletions(-)

diff --git a/THANKS b/THANKS
index 5a11cf4d25d9..554013ca87c3 100644
--- a/THANKS
+++ b/THANKS
@@ -244,7 +244,7 @@ Leonardo Boiko                  leoboiko <at> conectiva.com.br
 Libor Bukata                    libor.bukata <at> oracle.com
 Loulou Pouchet                  loulou <at> lrde.epita.fr
 Ludovic Courtès                 ludo <at> gnu.org
-Lukas Fleischer                        lfleischer <at> lfos.de
+Lukas Fleischer                 lfleischer <at> lfos.de
 Luo Yi                          luoyi.ly <at> gmail.com
 Maciej Stachowiak               mstachow <at> mit.edu
 Maciej W. Rozycki               macro <at> ds2.pg.gda.pl
@@ -285,6 +285,7 @@ Michael Daniels                 mdaniels <at> rim.com
 Michael Hofmann                 mhofma <at> googlemail.com
 Michael Ploujnikov              ploujj <at> gmail.com
 Michael Zucchi                  notzed <at> gmail.com
+Michał Górny                    mgorny <at> gentoo.org
 Michel de Ruiter                mdruiter <at> cs.vu.nl
 Mike Castle                     dalgoda <at> ix.netcom.com
 Mike Frysinger                  vapier <at> gentoo.org
diff --git a/lib/py-compile b/lib/py-compile
index f72d4945da96..d34bb2dd2364 100755
--- a/lib/py-compile
+++ b/lib/py-compile
@@ -131,6 +131,13 @@ case $python_major in
   ;;
 esac
 
+python_minor=`$PYTHON -c 'import sys; print(sys.version_info[1])'`
+
+# NB: When adding support for newer versions, prefer copying & adding new cases
+# rather than try to keep things merged with shell variables.
+
+# First byte compile (no optimization) all the modules.
+# This works for all currently known Python versions.
 $PYTHON -c "
 import sys, os, py_compile, importlib
 
@@ -149,7 +156,10 @@ for file in sys.argv[1:]:
         py_compile.compile(filepath, filepath + 'c', path)
 sys.stdout.write('\n')" "$@" || exit $?
 
-$PYTHON -O -c "
+# Then byte compile w/optimization all the modules.
+case $python_major.$python_minor in
+2.*)
+  $PYTHON -O -c "
 import sys, os, py_compile, importlib
 
 # pypy does not use .pyo optimization
@@ -170,6 +180,51 @@ for file in sys.argv[1:]:
     else:
         py_compile.compile(filepath, filepath + 'o', path)
 sys.stdout.write('\n')" "$@" 2>/dev/null || exit $?
+  ;;
+*)  # Python 3+
+  $PYTHON -O -c "
+import sys, os, py_compile, importlib
+
+print('Byte-compiling python modules (optimized versions) ...')
+for file in sys.argv[1:]:
+    $pathtrans
+    $filetrans
+    if not os.path.exists(filepath) or not (len(filepath) >= 3
+                                            and filepath[-3:] == '.py'):
+	    continue
+    print(file, end=' ', flush=True)
+    if hasattr(sys.implementation, 'cache_tag'):
+        py_compile.compile(filepath, importlib.util.cache_from_source(filepath), path)
+    else:
+        py_compile.compile(filepath, filepath + 'o', path)
+print()" "$@" 2>/dev/null || exit $?
+  ;;
+esac
+
+# Then byte compile w/more optimization.
+case $python_major.$python_minor in
+2.*|3.[0-4])
+  ;;
+*)  # Python 3.5+
+  # See https://bugs.gnu.org/38043 for background.
+  $PYTHON -OO -c "
+import sys, os, py_compile, imp
+
+print('Byte-compiling python modules (-OO version) ...')
+for file in sys.argv[1:]:
+    $pathtrans
+    $filetrans
+    if not os.path.exists(filepath) or not (len(filepath) >= 3
+                                            and filepath[-3:] == '.py'):
+        continue
+    print(file, end=' ', flush=True)
+    if hasattr(imp, 'get_tag'):
+        py_compile.compile(filepath, imp.cache_from_source(filepath), path)
+    else:
+        py_compile.compile(filepath, filepath + 'o', path)
+print()" "$@" 2>/dev/null || exit $?
+  ;;
+esac
 
 # Local Variables:
 # mode: shell-script
-- 
2.34.1

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.