Subject: auto-compile noise can't be avoided by script

From: Zefram <[email-masked]>
Date: Sun, 5 Jan 2014 23:41:06 +0000

Guile 2.0.9 has a facility to automatically cache a compiled version
of any Scheme source file that it loads, and it wants the world to
know about it!  If auto-compilation is enabled, which it is by default,
then when guile loads a file (that was not already compiled) it emits a
banner describing the auto-compilation.  This interferes with the proper
functionality of any program written as a guile script, by producing
output that the program did not intend.  Working around this is tricky
(discussed below).  There's no straightforward way for a script to avoid
the noise while being portable between guile versions 1.8 and 2.0.
There's also no way to avoid the noise while actually getting the
auto-compilation behaviour.

In my particular case, my script makes interesting use of the
read-eval (#.) feature, which means that the compilation process
actually can't work.  This means that *every* time the script is run,
not just the first time, guile emits the banner about auto-compilation,
followed by a rather misleading warning/error about compilation failure.
It's misleading because it then goes on to execute the script just fine.
I can demonstrate this with a minimal test case (using read-eval in an
uninteresting way, just making the compiler barf by not having applied
eval-when to enable it):

$ cat t0
#!/usr/bin/guile -s
!#
(fluid-set! read-eval? #t)
(display #."hello world")
(newline)
$ guile-1.8 -s t0
hello world
$ guile-2.0 -s t0
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/zefram/usr/guile/t0
;;; WARNING: compilation of /home/zefram/usr/guile/t0 failed:
;;; ERROR: #. read expansion found and read-eval? is #f.
hello world
$

I can turn off the auto-compilation from within the script by using the
--no-auto-compile option, but that breaks compatibility to 1.8:

$ cat t1
#!/usr/bin/guile \
--no-auto-compile -s
!#
(fluid-set! read-eval? #t)
(display #."hello world")
(newline)
$ guile-2.0 '\' t1
hello world
$ guile-1.8 '\' t1
guile-1.8: Unrecognized switch `--no-auto-compile'
Usage: guile-1.8 OPTION ...
Evaluate Scheme code, interactively or from a script.
...

Aside from the portability concern, turning off auto-compilation doesn't
actually fix the problem.  If a compiled version has previously been
cached for the filename of a script being run, guile will consider
using the cached version even if --no-auto-compile was supplied: the
switch only controls the attempt to compile for the cache.  If the
cached compilation is up to date then it is used silently, which is OK.
But if it's out of date, because the cache was for a different script
that previously existed under the same name, then guile emits a banner
saying that it's out of date (implying that the cached compilation is
therefore not being used).  So the script's visible behaviour is defiled
even if it applies the option.

Observe what happens to the second script in this sequence:

$ echo '(display "hello world\n")' >t10
$ guile-2.0 t10
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/zefram/usr/guile/t10
;;; compiled /home/zefram/.cache/guile/ccache/2.0-LE-8-2.0/home/zefram/usr/guile/t10.go
hello world
$ echo '(display "goodbye world\n")' >t10
$ guile-2.0 --no-auto-compile t10        
;;; note: source file /home/zefram/usr/guile/t10
;;;       newer than compiled /home/zefram/.cache/guile/ccache/2.0-LE-8-2.0/home/zefram/usr/guile/t10.go
goodbye world

I have, however, come up with a truly ugly workaround.  The meta option
system can be used to introduce a -c option that explicitly loads the
script file via primitive-eval, which does not attempt compilation.
(Nor does it look at the compilation cache, so this even avoids the
problem that --no-auto-compile runs into.)  Running the script this way
yields a different command line (visible through (program-arguments))
from that which arrives when the script is run via -s, so if the script
is to process its command line, for robustness it must pay attention to
which way it was invoked.  All together, this looks like:

$ cat t11
#!/usr/bin/guile \
-c (begin\
\ \ \ (define\ arg-hack\ #t)\
\ \ \ (primitive-load\ (cadr\ (program-arguments))))
!#
(define argv
  (if (false-if-exception arg-hack)
    (cdr (program-arguments))
    (program-arguments)))
(write argv)
(newline)
$ guile-1.6 '\' t11 a b c
("t11" "a" "b" "c")
$ guile-1.6 -s t11 a b c 
("t11" "a" "b" "c")
$ guile-1.8 '\' t11 a b c
("t11" "a" "b" "c")
$ guile-1.8 -s t11 a b c 
("t11" "a" "b" "c")
$ guile-2.0 '\' t11 a b c
("t11" "a" "b" "c")
$ guile-2.0 -s t11 a b c 
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/zefram/usr/guile/t11
;;; /home/zefram/usr/guile/t11:7:6: warning: possibly unbound variable `arg-hack'
;;; compiled /home/zefram/.cache/guile/ccache/2.0-LE-8-2.0/home/zefram/usr/guile/t11.go
("t11" "a" "b" "c")
$ guile-2.0 -s t11 a b c
("t11" "a" "b" "c")

I'm not comfortable with this as a workaround.  It smells fragile.
Also note that though this does avoid the banner appearing for
#!-based executions, it's not muffling the banner per se but actually
preventing compilation.  While for some programs it's desirable to
prevent compilation per se (because of the compiler's limitations),
there are plenty of programs that would like to be compiled and only
want to muffle the banner.  Losing the efficiency of compilation is
potentially a high price to pay for clean output.

Guile should not be emitting this banner by default.  It's really not
acceptable to damage the visible behaviour of a program that worked
fine on older guile versions.  It also, for this auto-compilation to
serve as the invisible optimising cache as which it's intended, ought
to keep quiet about compilation failure: the fallback to interpreting
the script should be silent.

Debian incarnation of this bug report:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=734009

-zefram