Subject: bug#10474: Building guile 2.x under mingw + msys

From: Eli Zaretskii <[email-masked]>
Date: Thu, 19 Jan 2012 19:35:04 +0200

> From: [email-masked] (Ludovic Courtès)
> Cc: [email-masked], [email-masked], [email-masked]
> Date: Thu, 19 Jan 2012 00:55:31 +0100
> 
> Can you send this one to [email-masked]?  (All the code under lib/
> comes from Gnulib.)

For the record, my report to bug-gnulib is here:

  http://lists.gnu.org/archive/html/bug-gnulib/2012-01/msg00253.html

> (The rest of your investigation is interesting!)

Here's some more ;-)

In the last episode, we stopped here:

       GUILEC ice-9/psyntax-pp.go
     Throw without catch before boot:
     Throw to key system-error with args ("make_objcode_from_file" "~A" ("No error") (0))Aborting.

It turns out that this happens because Guile reads and writes objcodes
from/to *.go files in text mode.  My solution was to use binary mode
in writing in mkstemp.c:

      fd = open (template, O_RDWR|O_BINARY|O_CREAT|O_EXCL, 0600);

and in reading in objcodes.c:load-objcode:

      fd = open (c_file, O_RDONLY | O_BINARY | O_CLOEXEC);

While the latter change looks as TRT in all cases, the former does
not: there's no guarantee that mkstemp! will be used only for
outputting *.go files.  It is probably best to have a wrapper around
mkstemp!, and use only that for writing to *.go files in compile.scm.

Next obstacle: Abort while compiling ice-9/poll.scm:

     In ice-9/eval.scm:
      389: 19 [eval # #]
      350: 18 [eval # #]
      434: 17 [lp (#<fluid 22>) ("")]
     In system/base/compile.scm:
      148: 16 [compile-file "ice-9/poll.scm" #:output-file ...]
       43: 15 [call-once #<procedure 184fa00 at system/base/compile.scm:56:5 ()>]
     In ice-9/boot-9.scm:
      184: 14 [with-throw-handler #t ...]
     In system/base/compile.scm:
       59: 13 [#<procedure 184f9e0 at system/base/compile.scm:58:9 ()>]
      151: 12 [#<procedure 184fa20 at system/base/compile.scm:149:8 (port)> #<input-output: ice-9/poll.go.LORZMf 6>]
      200: 11 [read-and-compile #<input: ice-9/poll.scm 5> #:from ...]
      212: 10 [lp (#) #<directory # 14ec900> #<directory # 14ec900>]
      178: 9 [lp (#<procedure compile-tree-il (x e opts)>) (eval-when # #) ...]
     In ice-9/boot-9.scm:
     2095: 8 [save-module-excursion #<procedure 1b9b2d0 at language/scheme/compile-tree-il.scm:29:3 ()>]
     In language/scheme/compile-tree-il.scm:
       31: 7 [#<procedure 1b9b2d0 at language/scheme/compile-tree-il.scm:29:3 ()>]
     In ./ice-9/psyntax.scm:
     1011: 6 [chi-top-sequence ((eval-when # #)) () ((top)) ...]
      898: 5 [scan ((eval-when # #)) () ((top)) ...]
      269: 4 [scan ((load-extension # "scm_init_poll")) () ((top)) ...]
     In unknown file:
	?: 3 [load-extension "libguile-2.0" "scm_init_poll"]
     In ice-9/boot-9.scm:
      115: 2 [#<procedure 184f9c0 at ice-9/boot-9.scm:110:6 (thrown-k . args)> misc-error ...]
     In unknown file:
	?: 1 [delete-file "ice-9/poll.go.LORZMf"]
     In ice-9/boot-9.scm:
      119: 0 [#<procedure 184f9c0 at ice-9/boot-9.scm:110:6 (thrown-k . args)> system-error ...]

     ice-9/boot-9.scm:119:20: In procedure #<procedure 184f9c0 at ice-9/boot-9.scm:110:6 (thrown-k . args)>:
     ice-9/boot-9.scm:119:20: In procedure delete-file: Permission denied
     make[2]: *** [ice-9/poll.go] Error 1

This happens because poll.scm does this:

     (eval-when (eval load compile)
       (load-extension (string-append "libguile-" (effective-version))
		       "scm_init_poll"))

while scm_init_poll has this snippet:

     static void
     scm_init_poll (void)
     {
     #if HAVE_POLL
       scm_c_define_gsubr ("primitive-poll", 4, 0, 0, scm_primitive_poll);
     #else
       scm_misc_error ("%init-poll", "`poll' unavailable on this platform", SCM_EOL);
     #endif

Since Windows doesn't HAVE_POLL, scm_misc_error throws an error.

It doesn't sound wise to fail the entire build procedure due to `poll'
being unavailable (AFAIK, not only MS-Windows lacks that library
function).  Maybe poll.scm should be compiled, or maybe it should do
some clever tricks around load-extension to avoid throwing an error at
compile time.  I'm not a Guile or Scheme person, so I cannot suggest a
good solution.

As a workaround, I used "make -k" to allow the build to continue past
this point.

There's another problem in the above backtrace: The form that catches
the thrown error cannot delete the temporary file
ice-9/poll.go.LORZMf, because it tries to delete a file that is still
open.  Changing call-with-output-file/atomic to close the temporary
file before deletion, like this:

     (define* (call-with-output-file/atomic filename proc #:optional reference)
       (let* ((template (string-append filename ".XXXXXX"))
	      (tmp (mkstemp! template)))
	 (call-once
	  (lambda ()
	    (with-throw-handler #t
	      (lambda ()
		(proc tmp)
		(chmod tmp (logand #o0666 (lognot (umask))))
		(close-port tmp)
		(rename-file template filename))
	      (lambda args
		(close-port tmp) ;; <<<<<<<<<<<<<<<<<<<<<<<<<<<
		(delete-file template)))))))

solves the problem, and we can now see the correct backtrace:

     Backtrace:
     In ice-9/eval.scm:
      389: 19 [eval # #]
      350: 18 [eval # #]
      389: 17 [eval # #]
      350: 16 [eval # #]
      434: 15 [lp (#<fluid 22>) ("")]
     In system/base/compile.scm:
      149: 14 [compile-file "ice-9/poll.scm" #:output-file ...]
       43: 13 [call-once #<procedure 17c84a0 at system/base/compile.scm:56:5 ()>]
     In ice-9/boot-9.scm:
      184: 12 [with-throw-handler #t ...]
     In system/base/compile.scm:
       59: 11 [#<procedure 17c8480 at system/base/compile.scm:58:9 ()>]
      152: 10 [#<procedure 17c84c0 at system/base/compile.scm:150:8 (port)> #<closed: file 0>]
      201: 9 [read-and-compile #<input: ice-9/poll.scm 5> #:from ...]
      213: 8 [lp (#) #<directory # 1bafca8> #<directory # 1bafca8>]
      179: 7 [lp (#<procedure compile-tree-il (x e opts)>) (eval-when # #) ...]
     In ice-9/boot-9.scm:
     2095: 6 [save-module-excursion #<procedure 185dea0 at language/scheme/compile-tree-il.scm:29:3 ()>]
     In language/scheme/compile-tree-il.scm:
       31: 5 [#<procedure 185dea0 at language/scheme/compile-tree-il.scm:29:3 ()>]
     In ./ice-9/psyntax.scm:
     1011: 4 [chi-top-sequence ((eval-when # #)) () ((top)) ...]
      898: 3 [scan ((eval-when # #)) () ((top)) ...]
      269: 2 [scan ((load-extension # "scm_init_poll")) () ((top)) ...]
     In unknown file:
	?: 1 [load-extension "libguile-2.0" "scm_init_poll"]
     In ice-9/boot-9.scm:
      119: 0 [#<procedure 17c8460 at ice-9/boot-9.scm:110:6 (thrown-k . args)> misc-error ...]

     ice-9/boot-9.scm:119:20: In procedure #<procedure 17c8460 at ice-9/boot-9.scm:110:6 (thrown-k . args)>:
     ice-9/boot-9.scm:119:20: In procedure %init-poll: `poll' unavailable on this platform

The build still stops, but now we have a human-readable description of
the reason.

The problem in poll.scm causes another failure while compiling
web/server/http.scm:

       GUILEC web/server/http.go
     Backtrace:
     In system/base/compile.scm:
      152: 19 [#<procedure 15d0b00 at system/base/compile.scm:150:8 (port)> #<closed: file 0>]
      201: 18 [read-and-compile #<input: web/server/http.scm 5> #:from ...]
      213: 17 [lp () #f #<module (#{ g60}#) 16324c8>]
      179: 16 [lp (#<procedure compile-tree-il (x e opts)>) (define-module # # ...) ...]
     In ice-9/boot-9.scm:
     2095: 15 [save-module-excursion #<procedure 194edb0 at language/scheme/compile-tree-il.scm:29:3 ()>]
     In language/scheme/compile-tree-il.scm:
       31: 14 [#<procedure 194edb0 at language/scheme/compile-tree-il.scm:29:3 ()>]
     In ./ice-9/psyntax.scm:
     1011: 13 [chi-top-sequence ((define-module # # # ...)) () ((top)) ...]
      898: 12 [scan ((define-module (web server http) #:use-module ...)) () ...]
      269: 11 [scan ((#(syntax-object let # ...) (#) (# #) ...)) () ...]
     In ice-9/eval.scm:
      374: 10 [eval # ()]
     In ice-9/boot-9.scm:
     2651: 9 [define-module* (web server http) #:filename ...]
     2626: 8 [resolve-imports ((# # #) (#) (#) (#) ...)]
     2564: 7 [resolve-interface (ice-9 poll) #:select ...]
     2489: 6 [#<procedure 14c98f0 at ice-9/boot-9.scm:2477:4 (name #:optional autoload version #:key ensure)> # ...]
     2756: 5 [try-module-autoload (ice-9 poll) #f]
     2095: 4 [save-module-excursion #<procedure 1b7f408 at ice-9/boot-9.scm:2757:17 ()>]
     2767: 3 [#<procedure 1b7f408 at ice-9/boot-9.scm:2757:17 ()>]
     In unknown file:
	?: 2 [primitive-load-path "ice-9/poll" #f]
	?: 1 [load-extension "libguile-2.0" "scm_init_poll"]
     In ice-9/boot-9.scm:
      119: 0 [#<procedure 15d0aa0 at ice-9/boot-9.scm:110:6 (thrown-k . args)> misc-error ...]

     ice-9/boot-9.scm:119:20: In procedure #<procedure 15d0aa0 at ice-9/boot-9.scm:110:6 (thrown-k . args)>:
     ice-9/boot-9.scm:119:20: In procedure %init-poll: `poll' unavailable on this platform
     make[2]: *** [web/server/http.go] Error 1

Other than that, the build finally succeeds!

P.S.  I found a couple of other possible places that don't support
Windows file names:

   ice-9/boot-9.scm:

     (define* (load-in-vicinity dir path #:optional reader)
       (define (canonical->suffix canon)
	 (cond
	  ((string-prefix? "/" canon) canon)
	  ((and (> (string-length canon) 2)
		(eqv? (string-ref canon 1) #\:))
	   ;; Paths like C:... transform to /C...
	   (string-append "/" (substring canon 0 1) (substring canon 2)))
	  (else canon)))

   system/base/compile.scm:

       (define (canonical->suffix canon)
	 (cond
	  ((string-prefix? "/" canon) canon)
	  ((and (> (string-length canon) 2)
		(eqv? (string-ref canon 1) #\:))
	   ;; Paths like C:... transform to /C...
	   (string-append "/" (substring canon 0 1) (substring canon 2)))
	  (else canon)))

I don't understand the "Paths like C:... transform to /C..." part.
Does that assume Cygwin?  Because the "transformed" file names will
not work in the native Windows build, which is what MinGW produces.

P.P.S.  I have still a few non-fatal warnings to investigate and
perhaps report, if they turn out to be real problems.  And then
there's a test suite.  Stay tuned.