Subject: Re: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files'

From: Michael Heerdegen <[email-masked]>
Date: Fri, 02 Mar 2018 14:04:53 +0100

Michael Heerdegen <[email-masked]> writes:

> Noam Postavsky <[email-masked]> writes:
>
> > Maybe I've misunderstood, but is it not the case that iterating over
> > (stream-range 1 n) should require only a constant amount of memory,
> > regardless of the value of n?

But in this regard, we have a problem with how lexical-binding is
implemented for interpreted code.  Nested thunks (as implemented in
"thunk.el") accumulate useless variable bindings - e.g.

(defun test ()
  (thunk-force
   (thunk-delay
    (thunk-force
     (thunk-delay
      (thunk-force
       (thunk-delay
        (lambda () 1))))))))
(test)
==>
#1=(closure
((check)
(#:val . #1#)
(#:forced . t)
(check)
(#:val . #1#)
(#:forced . t)
(check)
(#:val . #1#)
(#:forced . t)
t)
nil 1)

The length of the variable list is equivalent to the number of thunk
wrappers.  I believe that these useless variable lists are responsible
for the crashes of the uncompiled versions of the test files I had
posted.  I think this problem is different from the gc issue.

Streams use nested thunks.  Of course does thunk.el not explicitly add
such variable lists to the result - this is how closures are built in
interpreted code.  For nested thunks these just add up.

BTW, if you byte-compile the above `test' function, then

(disassemble (test))
==>
byte code:
  args: nil
0       constant  1
1       return

and this problem is gone.

Michael.