Subject: Manual gc helps

From: Linas Vepstas <[email-masked]>
Date: Sat, 7 Jan 2017 20:57:26 -0600

I did a fairly through review of the thread-creation and thread-join
code in the git master branch, and it looks to be just fine. Thus,
some experimentation is in order:

Going back to guile-2.0, I see this behavior:
guile -v
guile (GNU Guile) 2.0.11
Packaged by Debian (2.0.11-deb+1-10)

If I add a manual gc to the exit of the thread, like so:

   (define (mkthr v) (call-with-new-thread (lambda ()
         (set! junk (+ junk 1)) (gc) )))

then the heap blows up, in minutes, to about 180MB but then stops
growing, even after hours and millions of thread creates:
(heap-size . 183734272) (gc-times . 1957954)

If I gc only every third thread create, it quickly blows up to about
400MB, and then stablizes, for hours:
(heap-size . 428638208) (gc-times . 1292663)

If I gc every 17th thread, it blows up to about 1.8GB and then is stable:
(heap-size . 1875902464) (gc-times . 327462)

This last one after about 5.5 million thread creates and joins.
The counting is done like so:

   (define (mkthr v) (call-with-new-thread (lambda ()
                (lock-mutex mtx)
                (if (eq? 0 (modulo junk 17)) (gc))
                (set! junk (+ junk 1))
                (unlock-mutex mtx)
                )))

In each case, it seems to hit a plateau at about (n+1)*100MB when gc
is done on one out of every n threads.  This seems quite bizarre to
me: why does this inverted relation on number of gcs vs number of
thread creates? What's magic about 100MB? Clearly 100MB is wayyy too
large for this very simple program.  I mean, even if I gc at *every*
thread-exit ...

(I have not yet explored above in guile-2.2)

Since I cannot find any 'obvious' bugs in guile, this suggests some
strange stochastic behavior in bdw-gc?