GNU bug report logs - #31879
Caching in the "module import obarray" is not thread-safe

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: tracker <at> debbugs.gnu.org, bug-strong-list <at> debbugs.gnu.org
Subject: bug#31879: closed (Caching in the "module import obarray" is not
 thread-safe)
Date: Mon, 18 Jun 2018 16:17:04 +0000

[Message part 1 (text/plain, inline)]

Your message dated Mon, 18 Jun 2018 18:16:37 +0200
with message-id <871sd4ul22.fsf <at> gnu.org>
and subject line Re: bug#31879: Caching in the "module import obarray" is not thread-safe
has caused the debbugs.gnu.org bug report #31879,
regarding Caching in the "module import obarray" is not thread-safe
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
31879: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=31879
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: ludo <at> gnu.org (Ludovic Courtès)
To: bug-guile <at> gnu.org
Subject: Caching in the "module import obarray" is not thread-safe
Date: Mon, 18 Jun 2018 15:48:33 +0200

Dear thread lovers,

The code below spawns a bunch of threads that look up imported bindings
in a module.  It usually enters an infinite loop and possibly eats all
your memory (on Guile 2.2.3):

--8<---------------cut here---------------start------------->8---
(use-modules (ice-9 threads)
             (ice-9 match)
             (srfi srfi-1))

(define bindings
  ;; Bindings exported by (guile).
  '())

(module-for-each (lambda (name var)
                   (set! bindings (cons name bindings)))
                 (resolve-module '(guile)))

(define thread-count 8)
(define bindings-per-thread
  (floor (/ (length bindings) thread-count)))
(define iface
  (resolve-module '(ice-9 q)))

(define threads
  (unfold (lambda (x) (>= x thread-count))
          (lambda (n)
            (call-with-new-thread
             (lambda ()
               (let loop ((bindings (take (drop bindings
                                                (* n bindings-per-thread))
                                          bindings-per-thread)))
                 (match bindings
                   (() #t)
                   ((head . tail)
                    (module-variable iface head)
                    (loop tail)))))))
          1+
          0))

(for-each join-thread threads)
(pk 'done thread-count)
--8<---------------cut here---------------end--------------->8---

The issue lies in the “import obarray”, which is used as a cache and is
accessed in a non-thread-safe manner:

--8<---------------cut here---------------start------------->8---
static inline SCM
module_imported_variable (SCM module, SCM sym)
{
#define SCM_BOUND_THING_P scm_is_true
  register SCM var, imports;

  /* Search cached imported bindings.  */
  imports = SCM_MODULE_IMPORT_OBARRAY (module);
  var = scm_hashq_ref (imports, sym, SCM_UNDEFINED);
  if (SCM_BOUND_THING_P (var))
    return var;

[...]

    if (SCM_BOUND_THING_P (found_var))
      {
	/* Save the lookup result for future reference.  */
	(void) scm_hashq_set_x (imports, sym, found_var);
	return found_var;
      }
--8<---------------cut here---------------end--------------->8---

Possible solutions:

  1. Add a field in the module record containing a mutex.  Downside is
     that this breaks the ABI, though I’m not sure how much of a problem
     it is.

  2. Make the import obarray a weak hash table since they are
     thread-safe in 2.2.  This should be semantically equivalent to
     using a hash table provided interned symbols are not GC’d.

Thoughts?

Ludo’.

[Message part 3 (message/rfc822, inline)]

From: ludo <at> gnu.org (Ludovic Courtès)
To: 31879-done <at> debbugs.gnu.org
Subject: Re: bug#31879: Caching in the "module import obarray" is not
 thread-safe
Date: Mon, 18 Jun 2018 18:16:37 +0200

ludo <at> gnu.org (Ludovic Courtès) skribis:

> The issue lies in the “import obarray”, which is used as a cache and is
> accessed in a non-thread-safe manner:

Fixed in commit 46bcbfa566de19a88e925bd0369e110cae5a6b03, following a
suggestion by Andy.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.