GNU bug report logs - #24357
pure guile program leaks memory

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 24357 in the body.
You can then email your comments to 24357 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Amirouche Boubekki <amirouche <at> hypermove.net>
To: bug-guile <at> gnu.org
Subject: pure guile program leaks memory
Date: Sat, 03 Sep 2016 09:20:36 +0200

[Message part 1 (text/plain, inline)]

Using guile 2.1.3, I have a program that:

- reads urls from a text file
- download the urls using curl command via popen
- output the result to stdout

Also, it relies on n-for-each-par-map for ice-9 threads.

IMO, the most suspicious is the definition of the `curl' proc:


  (define (curl url)
     (let* ((port (open-input-pipe (format #f "curl -is \"~a\"" url)))
            (response (read-string port)))
       (close-pipe port)
       response))


I fail to see how this can leak.

To reproduce the bug you need a giant list of preferably different urls.
Such a list is available at http://hyperdev.fr/data/hn/hn.urls.txt.xz

You can run the test program with the following command:

   cat hn.urls.txt | guile urls-step00-fetch.scm > /dev/null

This is not the only program I tried on guile-2.1.3 that leaks but
it's the easiest to reproduce.

I will try to reproduce the bug on 2.0.12.

[urls-step00-fetch.scm (text/plain, attachment)]

Message #8 received at 24357 <at> debbugs.gnu.org (full text, mbox):

From: Andy Wingo <wingo <at> pobox.com>
To: Amirouche Boubekki <amirouche <at> hypermove.net>
Cc: 24357 <at> debbugs.gnu.org
Subject: Re: bug#24357: pure guile program leaks memory
Date: Sat, 03 Sep 2016 11:54:58 +0200

On Sat 03 Sep 2016 09:20, Amirouche Boubekki <amirouche <at> hypermove.net> writes:

> Using guile 2.1.3, I have a program that:
>
> - reads urls from a text file
> - download the urls using curl command via popen
> - output the result to stdout
>
> Also, it relies on n-for-each-par-map for ice-9 threads.

Can you reduce it please?  For example, remove the use of threads.

Andy

Message #11 received at 24357 <at> debbugs.gnu.org (full text, mbox):

From: Amirouche Boubekki <amirouche <at> hypermove.net>
To: Andy Wingo <wingo <at> pobox.com>
Cc: 24357 <at> debbugs.gnu.org
Subject: Re: bug#24357: pure guile program leaks memory
Date: Sat, 03 Sep 2016 20:49:41 +0200

[Message part 1 (text/plain, inline)]

On 2016-09-03 11:54, Andy Wingo wrote:
> On Sat 03 Sep 2016 09:20, Amirouche Boubekki <amirouche <at> hypermove.net> 
> writes:
> 
>> Using guile 2.1.3, I have a program that:
>> 
>> - reads urls from a text file
>> - download the urls using curl command via popen
>> - output the result to stdout
>> 
>> Also, it relies on n-for-each-par-map for ice-9 threads.
> 
> Can you reduce it please?  For example, remove the use of threads.
> 

Ok. I removed threads and only download the same url over and over 
again.

Here is the error I get on stdout:

   (23) Failed writing body

The program is:

(use-modules (ice-9 popen))


;;; wrapping curl command

(define (curl url)
  (let* ((port (open-input-pipe (format #f "curl -is \"~a\"" url)))
         (response (read-string port)))
    (close-pipe port)
    response))


(define (maybe-curl url)
  (catch #t
    (lambda ()
      (display "." (current-error-port))
      (write (cons url (curl url))))
    (lambda _ '())))

(define urls (map (lambda _ "http://hyperdev.fr/") (iota 1000)))

(display "started")

(for-each maybe-curl urls)

[guile-bug-24357.scm (text/plain, attachment)]

Message #14 received at 24357 <at> debbugs.gnu.org (full text, mbox):

From: Amirouche Boubekki <amirouche <at> hypermove.net>
To: Andy Wingo <wingo <at> pobox.com>
Cc: bug-guile <bug-guile-bounces+amirouche+dev=hypermove.net <at> gnu.org>,
 24357 <at> debbugs.gnu.org
Subject: Re: bug#24357: pure guile program leaks memory
Date: Mon, 12 Sep 2016 19:47:02 +0200

On 2016-09-03 20:49, Amirouche Boubekki wrote:
> On 2016-09-03 11:54, Andy Wingo wrote:
>> On Sat 03 Sep 2016 09:20, Amirouche Boubekki <amirouche <at> hypermove.net> 
>> writes:
>> 
>>> Using guile 2.1.3, I have a program that:
>>> 
>>> - reads urls from a text file
>>> - download the urls using curl command via popen
>>> - output the result to stdout
>>> 
>>> Also, it relies on n-for-each-par-map for ice-9 threads.
>> 
>> Can you reduce it please?  For example, remove the use of threads.
>> 
> 
> Ok. I removed threads and only download the same url over and over 
> again.
> 
> Here is the error I get on stdout:
> 
>    (23) Failed writing body
> 

Internet says this a curl issue. Please close this bug.

Message #19 received at 24357-done <at> debbugs.gnu.org (full text, mbox):

From: Andy Wingo <wingo <at> pobox.com>
To: Amirouche Boubekki <amirouche <at> hypermove.net>
Cc: 24357-done <at> debbugs.gnu.org
Subject: Re: bug#24357: pure guile program leaks memory
Date: Wed, 01 Mar 2017 10:06:50 +0100

On Mon 12 Sep 2016 19:47, Amirouche Boubekki <amirouche <at> hypermove.net> writes:

> On 2016-09-03 20:49, Amirouche Boubekki wrote:
>> On 2016-09-03 11:54, Andy Wingo wrote:
>>> On Sat 03 Sep 2016 09:20, Amirouche Boubekki
>>> <amirouche <at> hypermove.net> writes:
>>>
>>>> Using guile 2.1.3, I have a program that:
>>>>
>>>> - reads urls from a text file
>>>> - download the urls using curl command via popen
>>>> - output the result to stdout
>>>>
>>>> Also, it relies on n-for-each-par-map for ice-9 threads.
>>>
>>> Can you reduce it please?  For example, remove the use of threads.
>>>
>>
>> Ok. I removed threads and only download the same url over and over
>> again.
>>
>> Here is the error I get on stdout:
>>
>>    (23) Failed writing body
>
> Internet says this a curl issue. Please close this bug.

No problem.  For reference, in the future just add "-done" after the bug
number in the mail, as I did above.  Cheers :)

Andy

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.