GNU bug report logs - #20079
Fwd: Memory leak from seek/ftell with files larger than 2GB

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Anand Mohanadoss <anand108 <at> gmail.com>
Subject: bug#20079: closed (Re: bug#20079: Fwd: Memory leak from
 seek/ftell with files larger than 2GB)
Date: Thu, 23 Jun 2016 13:03:02 +0000

[Message part 1 (text/plain, inline)]

Your bug report

#20079: Fwd: Memory leak from seek/ftell with files larger than 2GB

which was filed against the guile package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to 20079 <at> debbugs.gnu.org.

-- 
20079: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20079
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Andy Wingo <wingo <at> pobox.com>
To: Anand Mohanadoss <anand108 <at> gmail.com>
Cc: 20079-done <at> debbugs.gnu.org
Subject: Re: bug#20079: Fwd: Memory leak from seek/ftell with files larger
 than 2GB
Date: Thu, 23 Jun 2016 15:01:57 +0200

Hi,

Thank you very much for this one!  Turns out we had an incredibly
embarrassing bug in which we forgot to attach finalizers for bignums
created by scm_from_{uint64,int64} on 32-bit platforms.  Fixed in master
and stable-2.0.

Cheers,

Andy

On Wed 11 Mar 2015 13:38, Anand Mohanadoss <anand108 <at> gmail.com> writes:

> Hi,
>
> I had sent the following to the user forum and did not receive any
> comments. I am reposting it in the bug forum with the hope that one of
> the experts may be able to comment...
>
> Thanks,
> Anand
>
> ---------- Forwarded message ----------
> From: Anand Mohanadoss <anand108 <at> gmail.com>
> Date: Wed, Feb 25, 2015 at 9:35 PM
> Subject: Memory leak from seek/ftell with files larger than 2GB
> To: guile-user <at> gnu.org
>
> Hi,
>
> We are seeing an issue with seek and ftell leaking memory with files
> larger than 2GB.
>
> We are using 2.0.11 guile built as a 32-bit application with large
> file support enabled (guile was built using gcc 4.4.0 for Linux with
> flags _FILE_OFFSET_BITS=64, _LARGEFILE_SOURCE and _
> LARGEFILE64_SOURCE). The issue also appears to happen with guile 2.2.
>
> The memory leaks start only after the offset exceeds maximum positive
> value for a 32-bit signed integer. ftell and seek do work as expected
> (given how lseek should work with large file support).
>
> Appended is a program that illustrates the problem. The first seek
> simply skips the part of the file where you won't see a memory leak.
> If you comment out ftell and the second seek lines and un-comment the
> lines that follow them, there is no memory leak. 
>
> Is this a bug in guile or should we be doing things differently? If
> this is a known issue, is there a recommended work around?
>
> Thanks,
> Anand
>
> (define MAX_SIGNED_INT 2147483647)
> (define BYTES_TO_READ 10)
>
> (define file "/tmp/test.pcap") ;sample file greater than 2.5GB
>
> (define (traverse file)
> (let* ((port (open-input-file file #:binary #t))
> (file-sz (stat:size (stat port)))
> (ua (make-bytevector BYTES_TO_READ 0))
> (cur-offset 0))
> (seek port (- MAX_UNSIGNED_INT 1000) SEEK_CUR)
> (while (< (ftell port) (- file-sz BYTES_TO_READ))
> ;(while (< cur-offset (- file-sz BYTES_TO_READ))
> (seek port BYTES_TO_READ SEEK_CUR)
> ;(get-bytevector-n! port ua 0 BYTES_TO_READ)
> (set! cur-offset (+ BYTES_TO_READ cur-offset)))
> (close-port port))) 
>
> (traverse file)

[Message part 3 (message/rfc822, inline)]

From: Anand Mohanadoss <anand108 <at> gmail.com>
To: bug-guile <at> gnu.org
Subject: Fwd: Memory leak from seek/ftell with files larger than 2GB
Date: Wed, 11 Mar 2015 18:08:18 +0530

[Message part 4 (text/plain, inline)]

Hi,

I had sent the following to the user forum and did not receive any
comments.  I am reposting it in the bug forum with the hope that one of the
experts may be able to comment...

Thanks,
Anand

---------- Forwarded message ----------
From: Anand Mohanadoss <anand108 <at> gmail.com>
Date: Wed, Feb 25, 2015 at 9:35 PM
Subject: Memory leak from seek/ftell with files larger than 2GB
To: guile-user <at> gnu.org


Hi,

We are seeing an issue with seek and ftell leaking memory with files larger
than 2GB.

We are using 2.0.11 guile built as a 32-bit application with large file
support enabled (guile was built using gcc 4.4.0 for Linux with flags
_FILE_OFFSET_BITS=64, _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE).  The
issue also appears to happen with guile 2.2.

The memory leaks start only after the offset exceeds maximum positive value
for a 32-bit signed integer. ftell and seek do work as expected (given how
lseek should work with large file support).

Appended is a program that illustrates the problem.  The first seek simply
skips the part of the file where you won't see a memory leak. If you
comment out ftell and the second seek lines and un-comment the lines that
follow them, there is no memory leak.

Is this a bug in guile or should we be doing things differently?  If this
is a known issue, is there a recommended work around?

Thanks,
Anand

(define MAX_SIGNED_INT 2147483647)
(define BYTES_TO_READ 10)

(define file "/tmp/test.pcap")  ;sample file greater than 2.5GB

(define (traverse file)
 (let* ((port (open-input-file file #:binary #t))
        (file-sz (stat:size (stat port)))
        (ua (make-bytevector BYTES_TO_READ 0))
        (cur-offset 0))
   (seek port (- MAX_UNSIGNED_INT 1000) SEEK_CUR)
   (while (< (ftell port) (- file-sz BYTES_TO_READ))
   ;(while (< cur-offset (- file-sz BYTES_TO_READ))
       (seek  port BYTES_TO_READ SEEK_CUR)
       ;(get-bytevector-n! port ua 0 BYTES_TO_READ)
       (set! cur-offset (+ BYTES_TO_READ cur-offset)))
   (close-port port)))

(traverse file)

[Message part 5 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.