Subject: bug#20079: closed (Re: bug#20079: Fwd: Memory leak from seek/ftell with files larger than 2GB)

From: [email-masked] (GNU bug Tracking System)
Date: Thu, 23 Jun 2016 13:03:02 +0000

Your bug report

#20079: Fwd: Memory leak from seek/ftell with files larger than 2GB

which was filed against the guile package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
20079: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20079
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#20079: Fwd: Memory leak from seek/ftell with files larger than 2GB
From: Andy Wingo <[email-masked]>
To: Anand Mohanadoss <[email-masked]>
Cc: [email-masked]
Date: Thu, 23 Jun 2016 15:01:57 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi,

Thank you very much for this one!  Turns out we had an incredibly
embarrassing bug in which we forgot to attach finalizers for bignums
created by scm_from_{uint64,int64} on 32-bit platforms.  Fixed in master
and stable-2.0.

Cheers,

Andy

On Wed 11 Mar 2015 13:38, Anand Mohanadoss <[email-masked]> writes:

> Hi,
>
> I had sent the following to the user forum and did not receive any
> comments. I am reposting it in the bug forum with the hope that one of
> the experts may be able to comment...
>
> Thanks,
> Anand
>
> ---------- Forwarded message ----------
> From: Anand Mohanadoss <[email-masked]>
> Date: Wed, Feb 25, 2015 at 9:35 PM
> Subject: Memory leak from seek/ftell with files larger than 2GB
> To: [email-masked]
>
> Hi,
>
> We are seeing an issue with seek and ftell leaking memory with files
> larger than 2GB.
>
> We are using 2.0.11 guile built as a 32-bit application with large
> file support enabled (guile was built using gcc 4.4.0 for Linux with
> flags _FILE_OFFSET_BITS=64, _LARGEFILE_SOURCE and _
> LARGEFILE64_SOURCE). The issue also appears to happen with guile 2.2.
>
> The memory leaks start only after the offset exceeds maximum positive
> value for a 32-bit signed integer. ftell and seek do work as expected
> (given how lseek should work with large file support).
>
> Appended is a program that illustrates the problem. The first seek
> simply skips the part of the file where you won't see a memory leak.
> If you comment out ftell and the second seek lines and un-comment the
> lines that follow them, there is no memory leak. 
>
> Is this a bug in guile or should we be doing things differently? If
> this is a known issue, is there a recommended work around?
>
> Thanks,
> Anand
>
> (define MAX_SIGNED_INT 2147483647)
> (define BYTES_TO_READ 10)
>
> (define file "/tmp/test.pcap") ;sample file greater than 2.5GB
>
> (define (traverse file)
> (let* ((port (open-input-file file #:binary #t))
> (file-sz (stat:size (stat port)))
> (ua (make-bytevector BYTES_TO_READ 0))
> (cur-offset 0))
> (seek port (- MAX_UNSIGNED_INT 1000) SEEK_CUR)
> (while (< (ftell port) (- file-sz BYTES_TO_READ))
> ;(while (< cur-offset (- file-sz BYTES_TO_READ))
> (seek port BYTES_TO_READ SEEK_CUR)
> ;(get-bytevector-n! port ua 0 BYTES_TO_READ)
> (set! cur-offset (+ BYTES_TO_READ cur-offset)))
> (close-port port))) 
>
> (traverse file)

Subject: Fwd: Memory leak from seek/ftell with files larger than 2GB
From: Anand Mohanadoss <[email-masked]>
To: [email-masked]
Date: Wed, 11 Mar 2015 18:08:18 +0530
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi,

I had sent the following to the user forum and did not receive any
comments.  I am reposting it in the bug forum with the hope that one of the
experts may be able to comment...

Thanks,
Anand

---------- Forwarded message ----------
From: Anand Mohanadoss <[email-masked]>
Date: Wed, Feb 25, 2015 at 9:35 PM
Subject: Memory leak from seek/ftell with files larger than 2GB
To: [email-masked]

Hi,

We are seeing an issue with seek and ftell leaking memory with files larger
than 2GB.

We are using 2.0.11 guile built as a 32-bit application with large file
support enabled (guile was built using gcc 4.4.0 for Linux with flags
_FILE_OFFSET_BITS=64, _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE).  The
issue also appears to happen with guile 2.2.

The memory leaks start only after the offset exceeds maximum positive value
for a 32-bit signed integer. ftell and seek do work as expected (given how
lseek should work with large file support).

Appended is a program that illustrates the problem.  The first seek simply
skips the part of the file where you won't see a memory leak. If you
comment out ftell and the second seek lines and un-comment the lines that
follow them, there is no memory leak.

Is this a bug in guile or should we be doing things differently?  If this
is a known issue, is there a recommended work around?

Thanks,
Anand

(define MAX_SIGNED_INT 2147483647)
(define BYTES_TO_READ 10)

(define file "/tmp/test.pcap")  ;sample file greater than 2.5GB

(define (traverse file)
 (let* ((port (open-input-file file #:binary #t))
        (file-sz (stat:size (stat port)))
        (ua (make-bytevector BYTES_TO_READ 0))
        (cur-offset 0))
   (seek port (- MAX_UNSIGNED_INT 1000) SEEK_CUR)
   (while (< (ftell port) (- file-sz BYTES_TO_READ))
   ;(while (< cur-offset (- file-sz BYTES_TO_READ))
       (seek  port BYTES_TO_READ SEEK_CUR)
       ;(get-bytevector-n! port ua 0 BYTES_TO_READ)
       (set! cur-offset (+ BYTES_TO_READ cur-offset)))
   (close-port port)))

(traverse file)