GNU bug report logs - #66375
30.0.50; (error "Maximum buffer size exceeded") from (insert-file-contents "/dev/null")

Previous Next

To reply to this bug, email your comments to 66375 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sam Steingold <sds <at> gnu.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.0.50; (error "Maximum buffer size exceeded") from
 (insert-file-contents "/dev/null")
Date: Fri, 06 Oct 2023 13:11:35 -0400

emacs -Q

(insert-file-contents "/dev/null")
==>
Debugger entered--Lisp error: (error "Maximum buffer size exceeded")
  insert-file-contents("/dev/null")


In GNU Emacs 30.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version
 3.24.33, cairo version 1.16.0) of 2023-10-06 built on pop-os
Repository revision: 505c80623049d9e181918acdac8229c9a2041b1e
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101004
System Description: Pop!_OS 22.04 LTS

Configured using:
 'configure --with-mailutils --with-native-compilation
 --with-imagemagick'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ IMAGEMAGICK
JPEG JSON LCMS2 LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY
PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LC_COLLATE: C
  value of $LANG: C
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  pyvenv-mode: t
  global-atomic-chrome-edit-mode: t
  global-edit-server-edit-mode: t
  server-mode: t
  winner-mode: t
  which-function-mode: t
  url-handler-mode: t
  desktop-save-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  column-number-mode: t
  line-number-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Memory information:
((conses 16 1197252 211019) (symbols 48 35274 49) (strings 32 325111 14525)
 (string-bytes 1 9424577) (vectors 16 159905) (vector-slots 8 2352900 163968)
 (floats 8 927 484) (intervals 56 9746 2819) (buffers 992 73))

-- 
Sam Steingold (https://aphar.dreamwidth.org/) on Pop 22.04 (jammy) X 11.0.12101004
https://lastingimpactpsychology.com https://steingoldpsychology.com
https://iris.org.il https://honestreporting.com https://mideasttruth.com
Any connection between your reality and mine is purely coincidental.

Message #8 received at 66375 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: sds <at> gnu.org, Po Lu <luangruo <at> yahoo.com>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 66375 <at> debbugs.gnu.org
Subject: Re: bug#66375: 30.0.50;
 (error "Maximum buffer size exceeded") from (insert-file-contents
 "/dev/null")
Date: Sat, 07 Oct 2023 08:54:55 +0300

> From: Sam Steingold <sds <at> gnu.org>
> Date: Fri, 06 Oct 2023 13:11:35 -0400
> 
> 
> emacs -Q
> 
> (insert-file-contents "/dev/null")
> ==>
> Debugger entered--Lisp error: (error "Maximum buffer size exceeded")
>   insert-file-contents("/dev/null")

This is a "feature".  From the doc string:

  When inserting data from a special file (e.g., /dev/urandom), you
  can’t specify VISIT or BEG, and END should be specified to avoid
  inserting unlimited data into the buffer from some special files
  which otherwise could supply infinite amounts of data.

Maybe we could make a special exception for the null-device, whose
name we generally know on each supported system.

Technically, the problem is that the null device is seekable, so this:

  if (seekable || !NILP (end))
    total = end_offset - beg_offset;

computes 'total' to be a very large value, and then this:

  /* Ensure the gap is at least one byte larger than needed for the
     estimated file size, so that in the usual case we read to EOF
     without reallocating.  */
  if (GAP_SIZE <= total)
    make_gap (total - GAP_SIZE + 1);

attempts to make a gap very large, which errors out.  And that is
before we try to read even a single byte from the file.

Po Lu and Paul, any ideas?

Message #11 received at 66375 <at> debbugs.gnu.org (full text, mbox):

From: Po Lu <luangruo <at> yahoo.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: sds <at> gnu.org, 66375 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#66375: 30.0.50; (error "Maximum buffer size exceeded") from
 (insert-file-contents "/dev/null")
Date: Sat, 07 Oct 2023 14:43:45 +0800

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Sam Steingold <sds <at> gnu.org>
>> Date: Fri, 06 Oct 2023 13:11:35 -0400
>> 
>> 
>> emacs -Q
>> 
>> (insert-file-contents "/dev/null")
>> ==>
>> Debugger entered--Lisp error: (error "Maximum buffer size exceeded")
>>   insert-file-contents("/dev/null")
>
> This is a "feature".  From the doc string:
>
>   When inserting data from a special file (e.g., /dev/urandom), you
>   can’t specify VISIT or BEG, and END should be specified to avoid
>   inserting unlimited data into the buffer from some special files
>   which otherwise could supply infinite amounts of data.
>
> Maybe we could make a special exception for the null-device, whose
> name we generally know on each supported system.
>
> Technically, the problem is that the null device is seekable, so this:
>
>   if (seekable || !NILP (end))
>     total = end_offset - beg_offset;
>
> computes 'total' to be a very large value, and then this:
>
>   /* Ensure the gap is at least one byte larger than needed for the
>      estimated file size, so that in the usual case we read to EOF
>      without reallocating.  */
>   if (GAP_SIZE <= total)
>     make_gap (total - GAP_SIZE + 1);
>
> attempts to make a gap very large, which errors out.  And that is
> before we try to read even a single byte from the file.
>
> Po Lu and Paul, any ideas?

I think treating /dev/null specially is suitable, but there is a
different shortcoming in the approach presently taken by
insert-file-contents:

      if (!regular)
	end_offset = TYPE_MAXIMUM (off_t);
      else
	{
	  end_offset = st.st_size;

I think that the code computing total is presently beset by a
presumption that special files are uniformly non-seekable and loses if
that assumption proves untrue.  Perhaps it should disregard end_offset
if isn't expressly provided by the caller and the file is special, which
if true implies that end_offset is TYPE_MAXIMUM (off_t); much as in the
following untested patch:

diff --git a/src/fileio.c b/src/fileio.c
index 8919e08e1fd..e09616fb337 100644
--- a/src/fileio.c
+++ b/src/fileio.c
@@ -4746,7 +4746,7 @@ DEFUN ("insert-file-contents", Finsert_file_contents, Sinsert_file_contents,
       goto handled;
     }
 
-  if (seekable || !NILP (end))
+  if (((regular || !NILP (end)) && seekable) || !NILP (end))
     total = end_offset - beg_offset;
   else
     /* For a special file, all we can do is guess.  */

Message #14 received at 66375 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Po Lu <luangruo <at> yahoo.com>
Cc: sds <at> gnu.org, 66375 <at> debbugs.gnu.org, eggert <at> cs.ucla.edu
Subject: Re: bug#66375: 30.0.50; (error "Maximum buffer size exceeded") from
 (insert-file-contents "/dev/null")
Date: Sat, 07 Oct 2023 10:43:24 +0300

> From: Po Lu <luangruo <at> yahoo.com>
> Cc: sds <at> gnu.org,  Paul Eggert <eggert <at> cs.ucla.edu>,  66375 <at> debbugs.gnu.org
> Date: Sat, 07 Oct 2023 14:43:45 +0800
> 
> --- a/src/fileio.c
> +++ b/src/fileio.c
> @@ -4746,7 +4746,7 @@ DEFUN ("insert-file-contents", Finsert_file_contents, Sinsert_file_contents,
>        goto handled;
>      }
>  
> -  if (seekable || !NILP (end))
> +  if (((regular || !NILP (end)) && seekable) || !NILP (end))
>      total = end_offset - beg_offset;
>    else
>      /* For a special file, all we can do is guess.  */
> 

This does a couple of redundant tests:

  . regular non-zero means seekable must be non-zero
  . NILP (end) is tested twice for now good reason

I think the above should be cleaned up to better understand the logic.

More generally, I think 'total' should not be computed in this naïve
way when end_offset is -1, i.e. unless end_offset is either the result
of fstat/stat or the result of actually seeking to the end of
file/device.

Message #17 received at 66375 <at> debbugs.gnu.org (full text, mbox):

From: Sam Steingold <sds <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Po Lu <luangruo <at> yahoo.com>, Paul Eggert <eggert <at> cs.ucla.edu>,
 66375 <at> debbugs.gnu.org
Subject: Re: bug#66375: 30.0.50; (error "Maximum buffer size exceeded") from
 (insert-file-contents "/dev/null")
Date: Sun, 8 Oct 2023 21:57:02 -0400

On Sat, 7 Oct 2023 at 01:54, Eli Zaretskii <eliz <at> gnu.org> wrote:
>
> > From: Sam Steingold <sds <at> gnu.org>
> > Date: Fri, 06 Oct 2023 13:11:35 -0400
> > emacs -Q
> >
> > (insert-file-contents "/dev/null")
> > ==>
> > Debugger entered--Lisp error: (error "Maximum buffer size exceeded")
> >   insert-file-contents("/dev/null")
>
> This is a "feature".

this is new behavior.
I used /dev/null as "empty gnu score file since forever"


-- 
Sam Steingold <http://sds.podval.org> <http://www.childpsy.net>
<http://steingoldpsychology.com>

Message #20 received at 66375 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Sam Steingold <sds <at> gnu.org>
Cc: luangruo <at> yahoo.com, eggert <at> cs.ucla.edu, 66375 <at> debbugs.gnu.org
Subject: Re: bug#66375: 30.0.50; (error "Maximum buffer size exceeded") from
 (insert-file-contents "/dev/null")
Date: Mon, 09 Oct 2023 13:56:48 +0300

> From: Sam Steingold <sds <at> gnu.org>
> Date: Sun, 8 Oct 2023 21:57:02 -0400
> Cc: Po Lu <luangruo <at> yahoo.com>, Paul Eggert <eggert <at> cs.ucla.edu>, 66375 <at> debbugs.gnu.org
> 
> On Sat, 7 Oct 2023 at 01:54, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >
> > > From: Sam Steingold <sds <at> gnu.org>
> > > Date: Fri, 06 Oct 2023 13:11:35 -0400
> > > emacs -Q
> > >
> > > (insert-file-contents "/dev/null")
> > > ==>
> > > Debugger entered--Lisp error: (error "Maximum buffer size exceeded")
> > >   insert-file-contents("/dev/null")
> >
> > This is a "feature".
> 
> this is new behavior.
> I used /dev/null as "empty gnu score file since forever"

You still can do that, you just need to use a non-nil END argument.

And maybe we will find a better solution.

Message #23 received at 66375 <at> debbugs.gnu.org (full text, mbox):

From: Sam Steingold <sds <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: luangruo <at> yahoo.com, eggert <at> cs.ucla.edu, 66375 <at> debbugs.gnu.org
Subject: Re: bug#66375: 30.0.50; (error "Maximum buffer size exceeded") from
 (insert-file-contents "/dev/null")
Date: Sun, 15 Oct 2023 11:43:24 -0400

I am not sure what you mean here, sorry.
I am unable to use Gnus because of this new bug.

On Mon, 9 Oct 2023 at 06:57, Eli Zaretskii <eliz <at> gnu.org> wrote:
>
> > From: Sam Steingold <sds <at> gnu.org>
> > Date: Sun, 8 Oct 2023 21:57:02 -0400
> > Cc: Po Lu <luangruo <at> yahoo.com>, Paul Eggert <eggert <at> cs.ucla.edu>, 66375 <at> debbugs.gnu.org
> >
> > On Sat, 7 Oct 2023 at 01:54, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > >
> > > > From: Sam Steingold <sds <at> gnu.org>
> > > > Date: Fri, 06 Oct 2023 13:11:35 -0400
> > > > emacs -Q
> > > >
> > > > (insert-file-contents "/dev/null")
> > > > ==>
> > > > Debugger entered--Lisp error: (error "Maximum buffer size exceeded")
> > > >   insert-file-contents("/dev/null")
> > >
> > > This is a "feature".
> >
> > this is new behavior.
> > I used /dev/null as "empty gnu score file since forever"
>
> You still can do that, you just need to use a non-nil END argument.
>
> And maybe we will find a better solution.



-- 
Sam Steingold <http://sds.podval.org> <http://www.childpsy.net>
<http://steingoldpsychology.com>

Message #26 received at 66375 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Sam Steingold <sds <at> gnu.org>
Cc: luangruo <at> yahoo.com, eggert <at> cs.ucla.edu, 66375 <at> debbugs.gnu.org
Subject: Re: bug#66375: 30.0.50; (error "Maximum buffer size exceeded") from
 (insert-file-contents "/dev/null")
Date: Sun, 15 Oct 2023 19:33:26 +0300

> From: Sam Steingold <sds <at> gnu.org>
> Date: Sun, 15 Oct 2023 11:43:24 -0400
> Cc: luangruo <at> yahoo.com, eggert <at> cs.ucla.edu, 66375 <at> debbugs.gnu.org
> 
> I am not sure what you mean here, sorry.
> I am unable to use Gnus because of this new bug.

Where does Gnus do something like that?  We can at least install a
quick workaround if you can tell where is this code in Gnus.

Message #29 received at 66375 <at> debbugs.gnu.org (full text, mbox):

From: Sam Steingold <sds <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: luangruo <at> yahoo.com, eggert <at> cs.ucla.edu, 66375 <at> debbugs.gnu.org
Subject: Re: bug#66375: 30.0.50; (error "Maximum buffer size exceeded") from
 (insert-file-contents "/dev/null")
Date: Sun, 15 Oct 2023 12:44:18 -0400

When I enter a group for which I set score file to nil or /dev/null in
gnus-score-file-single-match-alist
I get this:

Debugger entered--Lisp error: (error "Maximum buffer size exceeded")
  insert-file-contents("/dev/null")
  gnus-score-load-score-alist("/dev/null")
  gnus-score-load-file("/dev/null")
  gnus-score-load-files(("/home/sds/.gnus-kill/new-thread"
"/home/sds/.gnus-kill/lisp" "/dev/null" nil
"/home/sds/.gnus-kill/non-ascii" "/home/sds/.gnus-kill/all"
"/home/sds/.gnus-kill/gmane" "/home/sds/.gnus-kill/gnu"
"/home/sds/.gnus-kill/emacs"))
  gnus-score-headers(("/home/sds/.gnus-kill/new-thread"
"/home/sds/.gnus-kill/lisp" "/dev/null" nil
"/home/sds/.gnus-kill/non-ascii" "/home/sds/.gnus-kill/all"
"/home/sds/.gnus-kill/gmane" "/home/sds/.gnus-kill/gnu"
"/home/sds/.gnus-kill/emacs") nil)
  gnus-possibly-score-headers()
  gnus-summary-read-group-1("nntp+news.gwene.org:gmane.emacs.bbdb.user"
nil t nil nil nil)
  gnus-summary-read-group("nntp+news.gwene.org:gmane.emacs.bbdb.user"
nil t nil nil nil nil)
  gnus-group-read-group(nil t)
  gnus-group-select-group(nil)
  funcall-interactively(gnus-group-select-group nil)
  command-execute(gnus-group-select-group)



On Sun, 15 Oct 2023 at 12:33, Eli Zaretskii <eliz <at> gnu.org> wrote:
>
> > From: Sam Steingold <sds <at> gnu.org>
> > Date: Sun, 15 Oct 2023 11:43:24 -0400
> > Cc: luangruo <at> yahoo.com, eggert <at> cs.ucla.edu, 66375 <at> debbugs.gnu.org
> >
> > I am not sure what you mean here, sorry.
> > I am unable to use Gnus because of this new bug.
>
> Where does Gnus do something like that?  We can at least install a
> quick workaround if you can tell where is this code in Gnus.



-- 
Sam Steingold <http://sds.podval.org> <http://www.childpsy.net>
<http://steingoldpsychology.com>

Message #32 received at 66375 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Sam Steingold <sds <at> gnu.org>
Cc: luangruo <at> yahoo.com, eggert <at> cs.ucla.edu, 66375 <at> debbugs.gnu.org
Subject: Re: bug#66375: 30.0.50; (error "Maximum buffer size exceeded") from
 (insert-file-contents "/dev/null")
Date: Sun, 15 Oct 2023 21:22:20 +0300

> From: Sam Steingold <sds <at> gnu.org>
> Date: Sun, 15 Oct 2023 12:44:18 -0400
> Cc: luangruo <at> yahoo.com, eggert <at> cs.ucla.edu, 66375 <at> debbugs.gnu.org
> 
> When I enter a group for which I set score file to nil or /dev/null in
> gnus-score-file-single-match-alist
> I get this:
> 
> Debugger entered--Lisp error: (error "Maximum buffer size exceeded")
>   insert-file-contents("/dev/null")
>   gnus-score-load-score-alist("/dev/null")
>   gnus-score-load-file("/dev/null")

gnus-score-file-single-match-alist is a defcustom, so you could
replace "/dev/null" in the alist with some real file that is empty,
couldn't you?

Message #35 received at 66375 <at> debbugs.gnu.org (full text, mbox):

From: Ed Santiago <bug-emacs <at> edsantiago.com>
To: 66375 <at> debbugs.gnu.org
Subject: 30.0.50;
 (error "Maximum buffer size exceeded") from (insert-file-contents
 "/dev/null")
Date: Thu, 14 Dec 2023 09:46:32 -0700

Another reproducer, somewhat more common than /dev/null:

    $ emacs --insert <(date)

"date" is trivial, but '<(some-command-that-produces-output)' is
a common pattern. Emacs 29 breaks this.

Message #38 received at 66375 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Ed Santiago <bug-emacs <at> edsantiago.com>
Cc: 66375 <at> debbugs.gnu.org
Subject: Re: bug#66375: 30.0.50;
 (error "Maximum buffer size exceeded") from (insert-file-contents
 "/dev/null")
Date: Thu, 14 Dec 2023 20:43:32 +0200

> From: Ed Santiago <bug-emacs <at> edsantiago.com>
> Phone: 1-505-662-5142
> Date: Thu, 14 Dec 2023 09:46:32 -0700
> 
> Another reproducer, somewhat more common than /dev/null:
> 
>     $ emacs --insert <(date)
> 
> "date" is trivial, but '<(some-command-that-produces-output)' is
> a common pattern. Emacs 29 breaks this.

The issue with '<(some-command-that-produces-output)' is already fixed
on the emacs-29 branch, so it is no longer part of this bug report.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.