GNU bug report logs - #25355
26.0.50; Provide function to non-destructively replace buffer contents

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 25355 in the body.
You can then email your comments to 25355 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Philipp <p.stephani2 <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 26.0.50; Provide function to non-destructively replace buffer contents
Date: Wed, 04 Jan 2017 13:45:32 +0100

There are many tools (e.g. auto-formatters) that take buffer contents,
reformat them, and write the reformatted output somewhere.  However,
there is no good way how to apply the modified output to the source
buffer.  The naive way (erasing and re-inserting the buffer contents)
loses point and markers.  Therefore there should be a function
(e.g. `replace-buffer-contents') that calculates a minimal diff between
the old and the new contents and uses editing operations (insert,
delete, etc.) to apply the diff.  Ideally this would be done without any
external tools (e.g. 'diff').


In GNU Emacs 26.0.50.25 (x86_64-unknown-linux-gnu, GTK+ Version 3.10.8)
 of 2017-01-04 built on unknown
Repository revision: 44c588a25ce231ce05fb535cd6d7162e91214f45
Windowing system distributor 'The X.Org Foundation', version 11.0.11501000
System Description:	Ubuntu 14.04 LTS

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --with-modules --enable-checking
 --enable-check-lisp-object-type 'CFLAGS=-ggdb3 -O0''

Configured features:
XPM JPEG TIFF GIF PNG SOUND GSETTINGS NOTIFY GNUTLS FREETYPE XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 MODULES

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message subr-x puny seq byte-opt gv
bytecomp byte-compile cl-extra help-mode cconv cl-loaddefs pcase cl-lib
dired dired-loaddefs format-spec rfc822 mml easymenu mml-sec
password-cache epa derived epg epg-config gnus-util rmail rmail-loaddefs
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr
mail-utils time-date mule-util tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript case-table epa-hook jka-cmpr-hook help
simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs button
faces cus-face macroexp files text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote inotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 97764 8542)
 (symbols 48 20217 1)
 (miscs 40 331 131)
 (strings 32 18017 4427)
 (string-bytes 1 592819)
 (vectors 16 14087)
 (vector-slots 8 472891 7102)
 (floats 8 181 43)
 (intervals 56 218 0)
 (buffers 976 12)
 (heap 1024 36851 1046))

-- 
Google Germany GmbH
Erika-Mann-Straße 33
80636 München

Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle

Diese E-Mail ist vertraulich.  Wenn Sie nicht der richtige Adressat sind,
leiten Sie diese bitte nicht weiter, informieren Sie den Absender und löschen
Sie die E-Mail und alle Anhänge.  Vielen Dank.

This e-mail is confidential.  If you are not the right addressee please do not
forward it, please inform the sender, and please erase this e-mail including
any attachments.  Thanks.

Message #8 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Philipp <p.stephani2 <at> gmail.com>
Cc: 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50;
 Provide function to non-destructively replace buffer contents
Date: Wed, 04 Jan 2017 08:38:15 -0500

severity 25355 wishlist
quit

Philipp <p.stephani2 <at> gmail.com> writes:

> There are many tools (e.g. auto-formatters) that take buffer contents,
> reformat them, and write the reformatted output somewhere.  However,
> there is no good way how to apply the modified output to the source
> buffer.  The naive way (erasing and re-inserting the buffer contents)
> loses point and markers.  Therefore there should be a function
> (e.g. `replace-buffer-contents') that calculates a minimal diff between
> the old and the new contents and uses editing operations (insert,
> delete, etc.) to apply the diff.  Ideally this would be done without any
> external tools (e.g. 'diff').

This is pretty similar (though not identical) to
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=18.

Message #13 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50; Provide function to non-destructively replace
 buffer contents
Date: Sun, 07 May 2017 17:44:27 +0000

[Message part 1 (text/plain, inline)]

Philipp <p.stephani2 <at> gmail.com> schrieb am Mi., 4. Jan. 2017 um 13:46 Uhr:

>
> There are many tools (e.g. auto-formatters) that take buffer contents,
> reformat them, and write the reformatted output somewhere.  However,
> there is no good way how to apply the modified output to the source
> buffer.  The naive way (erasing and re-inserting the buffer contents)
> loses point and markers.  Therefore there should be a function
> (e.g. `replace-buffer-contents') that calculates a minimal diff between
> the old and the new contents and uses editing operations (insert,
> delete, etc.) to apply the diff.  Ideally this would be done without any
> external tools (e.g. 'diff').
>
>
FYI, I've now implemented this based on libmba (
http://www.ioplex.com/~miallen/libmba/). Unless there are concerns
importing part of that library into the Emacs source tree, I'll send a
patch.

[Message part 2 (text/html, inline)]

Message #16 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50;
 Provide function to non-destructively replace buffer contents
Date: Sun, 07 May 2017 21:05:50 +0300

> From: Philipp Stephani <p.stephani2 <at> gmail.com>
> Date: Sun, 07 May 2017 17:44:27 +0000
> 
> FYI, I've now implemented this based on libmba (http://www.ioplex.com/~miallen/libmba/). Unless there are
> concerns importing part of that library into the Emacs source tree, I'll send a patch.

I'm not sure I understand what you mean by "importing part of that
library".  Are you going to ask the author to sign legal papers
assigning copyright to the FSF?

P.S. Doesn't GNU Diff have code to do this?  Even dispnew.c has
something similar, see scrolling_window.

Message #19 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50; Provide function to non-destructively replace
 buffer contents
Date: Sun, 07 May 2017 18:21:35 +0000

[Message part 1 (text/plain, inline)]

Eli Zaretskii <eliz <at> gnu.org> schrieb am So., 7. Mai 2017 um 20:06 Uhr:

> > From: Philipp Stephani <p.stephani2 <at> gmail.com>
> > Date: Sun, 07 May 2017 17:44:27 +0000
> >
> > FYI, I've now implemented this based on libmba (
> http://www.ioplex.com/~miallen/libmba/). Unless there are
> > concerns importing part of that library into the Emacs source tree, I'll
> send a patch.
>
> I'm not sure I understand what you mean by "importing part of that
> library".  Are you going to ask the author to sign legal papers
> assigning copyright to the FSF?
>

I guess we can just use the library as-is? It's MIT-licensed.


>
> P.S. Doesn't GNU Diff have code to do this?


Sure, but it's not a library. libmba was the only library that I've found
so far with a usable interface (i.e. that doesn't assume that the input
sequences are byte arrays).


> Even dispnew.c has
> something similar, see scrolling_window.
>

That doesn't appear to be reusable.

[Message part 2 (text/html, inline)]

Message #22 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50; Provide function to non-destructively replace
 buffer contents
Date: Sun, 07 May 2017 21:37:27 +0300

> From: Philipp Stephani <p.stephani2 <at> gmail.com>
> Date: Sun, 07 May 2017 18:21:35 +0000
> Cc: 25355 <at> debbugs.gnu.org
> 
> I guess we can just use the library as-is? It's MIT-licensed.

What do you mean by "use"?  Treat it as an optional library, like we
do with image support and GnuTLS?  I thought you wanted this feature
to be present in every Emacs, not only Emacs linked against that
library.

>  P.S. Doesn't GNU Diff have code to do this? 
> 
> Sure, but it's not a library.

Why do you need a library, if you want the code to become part of
Emacs?

The advantage of Diff is that I think the situation with copyright
assignment will be much easier there ;-)

Message #25 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50; Provide function to non-destructively replace
 buffer contents
Date: Sun, 07 May 2017 18:45:07 +0000

[Message part 1 (text/plain, inline)]

Eli Zaretskii <eliz <at> gnu.org> schrieb am So., 7. Mai 2017 um 20:38 Uhr:

> > From: Philipp Stephani <p.stephani2 <at> gmail.com>
> > Date: Sun, 07 May 2017 18:21:35 +0000
> > Cc: 25355 <at> debbugs.gnu.org
> >
> > I guess we can just use the library as-is? It's MIT-licensed.
>
> What do you mean by "use"?  Treat it as an optional library, like we
> do with image support and GnuTLS?  I thought you wanted this feature
> to be present in every Emacs, not only Emacs linked against that
> library.
>

I just want to link it statically by adding the required source files to
the source tree, like we do for e.g. oldXmenu.


>
> >  P.S. Doesn't GNU Diff have code to do this?
> >
> > Sure, but it's not a library.
>
> Why do you need a library, if you want the code to become part of
> Emacs?
>

I want something that can be used right now without reimplementing the diff
algorithm from scratch.


>
> The advantage of Diff is that I think the situation with copyright
> assignment will be much easier there ;-)
>

If this requires copyright assignment, then I'll stop immediately and
reimplement this as a dynamic module outside of Emacs. Asking for copyright
assignment is way too much work for this.

[Message part 2 (text/html, inline)]

Message #28 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50; Provide function to non-destructively replace
 buffer contents
Date: Sun, 07 May 2017 22:19:07 +0300

> From: Philipp Stephani <p.stephani2 <at> gmail.com>
> Date: Sun, 07 May 2017 18:45:07 +0000
> Cc: 25355 <at> debbugs.gnu.org
> 
>  What do you mean by "use"? Treat it as an optional library, like we
>  do with image support and GnuTLS? I thought you wanted this feature
>  to be present in every Emacs, not only Emacs linked against that
>  library.
> 
> I just want to link it statically by adding the required source files to the source tree, like we do for e.g.
> oldXmenu.

That means we need legal papers, AFAIU.

> If this requires copyright assignment, then I'll stop immediately and reimplement this as a dynamic module
> outside of Emacs.

Maybe I'm missing something, but I don't see how legal paperwork could
be avoided in this case.  Perhaps ask RMS.

Message #31 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50; Provide function to non-destructively replace
 buffer contents
Date: Sun, 14 May 2017 19:30:48 +0000

[Message part 1 (text/plain, inline)]

Eli Zaretskii <eliz <at> gnu.org> schrieb am So., 7. Mai 2017 um 21:19 Uhr:

> > From: Philipp Stephani <p.stephani2 <at> gmail.com>
> > Date: Sun, 07 May 2017 18:45:07 +0000
> > Cc: 25355 <at> debbugs.gnu.org
> >
> >  What do you mean by "use"? Treat it as an optional library, like we
> >  do with image support and GnuTLS? I thought you wanted this feature
> >  to be present in every Emacs, not only Emacs linked against that
> >  library.
> >
> > I just want to link it statically by adding the required source files to
> the source tree, like we do for e.g.
> > oldXmenu.
>
> That means we need legal papers, AFAIU.
>

That's quite unfortunate, because it will prevent many useful libraries
from being added to Emacs.
How did we end up with oldXmenu? It's also MIT-licensed and not
FSF-copyrighted.

[Message part 2 (text/html, inline)]

Message #34 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Richard Stallman <rms <at> gnu.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: eliz <at> gnu.org, 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50;
 Provide function to non-destructively replace buffer contents
Date: Sun, 14 May 2017 21:47:49 -0400

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > > I just want to link it statically by adding the required source files to
  > > the source tree, like we do for e.g.
  > > > oldXmenu.
  > >
  > > That means we need legal papers, AFAIU.

We don't need papers for the source code of a free library that is
used outside of Emacs.

Originally we made Emacs link with the Xmenu library.  When
that library became obsolete but we kept using it, we put it
into the Emacs release and renamed it to oldXmenu, but we continued
to say it was not a part of Emacs, just used with Emacs.

We can include other libraries the same way.

BTW, please don't use the term "MIT license" or "MIT-licensed".  See
https://gnu.org/licenses/license-list.html.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Message #37 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: rms <at> gnu.org
Cc: p.stephani2 <at> gmail.com, 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50;
 Provide function to non-destructively replace buffer contents
Date: Mon, 15 May 2017 05:49:12 +0300

> From: Richard Stallman <rms <at> gnu.org>
> CC: eliz <at> gnu.org, 25355 <at> debbugs.gnu.org
> Date: Sun, 14 May 2017 21:47:49 -0400
> 
> We don't need papers for the source code of a free library that is
> used outside of Emacs.

Could you please clarify what does "used outside of Emacs" mean?

The proposal in this case, AFAIU, is to add an external library as
part of Emacs sources that are included in the tarball.

Message #40 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50;
 Provide function to non-destructively replace buffer contents
Date: Mon, 15 May 2017 18:09:06 -0400

Philipp Stephani wrote:

> I just want to link it statically by adding the required source files to
> the source tree, like we do for e.g. oldXmenu.

Please bear in mind that the negative aspects of this, as summarized at
eg https://wiki.gentoo.org/wiki/Why_not_bundle_dependencies

IMO oldXmenu is not a useful comparison because:
1) it doesn't exist as a separate entity any more
2) few (to no?) Emacs builds actually use it in practice
3) it's basically historical baggage that should be removed from Emacs
(but removing things from Emacs is difficult).

Message #43 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Richard Stallman <rms <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: p.stephani2 <at> gmail.com, 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50;
 Provide function to non-destructively replace buffer contents
Date: Mon, 15 May 2017 21:08:58 -0400

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > We don't need papers for the source code of a free library that is
  > > used outside of Emacs.

  > Could you please clarify what does "used outside of Emacs" mean?

I'm making the distinction between a contribution to Emacs
and a library that is not specifically for Emacs
but which we happen to use in Emacs.

Xmenu wasn't written for Emacs.  It was released for general use
and was used by various programs.  Then we made Emacs use it too.

Thus, Xmenu was not a contribution to Emacs, not specially for Emacs.
Rather, it was a free library that was used outside of Emacs.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Message #46 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50; Provide function to non-destructively replace
 buffer contents
Date: Mon, 12 Jun 2017 21:17:38 +0000

[Message part 1 (text/plain, inline)]

Philipp Stephani <p.stephani2 <at> gmail.com> schrieb am So., 7. Mai 2017 um
20:21 Uhr:

>
>
>> P.S. Doesn't GNU Diff have code to do this?
>
>
> Sure, but it's not a library. libmba was the only library that I've found
> so far with a usable interface (i.e. that doesn't assume that the input
> sequences are byte arrays).
>
>

Apparently I didn't look closely enough: Diffutils uses an implementation
of the Myers algorithm from Gnulib, which is customizable enough to fit our
needs. I've attached a patch that uses Gnulib.

[Message part 2 (text/html, inline)]

[0001-Add-command-to-replace-buffer-contents.txt (text/plain, attachment)]

Message #51 received at 25355-done <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 25355-done <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50; Provide function to non-destructively replace
 buffer contents
Date: Sat, 17 Jun 2017 13:47:48 +0000

[Message part 1 (text/plain, inline)]

Philipp Stephani <p.stephani2 <at> gmail.com> schrieb am Mo., 12. Juni 2017 um
23:17 Uhr:

> Philipp Stephani <p.stephani2 <at> gmail.com> schrieb am So., 7. Mai 2017 um
> 20:21 Uhr:
>
>>
>>
>>> P.S. Doesn't GNU Diff have code to do this?
>>
>>
>> Sure, but it's not a library. libmba was the only library that I've found
>> so far with a usable interface (i.e. that doesn't assume that the input
>> sequences are byte arrays).
>>
>>
>
> Apparently I didn't look closely enough: Diffutils uses an implementation
> of the Myers algorithm from Gnulib, which is customizable enough to fit our
> needs. I've attached a patch that uses Gnulib.
>

I've pushed a slightly modified version of this patch as d682f0daa3.

[Message part 2 (text/html, inline)]

Message #54 received at 25355 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: 25355 <at> debbugs.gnu.org
Subject: Re: bug#25355: 26.0.50; Provide function to non-destructively replace
 buffer contents
Date: Sat, 17 Jun 2017 19:06:45 +0300

> From: Philipp Stephani <p.stephani2 <at> gmail.com>
> Date: Sat, 17 Jun 2017 13:47:48 +0000
> Cc: 25355-done <at> debbugs.gnu.org
> 
>  Apparently I didn't look closely enough: Diffutils uses an implementation of the Myers algorithm from
>  Gnulib, which is customizable enough to fit our needs. I've attached a patch that uses Gnulib. 
> 
> I've pushed a slightly modified version of this patch as d682f0daa3. 

Thanks.  Could you also update the manuals with the information about
the new function?  After doing that, please mark the NEWS entry with
"+++".

Btw, while compiling I see this warning:

  In file included from editfns.c:3139:0:
  ../lib/diffseq.h: In function 'diag':
  ../lib/diffseq.h:210:12: warning: variable 'big_snake' set but not used [-Wunused-but-set-variable]
	 bool big_snake = false;
	      ^

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.