GNU bug report logs - #28962
cperl-mode: inserting newline at start of line in HERE

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 28962 in the body.
You can then email your comments to 28962 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Troy Hinckley <troyhinckley <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: inserting newline at start of line in HERE doc breaks syntax
 highlighting
Date: Mon, 23 Oct 2017 17:13:32 -0600

[Message part 1 (text/plain, inline)]

From: tjhinckl <at> sccj002243.i-did-not-set--mail-host-address--so-tickle-me
(troy.j.hinckley)
To: bug-gnu-emacs <at> gnu.org
Subject: 25.1; cperl syntax highlighting broken
--text follows this line--

When inside a cperl HERE doc, using any function that inserts a newline at
the start of a line will causes the rest of the file to highlighted with
font-lock-comment-face. The only way to fix this is to make changes to the
correctly highlighted part of the HERE doc which causes it to re-parse. My
guess is that cperl is getting mixed up with POD highlighting and trying to
apply cperl-pod-face since POD's and HERE documents are very closely
related in cperl-mode code.

To reproduce:
use the minimal cperl-mode file below
---------------------------------------------
my $here_doc = <<'_HERE_';
this is here doc line

_HERE_
---------------------------------------------
call `M-: (insert "\n")` when cursor is at the start of any line
inside the HERE doc.

You will see that the rest of the file is highlighted with the comment
face. Calling any function that finds the HERE doc region will be incorrect
(e.g. cperl-narrow-to-here-doc)



In GNU Emacs 25.1.1 (x86_64-suse-linux-gnu, GTK+ Version 2.18.6)
 of 2016-10-09 built on plxv1010
Windowing system distributor 'The X.Org Foundation', version 11.0.60900000
System Description: SUSE Linux Enterprise Server 11 (x86_64)

Configured using:
 'configure --prefix=/usr/intel/pkgs/emacs/25.1
 --prefix=/usr/intel/pkgs/emacs/25.1
 --libdir=/usr/intel/pkgs/emacs/25.1/lib64
 --x-includes=/usr/intel/pkgs/X11/R7.7/include
 --x-libraries=/usr/intel/pkgs/X11/R7.7/lib64 --with-x-toolkit=gtk2
 --without-sound --without-gif --build=x86_64-suse-linux
 --host=x86_64-suse-linux --target=x86_64-suse-linux CFLAGS= CPPFLAGS=
 'LDFLAGS=-O2 -fPIC -L/usr/intel/pkgs/emacs/25.1/lib64
 -L/usr/intel/pkgs/emacs/25.1/lib -Wl,--rpath
 -Wl,/usr/intel/pkgs/emacs/25.1/lib64:/usr/intel/pkgs/emacs/25.1/lib:/usr/intel/pkgs/emacs/25.1/lib64:/usr/intel/pkgs/emacs/25.1/lib:/usr/intel/pkgs/emacs/25.1/lib:/usr/intel/pkgs/emacs/25.1/lib64:/usr/intel/pkgs/gtk+/2.18.6-64/lib64:/usr/intel/pkgs/atk/1.28.0-64/lib64:/usr/intel/pkgs/pango/1.26.2-64/lib64:/usr/intel/pkgs/cairo/1.8.8-64/lib:/usr/intel/pkgs/poppler/0.12.1-64/lib:/usr/intel/pkgs/pixman/0.16.2-64/lib:/usr/intel/pkgs/X11/R7.5-64/lib:/usr/intel/pkgs/groff/1.20.1-64/:/usr/intel/pkgs/fontconfig/2.7.3-64/lib64:/usr/intel/pkgs/libexpat/2.0.1-64/lib:/usr/intel/pkgs/freetype/2.3.7-64/lib:/usr/intel/pkgs/zlib/1.2.x-64/lib64:/usr/intel/pkgs/libpng/1.2.40-64/lib:/usr/intel/pkgs/libxml2/2.7.6-64/lib64:/usr/intel/pkgs/lcms/1.19-64/lib64:/usr/intel/pkgs/tiff/3.9.1-64/lib64:/usr/intel/pkgs/zlib/1.2.x-64/lib64:/usr/intel/pkgs/jpeg/6b-64/lib:/usr/intel/pkgs/glib/2.22.3-64/lib64:/usr/intel/pkgs/libgettext/0.17-2-64/lib64:/usr/intel/pkgs/ncurses/5.6-64/lib64:/usr/intel/pkgs/libpng/1.2.40-64/lib:/usr/intel/pkgs/zlib/1.2.x-64/lib64:/usr/intel/pkgs/libiconv/1.13.1-64/lib:''

Configured features:
XPM JPEG TIFF PNG GPM NOTIFY ACL FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS
GTK2 X11

Important settings:
  value of $LC_ALL: en_US.UTF-8
  value of $LC_COLLATE: C
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: CPerl

Minor modes in effect:
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Features:
(shadow sort mail-extr emacsbug message dired format-spec rfc822 mml
mml-sec password-cache epg gnus-util mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util help-fns mail-prsvr mail-utils cperl-mode
use-package diminish bind-key easy-mmode finder-inf advice edmacro
kmacro rx info package epg-config seq byte-opt gv bytecomp byte-compile
cl-extra help-mode easymenu cconv cl-loaddefs pcase cl-lib time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd fontset
image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote inotify dynamic-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 129555 5523)
 (symbols 48 24348 0)
 (miscs 40 142 120)
 (strings 32 30653 5834)
 (string-bytes 1 1166381)
 (vectors 16 17248)
 (vector-slots 8 524591 2239)
 (floats 8 198 61)
 (intervals 56 260 0)
 (buffers 976 17)
 (heap 1024 17165 1148))

[Message part 2 (text/html, inline)]

Message #10 received at 28962 <at> debbugs.gnu.org (full text, mbox):

From: Harald Jörg <haj <at> posteo.de>
To: 28962 <at> debbugs.gnu.org
Subject: A workaround to this issue is available in cperl-mode
Date: Wed, 26 Aug 2020 18:26:11 +0200

The root cause is that within HERE documents and PODs there's no
syntax re-checking for every character inserted.

To recover from such a situation, cperl-mode offers a
command `cperl-find-pods-heres', also available from the menu
'Perl' -> 'Refresh "hard" constructions'.  A function which inserts
text in a POD or HERE document should call cperl-find-pods-heres
afterwards to avoid the issue.
-- 
Cheers,
haj

Message #13 received at 28962 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Harald Jörg <haj <at> posteo.de>, 28962 <at> debbugs.gnu.org
Subject: Re: bug#28962: A workaround to this issue is available in cperl-mode
Date: Wed, 26 Aug 2020 13:21:57 -0700

Harald Jörg <haj <at> posteo.de> writes:

> The root cause is that within HERE documents and PODs there's no
> syntax re-checking for every character inserted.
>
> To recover from such a situation, cperl-mode offers a
> command `cperl-find-pods-heres', also available from the menu
> 'Perl' -> 'Refresh "hard" constructions'.  A function which inserts
> text in a POD or HERE document should call cperl-find-pods-heres
> afterwards to avoid the issue.

Is that something we should work on improving, or does that mean that we
should close this as wontfix?

Message #16 received at 28962 <at> debbugs.gnu.org (full text, mbox):

From: Harald Jörg <haj <at> posteo.de>
To: Stefan Kangas <stefankangas <at> gmail.com>, 28962 <at> debbugs.gnu.org
Subject: Re: bug#28962: A workaround to this issue is available in cperl-mode
Date: Thu, 27 Aug 2020 10:58:51 +0200

On 8/26/20 10:21 PM, Stefan Kangas wrote:
> Harald Jörg writes:
> 
>> The root cause is that within HERE documents and PODs there's no
>> syntax re-checking for every character inserted.
>>
>> To recover from such a situation, cperl-mode offers a
>> command `cperl-find-pods-heres', also available from the menu
>> 'Perl' -> 'Refresh "hard" constructions'.  A function which inserts
>> text in a POD or HERE document should call cperl-find-pods-heres
>> afterwards to avoid the issue.
> 
> Is that something we should work on improving, or does that mean that we
> should close this as wontfix?

After some more digging (my experience with elisp is still limited)
I found that the problem can be avoided by using `insert-and-inherit'
instead of `insert'.  cperl-mode uses text properties to detect the
HEREiness of buffer contents - and `insert' just doesn't provide them.

So, the problem only occurs when text is inserted via elisp code. It
can be avoided by using `insert-and-inherit', or, if one can't modify
the function, recovered from by calling `cperl-find-pods-heres'.  In
my eyes this is good enough.  It seems that the reporter wasn't aware
of either possibility, so maybe they're happy if we tell them about
the workarounds - and then close as wontfix.
-- 
Cheers,
haj

Message #19 received at 28962 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Harald Jörg <haj <at> posteo.de>, 28962 <at> debbugs.gnu.org
Subject: Re: bug#28962: A workaround to this issue is available in cperl-mode
Date: Thu, 27 Aug 2020 02:36:56 -0700

Harald Jörg <haj <at> posteo.de> writes:

> After some more digging (my experience with elisp is still limited)
> I found that the problem can be avoided by using `insert-and-inherit'
> instead of `insert'.  cperl-mode uses text properties to detect the
> HEREiness of buffer contents - and `insert' just doesn't provide them.
>
> So, the problem only occurs when text is inserted via elisp code. It
> can be avoided by using `insert-and-inherit', or, if one can't modify
> the function, recovered from by calling `cperl-find-pods-heres'.  In
> my eyes this is good enough.  It seems that the reporter wasn't aware
> of either possibility, so maybe they're happy if we tell them about
> the workarounds - and then close as wontfix.

Makes sense to me.  I think you should do that.

Message #22 received at 28962 <at> debbugs.gnu.org (full text, mbox):

From: haj <at> posteo.de (Harald Jörg)
To: 28962 <at> debbugs.gnu.org
Date: Mon, 23 Aug 2021 10:50:20 +0000

merge 28962 14343
thanks

After some more digging, I'm merging this bug with Bug#14343.

The root cause of both bugs is the same: Here-documents in CPerl mode
are marked with text properties.  Normal editing (in Emacs: via
`self-insert-command' will make new characters inherit from their
surroundings, so all is fine.

If, however, text is entered with another elisp function, then no
automatic inheritance takes place.  Examples are `insert' (this bug
report), `yank' (Bug#14343), but also `replace-string' if the first
character of the here-document gets replaced.

I am confident that this can be fixed if CPerl mode "steals" from Perl
mode and marks here-documents as c-style comments for fontification.
I am going to prepare a patch for this.

-- 
Cheers,
haj

Message #29 received at 28962-done <at> debbugs.gnu.org (full text, mbox):

From: haj <at> posteo.de (Harald Jörg)
To: 28962-done <at> debbugs.gnu.org
Subject: cperl-mode: Inserting text into here-documents now works
Date: Tue, 31 Aug 2021 07:04:31 +0000

The bug which made CPerl mode mistreat changes to here-documents has
been fixed in the repository, for the cases of (insert "\n") (Bug#28962)
and yanking (Bug#14343) in particular, and in general for changes by
elisp code.

cperl-mode.el from the repository can be used with Emacs 26.1 or newer.
-- 
Cheers,
haj

Message #37 received at 28962 <at> debbugs.gnu.org (full text, mbox):

From: Vincent Lefevre <vincent <at> vinc17.net>
To: Harald Jörg <haj <at> posteo.de>
Cc: 28962 <at> debbugs.gnu.org, 14343 <at> debbugs.gnu.org
Subject: Re: cperl-mode: Inserting text into here-documents now works
Date: Tue, 31 Aug 2021 11:12:29 +0200

Hi,

On 2021-08-31 07:04:31 +0000, Harald Jörg wrote:
> The bug which made CPerl mode mistreat changes to here-documents has
> been fixed in the repository, for the cases of (insert "\n") (Bug#28962)
> and yanking (Bug#14343) in particular, and in general for changes by
> elisp code.
> 
> cperl-mode.el from the repository can be used with Emacs 26.1 or newer.

Concerning my bug #14343, I had noted 4 years ago in the associated
Debian bug that I could no longer reproduce the issue in Emacs 25:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=706701#17

So it seems that there had already been some fixes in the past...

(I could also check that there were no regressions in 26.1 and 27.1.)

-- 
Vincent Lefèvre <vincent <at> vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.