GNU bug report logs - #5989
23.1.96; bug 4209 not completely fixed (regression in re-search-forward)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 5989 in the body.
You can then email your comments to 5989 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Christopher J. Madsen" <cjm <at> cjmweb.net>
To: bug-gnu-emacs <at> gnu.org
Subject: 23.1.96; bug 4209 not completely fixed (regression in
	re-search-forward)
Date: Tue, 20 Apr 2010 13:19:21 -0500

I've discovered that bug #4209 was only partially fixed.  My original
recipe does not produce the bug in 23.1.96, but a slightly modified
one does:

;--- re-bug2.el starts here
(setq-default enable-multibyte-characters nil) ; this line is new

(set-buffer (get-buffer-create "*Test Buffer*"))

(insert "\xC2\x4C\xEF\x77\xC6\x69\x8C\x0A")

(goto-char (point-min))

(message "looking-at: %s" (looking-at
"\\`\xC2\x4C\xEF\x77\xC6\x69\x8C\x0A"))

(message "re-search-forward: %s"
         (re-search-forward "\\`\xC2\x4C\xEF\x77\xC6\x69\x8C\x0A" 100 t))
;--- re-bug2.el ends here

At the command line, this is the expected output:

$ emacs --batch -Q -l re-bug2.el
looking-at: t
re-search-forward: 9

With emacs 23.1.96, the output I actually get is:
looking-at: t
re-search-forward: nil


This recipe is closer to what I'm actually doing:

;--- re-bug-file.el starts here
(find-file-literally "sample.file")

(goto-char (point-min))

(message "looking-at: %s" (looking-at
"\\`\xC2\x4C\xEF\x77\xC6\x69\x8C\x0A"))

(message "re-search-forward: %s"
         (re-search-forward "\\`\xC2\x4C\xEF\x77\xC6\x69\x8C\x0A" 100 t))
;--- re-bug-file.el ends here

It should have the same output as the previous recipe.  sample.file
must begin with those 8 bytes.  The following Perl command can create
a suitable file (just redirect its output to sample.file):

perl -e 'binmode STDOUT; print "\xC2\x4C\xEF\x77\xC6\x69\x8C\x0Aand more\n"'



The original patch proposed by Matthew Dempsky
(http://debbugs.gnu.org/cgi/bugreport.cgi?bug=4209#12) appears to fix
the bug in all its variations, but the patch that was actually applied
by Kenichi Handa only fixes the original recipe.

I apologize for not trying the official patch sooner.  I just never
got around to rebuilding Emacs.


In GNU Emacs 23.1.96.4 (i686-pc-linux-gnu)
 of 2010-04-20 on byte
configured using `configure  '--without-pop' '--without-sound'
'--without-x' '--with-x-toolkit=no' '--without-xpm' '--without-jpeg'
'--without-tiff' '--without-gif' '--without-png' '--without-rsvg'
'--without-xft' '--without-libotf' '--without-m17n-flt'
'--without-toolkit-scroll-bars' '--without-xaw3d' '--without-xim'
'--without-gpm' '--without-dbus' '--without-makeinfo''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.utf8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Fundamental

Minor modes in effect:
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr message sendmail regexp-opt ecomplete rfc822 mml
mml-sec password-cache mm-decode mm-bodies mm-encode mailcap mail-parse
rfc2231 rfc2047 rfc2045 qp ietf-drums mailabbrev nnheader gnus-util
netrc time-date mm-util mail-prsvr gmm-utils wid-edit mailheader canlock
sha1 hex-util hashcash mail-utils emacsbug help-mode easymenu view
warnings ediff-hook vc-hooks lisp-float-type lisp-mode register page
menu-bar rfn-eshadow timer jit-lock font-lock syntax facemenu font-core
frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev loaddefs button minibuffer faces cus-face files
text-properties overlay md5 base64 format env code-pages mule custom
widget hashtable-print-readable backquote make-network-process multi-tty
emacs)

Message #8 received at 5989 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: 5989 <at> debbugs.gnu.org
Subject: bug#5989: 23.1.96;
 bug 4209 not completely fixed (regression in re-search-forward)
Date: Wed, 1 Jun 2016 21:29:36 -0400

found 5989 23.4
found 5989 24.5
found 5989 25.0.94
tag 5989 + confirmed
quit

I can reproduce this on recent Emacs, with a small change to the
recipe since enable-multibyte-characters can no longer be set
directly:

(set-buffer (get-buffer-create "*Test Buffer*"))
(set-buffer-multibyte nil) ; instead of setting `enable-multibyte-characters'

(insert "\xC2\x4C\xEF\x77\xC6\x69\x8C\x0A")

(goto-char (point-min))

(message "looking-at: %s" (looking-at
"\\`\xC2\x4C\xEF\x77\xC6\x69\x8C\x0A"))

(message "re-search-forward: %s"
         (re-search-forward "\\`\xC2\x4C\xEF\x77\xC6\x69\x8C\x0A" 100 t))

Message #19 received at 5989 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: 5989 <at> debbugs.gnu.org
Subject: Re: bug#5989: 23.1.96;
 bug 4209 not completely fixed (regression in re-search-forward)
Date: Sat, 04 Jun 2016 19:15:50 +0300

> Date: Wed, 1 Jun 2016 21:29:36 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> 
> I can reproduce this on recent Emacs, with a small change to the
> recipe since enable-multibyte-characters can no longer be set
> directly:
> 
> (set-buffer (get-buffer-create "*Test Buffer*"))
> (set-buffer-multibyte nil) ; instead of setting `enable-multibyte-characters'
> 
> (insert "\xC2\x4C\xEF\x77\xC6\x69\x8C\x0A")
> 
> (goto-char (point-min))
> 
> (message "looking-at: %s" (looking-at
> "\\`\xC2\x4C\xEF\x77\xC6\x69\x8C\x0A"))
> 
> (message "re-search-forward: %s"
>          (re-search-forward "\\`\xC2\x4C\xEF\x77\xC6\x69\x8C\x0A" 100 t))

If you don't make the buffer unibyte, everything works as expected.
Using multibyte buffers even for raw bytes is what we recommend.  So I
see no problem here that we should work on: unibyte buffers are not
meant to process text.

Thanks.

Message #22 received at 5989 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: 5989 <at> debbugs.gnu.org
Cc: Eli Zaretskii <eliz <at> gnu.org>, cjm <at> cjmweb.net
Subject: Re: bug#5989: 23.1.96;
 bug 4209 not completely fixed (regression in re-search-forward)
Date: Sat, 4 Jun 2016 12:51:07 -0400

tag 5989 + notabug
quit

On Sat, Jun 4, 2016 at 12:15 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> If you don't make the buffer unibyte, everything works as expected.
> Using multibyte buffers even for raw bytes is what we recommend.  So I
> see no problem here that we should work on: unibyte buffers are not
> meant to process text.


Makes sense. (adding OP back on CC; still getting used to debbugs, so
I had dropped them in my previous message)

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.