From unknown Thu Aug 14 18:40:31 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6286: Ruby Mode Missing Syntax Resent-From: Nick Ewing Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 27 May 2010 22:02:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 6286 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 6286@debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.127499771331078 (code B ref -1); Thu, 27 May 2010 22:02:02 +0000 Received: (at submit) by debbugs.gnu.org; 27 May 2010 22:01:53 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OHl9D-00085D-V2 for submit@debbugs.gnu.org; Thu, 27 May 2010 18:01:52 -0400 Received: from mail.gnu.org ([199.232.76.166] helo=mx10.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OHkzc-00080M-Vk for submit@debbugs.gnu.org; Thu, 27 May 2010 17:52:28 -0400 Received: from lists.gnu.org ([199.232.76.165]:40994) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1OHkzH-0001L8-Ak for submit@debbugs.gnu.org; Thu, 27 May 2010 17:51:35 -0400 Received: from [140.186.70.92] (port=50248 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OHkzA-0003yl-UQ for bug-gnu-emacs@gnu.org; Thu, 27 May 2010 17:51:34 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID,T_TO_NO_BRKTS_FREEMAIL autolearn=unavailable version=3.3.1 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OHkwP-0005ZH-BD for bug-gnu-emacs@gnu.org; Thu, 27 May 2010 17:48:38 -0400 Received: from mail-pz0-f192.google.com ([209.85.222.192]:49856) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OHkwP-0005Yr-0W for bug-gnu-emacs@gnu.org; Thu, 27 May 2010 17:48:37 -0400 Received: by pzk30 with SMTP id 30so482472pzk.6 for ; Thu, 27 May 2010 14:48:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=T5f39lQThaa52X7eNFsVFybL9ekdae8udvfLAQ4H/wg=; b=X0mj/uzqhgUszNx2r159higVRV/3SX+zBkVW2GgsiNi7h5GJItwI+fdUPw9xXuHrnO Dl/70XXOP9FMyTzvWf/5jUbBcV0emq4Gzk7kVBfzYtX6R69w+96YuNBUjLLzfv2shSRW 4RbqVMLI0MtVARHHrsZmY78KjFRMcXG6KrrX8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=JGiegHUNH63wWMqo9yDLqnMn6s6CWAXolH/WdApiB2pJ4kPTlZP3nYZRxtqmatb7Hw kXnCLWDEGWcM0hLibMPoVF9Z6ngznVBBN2WV5Zbn/c6m4fjqe/+GmWkOwtTr93hkhsJu oKltgk3aJCP99Xpj+Sh9LIxXlF/FduJMoomPE= MIME-Version: 1.0 Received: by 10.140.252.3 with SMTP id z3mr8488319rvh.285.1274996913846; Thu, 27 May 2010 14:48:33 -0700 (PDT) Received: by 10.141.13.18 with HTTP; Thu, 27 May 2010 14:48:33 -0700 (PDT) Date: Thu, 27 May 2010 14:48:33 -0700 X-Google-Sender-Auth: Sw-ZSHMOklksq-gYU_iMmWebP9I Message-ID: From: Nick Ewing Content-Type: text/plain; charset=ISO-8859-1 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-Spam-Score: -3.3 (---) X-Mailman-Approved-At: Thu, 27 May 2010 18:01:50 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.6 (----) Ruby mode does not fully support general delimited strings or regular expressions in many cases. General delimited strings using parentheses, brackets, or curly braces are not supported at all by Ruby mode, therefore the following valid code is not correctly highlighted: %Q{This is a string} %w(foo bar) etc... This problem is most apparent when there is a single quote or a double quote within the general delimited string. In such a case, the quote starts highlighting everything following it as a string. Example: %Q{foo 'bar} Ruby mode does support general delimited strings while using a single character delimiter as such: %Q|This is a string| However, if there is a quote contained within this syntax, as follows, the highlighting breaks: %Q|foo 'bar| Also, general delimited strings are restricted to one line by Ruby mode, however Ruby allows them to be multiline. Therefore the following valid Ruby code is not highlighted by Ruby mode: %w| element1 element2 element3 | Finally, regular expressions do not work in all cases where they should. For example, the following Cucumber step definition (using valid Ruby syntax) will cause the regular expression contained in it to not be highlighted. Given /A user is logged in/ It seems that Ruby mode highlights Given as a constant, which is understandable, but fails to recognize the regular expression as valid when after a constant. I attempted to fix some of these problems myself, but was unable. Hopefully a more experienced emacs user can provide some help. Thank you! In GNU Emacs 23.2.1 (x86_64-apple-darwin, NS apple-appkit-1038.29) of 2010-05-08 on black.local Windowing system distributor `Apple', version 10.3.949 configured using `configure '--host=x86_64-apple-darwin' '--build=i686-apple-darwin' '--with-ns' 'build_alias=i686-apple-darwin' 'host_alias=x86_64-apple-darwin' 'CC=gcc -mmacosx-version-min=10.5'' Important settings: value of $LC_ALL: nil value of $LC_COLLATE: nil value of $LC_CTYPE: nil value of $LC_MESSAGES: nil value of $LC_MONETARY: nil value of $LC_NUMERIC: nil value of $LC_TIME: nil value of $LANG: nil value of $XMODIFIERS: nil locale-coding-system: nil default enable-multibyte-characters: t Major mode: Debugger Minor modes in effect: global-whitespace-mode: t autopair-mode: t autopair-global-mode: t show-paren-mode: t ido-everywhere: t textmate-mode: t global-hl-line-mode: t yas/global-mode: t nxhtml-menu-mode: t tooltip-mode: t mouse-wheel-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Recent input: x x x r e p o r t Recent messages: [yas] warning: won't overwrite keybinding "k e y b i n d i n g" for snippet "module" in `ruby-mode-map' [yas] warning: won't overwrite keybinding "k e y b i n d i n g" for snippet "if" in `ruby-mode-map' [yas] warning: won't overwrite keybinding "k e y b i n d i n g" for snippet "for" in `ruby-mode-map' [yas] warning: won't overwrite keybinding "k e y b i n d i n g" for snippet "do" in `ruby-mode-map' [yas] warning: won't overwrite keybinding "k e y b i n d i n g" for snippet "class" in `ruby-mode-map' Ido mode enabled [2 times] For information about GNU Emacs and the GNU system, type C-h C-a. Entering debugger... goto-history-element: Beginning of history; no preceding item [2 times] call-interactively: Text is read-only [2 times] Load-path shadows: /Users/nick/.emacs.d/vendor/nxhtml/related/php-mode hides ~/.emacs.d/vendor/php-mode ~/.emacs.d/vendor/whitespace hides /Applications/Emacs.app/Contents/Resources/lisp/whitespace /Users/nick/.emacs.d/vendor/cedet-1.0pre7/speedbar/speedbar hides /Applications/Emacs.app/Contents/Resources/lisp/speedbar /Users/nick/.emacs.d/vendor/cedet-1.0pre7/speedbar/sb-image hides /Applications/Emacs.app/Contents/Resources/lisp/sb-image ~/.emacs.d/vendor/linum hides /Applications/Emacs.app/Contents/Resources/lisp/linum /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/ezimage hides /Applications/Emacs.app/Contents/Resources/lisp/ezimage /Users/nick/.emacs.d/vendor/cedet-1.0pre7/speedbar/dframe hides /Applications/Emacs.app/Contents/Resources/lisp/dframe /Users/nick/.emacs.d/elpa/ruby-mode-1.1/ruby-mode hides /Applications/Emacs.app/Contents/Resources/lisp/progmodes/ruby-mode /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio-speedbar hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-speedbar /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio-opt hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-opt /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio-datadebug hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-datadebug /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio-custom hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-custom /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio-comp hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-comp /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio-base hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-base /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/chart hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/chart /Users/nick/.emacs.d/vendor/cedet-1.0pre7/srecode/srecode hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/srecode /Users/nick/.emacs.d/vendor/cedet-1.0pre7/semantic/semantic hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/semantic /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/pulse hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/pulse /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/mode-local hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/mode-local /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/inversion hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/inversion /Users/nick/.emacs.d/vendor/cedet-1.0pre7/ede/ede hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/ede /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/data-debug hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/data-debug /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/cedet hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/cedet /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/cedet-idutils hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/cedet-idutils /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/cedet-global hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/cedet-global /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/cedet-files hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/cedet-files /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/cedet-cscope hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/cedet-cscope Features: (shadow sort mail-extr message ecomplete rfc822 mml mml-sec password-cache mm-decode mm-bodies mm-encode mailcap mail-parse rfc2231 rfc2047 rfc2045 qp ietf-drums mailabbrev nnheader gnus-util netrc time-date mm-util mail-prsvr gmm-utils mailheader canlock sha1 hex-util hashcash mail-utils emacsbug cedet-edebug debug semantic-el semantic-bovine bovine-debug semantic-debug highlight-parentheses warnings erlang-start whitespace autopair paren cus-edit cus-start cus-load linum ido textmate hl-line yasnippet dropdown-list mumamo-fun mumamo nxhtml-autostart nxhtml-autoload majmodpri rinari easy-mmode jump inflections findr ruby-compilation which-func imenu pcomplete ansi-color inf-ruby ruby-mode nxhtml-menu web-autoload nxhtml-base ecb-layout-defs ecb ecb-symboldef ecb-analyse ecb-compatibility ecb-winman-support ecb-autogen autoload ecb-tod ecb-cycle ecb-eshell ecb-help ecb-jde ecb-method-browser hideshow ecb-file-browser ecb-layout compile comint ecb-create-layout ecb-compilation ecb-speedbar ecb-common-browser ecb-cedet-wrapper semanticdb-mode semanticdb-find semanticdb-ref ecb-navigate ecb-mode-line ecb-face tree-buffer ecb-upgrade ecb-util ring thingatpt cedet cedet-contrib-load cogre-load cogre-srecode semantic-edit srecode-template-mode srecode-template srecode-template-wy semantic-wisent wisent srecode-map srecode-dictionary speedbar-load sb-info ede-load ede-speedbar ede-files ede ede-base ede-auto eieio-speedbar semantic-ia-sb semantic-analyze semantic-scope semantic-analyze-fcn semantic-sort semanticdb-el semanticdb semantic-ctxt semantic-format semantic-util-modes semantic-util semantic semantic-lex semantic-tag working fame speedbar sb-image ezimage dframe assoc eieio-custom ede-source eieio-base srecode-load srecode semantic-load semantic-fw mode-local find-func derived eieio-load cedet-load cedet-compat eieio byte-opt bytecomp byte-compile inversion blank-mode-autoloads gist-autoloads rspec-mode-autoloads ruby-test-mode-autoloads sass-mode-autoloads advice help-fns advice-preload info wrap-region-autoloads yaml-mode-autoloads package color-theme edmacro kmacro wid-edit cl cl-19 sendmail regexp-opt reporter tooltip ediff-hook vc-hooks lisp-float-type mwheel ns-win easymenu tool-bar dnd fontset image fringe lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar mldrag mouse jit-lock font-lock syntax facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev loaddefs button minibuffer faces cus-face files text-properties overlay md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote make-network-process ns multi-tty emacs) From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 07 19:17:47 2012 Received: (at control) by debbugs.gnu.org; 8 Feb 2012 00:17:48 +0000 Received: from localhost ([127.0.0.1]:59611 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1RuvEI-0005dl-NO for submit@debbugs.gnu.org; Tue, 07 Feb 2012 19:17:47 -0500 Received: from fencepost.gnu.org ([140.186.70.10]:52770 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1RuvEF-0005dd-TP for control@debbugs.gnu.org; Tue, 07 Feb 2012 19:17:44 -0500 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1RuvDJ-0005Ay-Co for control@debbugs.gnu.org; Tue, 07 Feb 2012 19:16:45 -0500 Date: Tue, 07 Feb 2012 19:16:45 -0500 Message-Id: Subject: control message for bug 6286 To: X-Mailer: mail (GNU Mailutils 2.1) From: Glenn Morris X-Spam-Score: -4.2 (----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.2 (----) tag 6286 patch forwarded 6286 http://lists.gnu.org/archive/html/emacs-devel/2012-02/msg00247.html From unknown Thu Aug 14 18:40:31 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6286: General delimited literals in ruby-mode patch Resent-From: Stefan Monnier Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 24 Apr 2012 15:45:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6286 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch To: Dmitry Gutov Cc: 6286@debbugs.gnu.org Received: via spool by 6286-submit@debbugs.gnu.org id=B6286.13352822531918 (code B ref 6286); Tue, 24 Apr 2012 15:45:01 +0000 Received: (at 6286) by debbugs.gnu.org; 24 Apr 2012 15:44:13 +0000 Received: from localhost ([127.0.0.1]:50915 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SMhuX-0000Ut-1Q for submit@debbugs.gnu.org; Tue, 24 Apr 2012 11:44:13 -0400 Received: from pruche.dit.umontreal.ca ([132.204.246.22]:47589) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SMhuT-0000Ui-LS for 6286@debbugs.gnu.org; Tue, 24 Apr 2012 11:44:10 -0400 Received: from faina.iro.umontreal.ca (lechon.iro.umontreal.ca [132.204.27.242]) by pruche.dit.umontreal.ca (8.14.1/8.14.1) with ESMTP id q3OFhKmb025113; Tue, 24 Apr 2012 11:43:20 -0400 Received: by faina.iro.umontreal.ca (Postfix, from userid 20848) id 63609B4018; Tue, 24 Apr 2012 11:43:20 -0400 (EDT) From: Stefan Monnier Message-ID: References: <8739ammd8l.fsf@yandex.ru> <87k43vecyt.fsf@yandex.ru> Date: Tue, 24 Apr 2012 11:43:20 -0400 In-Reply-To: <87k43vecyt.fsf@yandex.ru> (Dmitry Gutov's message of "Fri, 10 Feb 2012 04:42:02 +0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.94 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-NAI-Spam-Flag: NO X-NAI-Spam-Level: X-NAI-Spam-Threshold: 5 X-NAI-Spam-Score: 0.1 X-NAI-Spam-Rules: 2 Rules triggered BODY_START_GREETING=0.1, RV4202=0 X-NAI-Spam-Version: 2.2.0.9309 : core <4202> : streams <749773> : uri <1103105> X-Spam-Score: -3.5 (---) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.5 (---) Hi, Dmitry, > I wrote a patch that fixes all examples except the last one (which is a > different issue), and also supports nesting delimiters of the same type > inside the literal: %r(//([^/])*/) Thank you very much. > Do I submit the final patch here, or to the Ruby Redmine tracker? Here is fine, thanks (tho sending it to the bug-report address rather than to emacs-devel would have been even better in this case). > As I understand, the upstream version is maintained to be compatible > with earlier Emacs versions, and I'm using the > syntax-propertize-function feature here. The maintainership is sadly unclear, indeed. If someone could try to bring the two versions closer (and keep them in sync) that would be very appreciated. Maybe it is enough to bring changes from the Ruby code to the Emacs code and not the other way around, but only if there's a clear understanding that the Ruby version is "deprecated" and won't see new development. Also, I'm not sure the Emacs version includes all the functionality of the current Ruby code, so there's some work to do. > Am I trying to be too clever with the syntax table in > `ruby-syntax-propertize-general-delimiters'? I figured it's the > easiest way to avoid reimplementing `scan-lists'. It doesn't strike me as too clever (actually it looks like the exact same trick I used in perl-mode). > In general, if I have an undercooked patch, should I post a message > here, or just attach it to the related bug? Attaching it to the bug report is better, in general, unless the patch ends up attacking a larger problem. > Why is the case of (fboundp #'syntax-propertize-rules) being nil still > being handled, by the way? I tried to preserve backward compatibility, to ease up merging the two versions. > This ruby-mode is not compatible with Emacs 23 either way. That might be: I have not verified that it is indeed backward compatible, I just tried to avoid consciously introducing incompatibilities. > - (or (not (eq ?/ c)) > - (null (nth 0 (ruby-parse-region (or begin parse-start) (point))))) > + ;; not a regexp or general delimited literal > + (null (nth 0 (ruby-parse-region (or begin parse-start) (point)))) Could you explain this part of your patch? Stefan From unknown Thu Aug 14 18:40:31 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6286: General delimited literals in ruby-mode patch Resent-From: Stefan Monnier Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 24 Apr 2012 17:11:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6286 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch To: 6286@debbugs.gnu.org Cc: Dmitry Gutov Received: via spool by 6286-submit@debbugs.gnu.org id=B6286.13352874149677 (code B ref 6286); Tue, 24 Apr 2012 17:11:01 +0000 Received: (at 6286) by debbugs.gnu.org; 24 Apr 2012 17:10:14 +0000 Received: from localhost ([127.0.0.1]:51017 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SMjFm-0002W1-1U for submit@debbugs.gnu.org; Tue, 24 Apr 2012 13:10:14 -0400 Received: from chene.dit.umontreal.ca ([132.204.246.20]:40458) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SMjFi-0002Vs-UA for 6286@debbugs.gnu.org; Tue, 24 Apr 2012 13:10:12 -0400 Received: from faina.iro.umontreal.ca (lechon.iro.umontreal.ca [132.204.27.242]) by chene.dit.umontreal.ca (8.14.1/8.14.1) with ESMTP id q3OH9LhX023591; Tue, 24 Apr 2012 13:09:21 -0400 Received: by faina.iro.umontreal.ca (Postfix, from userid 20848) id 3E39FB4018; Tue, 24 Apr 2012 13:09:21 -0400 (EDT) From: Stefan Monnier Message-ID: References: <8739ammd8l.fsf@yandex.ru> <87k43vecyt.fsf@yandex.ru> <87ehu3mga2.fsf@yandex.ru> Date: Tue, 24 Apr 2012 13:09:21 -0400 In-Reply-To: <87ehu3mga2.fsf@yandex.ru> (Dmitry Gutov's message of "Fri, 10 Feb 2012 09:03:17 +0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.94 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-NAI-Spam-Flag: NO X-NAI-Spam-Threshold: 5 X-NAI-Spam-Score: 0 X-NAI-Spam-Rules: 1 Rules triggered RV4202=0 X-NAI-Spam-Version: 2.2.0.9309 : core <4202> : streams <749788> : uri <1103128> X-Spam-Score: -3.5 (---) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.5 (---) I've installed your 2 patches, plus the patch below. Could you take a look at my patch? Here is what it does: - Fix up commenting conventions at random places. - Split large regexp into more manageable chunks. - During the split I saw that gsub/sub/split/scan were matched (for regexp) without regards to what precedes them, so "asub / a + bsub / b" was taken for a regexp. - I found a problem in your approach to handling Cucumber code. I don't know Ruby and don't use it (I looked up http://web.njit.edu/all_topics/Prog_Lang_Docs/html/ruby/syntax.html for help). BTW, is it really true that "%Q(hello (my) world)" is correct? That web-page doesn't clearly mention such nesting. Stefan === modified file 'lisp/ChangeLog' --- lisp/ChangeLog 2012-04-24 16:00:08 +0000 +++ lisp/ChangeLog 2012-04-24 16:35:47 +0000 @@ -1,3 +1,10 @@ +2012-04-24 Stefan Monnier + + * progmodes/ruby-mode.el: Simplify last change, and cleanup code. + (ruby-syntax-propertize-regexp): Remove. + (ruby-syntax-propertize-function): Split regexp into chunks. + Match following code directly. + 2012-04-24 Dmitry Gutov * progmodes/ruby-mode.el: Handle Cucumber defs (bug#6286). === modified file 'lisp/progmodes/ruby-mode.el' --- lisp/progmodes/ruby-mode.el 2012-04-24 16:00:08 +0000 +++ lisp/progmodes/ruby-mode.el 2012-04-24 16:42:22 +0000 @@ -784,7 +784,7 @@ (not (looking-at "[a-z_]")))) (and (looking-at ruby-operator-re) (not (ruby-special-char-p)) - ;; operator at the end of line + ;; Operator at the end of line. (let ((c (char-after (point)))) (and ;; (or (null begin) @@ -794,8 +794,9 @@ ;; (not (or (eolp) (looking-at "#") ;; (and (eq (car (nth 1 state)) ?{) ;; (looking-at "|")))))) - ;; not a regexp or general delimited literal - (null (nth 0 (ruby-parse-region (or begin parse-start) (point)))) + ;; Not a regexp or general delimited literal. + (null (nth 0 (ruby-parse-region (or begin parse-start) + (point)))) (or (not (eq ?| (char-after (point)))) (save-excursion (or (eolp) (forward-char -1)) @@ -1110,6 +1111,8 @@ mlist))))) (declare-function ruby-syntax-propertize-heredoc "ruby-mode" (limit)) +(declare-function ruby-syntax-general-delimiters-goto-beg "ruby-mode" ()) +(declare-function ruby-syntax-propertize-general-delimiters "ruby-mode" (limit)) (if (eval-when-compile (fboundp #'syntax-propertize-rules)) ;; New code that works independently from font-lock. @@ -1121,18 +1124,37 @@ (ruby-syntax-general-delimiters-goto-beg) (funcall (syntax-propertize-rules - ;; #{ }, #$hoge, #@foo are not comments + ;; #{ }, #$hoge, #@foo are not comments. ("\\(#\\)[{$@]" (1 ".")) - ;; $' $" $` .... are variables - ;; ?' ?" ?` are ascii codes + ;; $' $" $` .... are variables. + ;; ?' ?" ?` are ascii codes. ("\\([?$]\\)[#\"'`]" (1 (unless (save-excursion ;; Not within a string. (nth 3 (syntax-ppss (match-beginning 0)))) (string-to-syntax "\\")))) - ;; regexps - ("\\(^\\|[[=(,~?:;<>]\\|\\(?:^\\|\\s \\)\\(?:if\\|elsif\\|unless\\|while\\|until\\|when\\|and\\|or\\|&&\\|||\\)\\|g?sub!?\\|scan\\|split!?\\)?\\s *\\(/\\)[^/\n\\\\]*\\(?:\\\\.[^/\n\\\\]*\\)*\\(/\\)" - (2 (ruby-syntax-propertize-regexp))) + ;; Regexps: regexps are distinguished from division either because + ;; of the keyword/symbol before them, or because of the code + ;; following them. + ((concat + ;; Special tokens that can't be followed by a division operator. + "\\(?:\\(^\\|[[=(,~?:;<>]\\|\\(?:^\\|\\s \\)" + (regexp-opt '("if" "elsif" "unless" "while" "until" "when" "and" + "or" "&&" "||" + "gsub" "gsub!" "sub" "sub!" "scan" "split" "split!")) + "\\)\\s *\\)?" + ;; The regular expression itself. + "\\(/\\)[^/\n\\\\]*\\(?:\\\\.[^/\n\\\\]*\\)*\\(/\\)" + ;; Special code that cannot follow a division operator. + ;; FIXME: Just because the second slash of "/foo/ do bar" can't + ;; be a division, doesn't mean it can't *start* a regexp, as in + ;; "x = toto/foo; if /do bar/". + "\\([imxo]*\\s *\\(?:,\\|\\_\\)\\)?") + (2 (when (or (match-beginning 1) (match-beginning 4)) + (string-to-syntax "\"/"))) + (3 (if (or (match-beginning 1) (match-beginning 4)) + (string-to-syntax "\"/") + (goto-char (match-end 2))))) ("^=en\\(d\\)\\_>" (1 "!")) ("^\\(=\\)begin\\_>" (1 "!")) ;; Handle here documents. @@ -1143,21 +1165,6 @@ (1 (prog1 "|" (ruby-syntax-propertize-general-delimiters end))))) (point) end)) - (defun ruby-syntax-propertize-regexp () - (let ((syn (string-to-syntax "\"/"))) - (goto-char (match-end 3)) - (if (or - ;; after paren, comma, operator, control flow keyword, - ;; or a method from hardcoded list - (match-beginning 1) - ;; followed by comma or block - (looking-at "[imxo]*\\s *\\(?:,\\|\\\\)")) - (progn - (put-text-property (1- (point)) (point) - 'syntax-table syn) - syn) - (goto-char (match-end 2))))) - (defun ruby-syntax-propertize-heredoc (limit) (let ((ppss (syntax-ppss)) (res '())) @@ -1199,7 +1206,7 @@ parse-sexp-lookup-properties) (ignore-errors (if cl - (progn ; paired delimiters + (progn ; Paired delimiters. ;; Delimiter pairs of the same kind can be nested ;; inside the literal, as long as they are balanced. ;; Create syntax table that ignores other characters. @@ -1210,10 +1217,10 @@ (save-restriction (narrow-to-region (point) limit) (forward-list)))) ; skip to the paired character - ;; single character delimiter + ;; Single character delimiter. (re-search-forward (concat "[^\\]\\(?:\\\\\\\\\\)*" (regexp-quote ops)) limit nil)) - ;; if we reached here, the closing delimiter was found + ;; If we reached here, the closing delimiter was found. (put-text-property (1- (point)) (point) 'syntax-table (string-to-syntax "|"))))) ) @@ -1260,7 +1267,7 @@ (4 (7 . ?/)) (6 (7 . ?/))) ("^=en\\(d\\)\\_>" 1 "!") - ;; general delimited string + ;; General delimited string. ("\\(^\\|[[ \t\n<+(,=]\\)\\(%[xrqQwW]?\\([^<[{(a-zA-Z0-9 \n]\\)[^\n\\\\]*\\(\\\\.[^\n\\\\]*\\)*\\(\\3\\)\\)" (3 "\"") (5 "\"")) === modified file 'test/ChangeLog' --- test/ChangeLog 2012-04-11 03:24:26 +0000 +++ test/ChangeLog 2012-04-24 17:02:20 +0000 @@ -1,3 +1,7 @@ +2012-04-24 Stefan Monnier + + * indent/ruby.rb: New file, to test new syntax-propertize code. + 2012-04-11 Glenn Morris * automated/vc-bzr.el (vc-bzr-test-faulty-bzr-autoloads): New test. === added file 'test/indent/ruby.rb' --- test/indent/ruby.rb 1970-01-01 00:00:00 +0000 +++ test/indent/ruby.rb 2012-04-24 17:00:30 +0000 @@ -0,0 +1,19 @@ +# Don't mis-match "sub" at the end of words. +a = asub / aslb + bsub / bslb; + +b = %Q{This is a "string"} +c = %w(foo + bar + baz) +d = %!hello! + +# A "do" after a slash means that slash is not a division, but it doesn't imply +# it's a regexp-ender, since it can be a regexp-starter instead! +x = toto / foo; if /do bar/ then + toto = 1 + end + +# Some Cucumber code: +Given /toto/ do + print "hello" +end From unknown Thu Aug 14 18:40:31 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6286: General delimited literals in ruby-mode patch Resent-From: Dmitry Gutov Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 24 Apr 2012 23:48:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6286 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch To: Stefan Monnier Cc: 6286@debbugs.gnu.org Received: via spool by 6286-submit@debbugs.gnu.org id=B6286.133531127812792 (code B ref 6286); Tue, 24 Apr 2012 23:48:01 +0000 Received: (at 6286) by debbugs.gnu.org; 24 Apr 2012 23:47:58 +0000 Received: from localhost ([127.0.0.1]:51171 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SMpSf-0003KF-2c for submit@debbugs.gnu.org; Tue, 24 Apr 2012 19:47:58 -0400 Received: from forward19.mail.yandex.net ([95.108.253.144]:49095) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SMpSa-0003Jv-Tt for 6286@debbugs.gnu.org; Tue, 24 Apr 2012 19:47:55 -0400 Received: from smtp18.mail.yandex.net (smtp18.mail.yandex.net [95.108.252.18]) by forward19.mail.yandex.net (Yandex) with ESMTP id 6724B11217A7; Wed, 25 Apr 2012 03:46:56 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1335311216; bh=7WL8NlZlLQpyu4ov1x2tMzUeHPI5xfMtrH3czHcSltg=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=OlUthLRbtL9dMK15y6NmJpzyqg1774zwSPoj5J2Yj1FEnBC61d925NCY5eCBFJfmN woQKbqxvCtSIsIgJ07WFhVfaq1WSVT+xWASElUPg/s25nKEPox9N5OvUzXLDSGDnCQ iZ/PlGD5hbEQ1nPrd+bRIZw4osdwDuzdXM19u1rI= Received: from smtp18.mail.yandex.net (localhost [127.0.0.1]) by smtp18.mail.yandex.net (Yandex) with ESMTP id 466A418A02F8; Wed, 25 Apr 2012 03:46:56 +0400 (MSK) Received: from 98-87.nwlink.spb.ru (98-87.nwlink.spb.ru [178.252.98.87]) by smtp18.mail.yandex.net (nwsmtp/Yandex) with ESMTP id ktV444Fo-ktVGsB2U; Wed, 25 Apr 2012 03:46:56 +0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1335311216; bh=7WL8NlZlLQpyu4ov1x2tMzUeHPI5xfMtrH3czHcSltg=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=LXy8C+ODysBISZQSBc3kd+mzrtJ5+TqwegMJqApeV0Dn7rPw3UV7GtKq8sM9MXVRg 2UfzOW0XEXgYDGyp1YpbZq4ormwFl8arzUQvsjDmSaAKAC/jYMXWitAoSr2MdW3l3+ SbAPQ+vrND4QHn1T8lF6eCGOuLbojOo/m+FlJRP0= Message-ID: <4F973B73.1000808@yandex.ru> Date: Wed, 25 Apr 2012 03:46:59 +0400 From: Dmitry Gutov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0) Gecko/20120327 Thunderbird/11.0.1 MIME-Version: 1.0 References: <8739ammd8l.fsf@yandex.ru> <87k43vecyt.fsf@yandex.ru> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -1.9 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.9 (-) Hi Stefan, Thanks for the answers. On 24.04.2012 19:43, Stefan Monnier wrote: > The maintainership is sadly unclear, indeed. If someone could try to > bring the two versions closer (and keep them in sync) that would be very > appreciated. Maybe it is enough to bring changes from the Ruby code to > the Emacs code and not the other way around, but only if there's a clear > understanding that the Ruby version is "deprecated" and won't see new > development. Also, I'm not sure the Emacs version includes all the > functionality of the current Ruby code, so there's some work to do. They seem to have a ticket for this, but the comments are all in Japanese: http://bugs.ruby-lang.org/issues/5142 And Babelfish doesn't help much. If nobody beats me to it, I'm going to ask on the mailing list later. >> This ruby-mode is not compatible with Emacs 23 either way. > > That might be: I have not verified that it is indeed > backward compatible, I just tried to avoid consciously > introducing incompatibilities. To be precise, when you load it in 23.3, it complains about prog-mode's function definition being void. I guess that means we don't need to worry about maintaining the "else" branch when implementing something that requires `syntax-propertize-rules'? >> - (or (not (eq ?/ c)) >> - (null (nth 0 (ruby-parse-region (or begin parse-start) (point))))) >> + ;; not a regexp or general delimited literal >> + (null (nth 0 (ruby-parse-region (or begin parse-start) (point)))) > > Could you explain this part of your patch? That's a fix for indentation after percent literals delimited with operator characters: %r-abc-. But you seem to have already worked that out. > BTW, is it really true that "%Q(hello (my) world)" is correct? > That web-page doesn't clearly mention such nesting. Yes, it seems to be one of the more obscure features: irb(main):002:0> %Q(hello [(my]) world) => "hello [(my]) world" It's mentioned here: http://phrogz.net/programmingruby/language.html (to be continued) -- Dmitry From unknown Thu Aug 14 18:40:31 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6286: General delimited literals in ruby-mode patch Resent-From: Dmitry Gutov Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 25 Apr 2012 03:05:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6286 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch To: Stefan Monnier Cc: 6286@debbugs.gnu.org Received: via spool by 6286-submit@debbugs.gnu.org id=B6286.133532304629941 (code B ref 6286); Wed, 25 Apr 2012 03:05:01 +0000 Received: (at 6286) by debbugs.gnu.org; 25 Apr 2012 03:04:06 +0000 Received: from localhost ([127.0.0.1]:51254 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SMsWS-0007mp-HE for submit@debbugs.gnu.org; Tue, 24 Apr 2012 23:04:05 -0400 Received: from forward2.mail.yandex.net ([77.88.46.7]:33336) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SMsWN-0007lj-9W for 6286@debbugs.gnu.org; Tue, 24 Apr 2012 23:04:02 -0400 Received: from smtp1.mail.yandex.net (smtp1.mail.yandex.net [77.88.46.101]) by forward2.mail.yandex.net (Yandex) with ESMTP id D0C5612A175D; Wed, 25 Apr 2012 07:03:01 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1335322981; bh=NTPhww19X/PtVLGbOThJvpBfM460KwRQtXGmyMySxfo=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type; b=l085fD4Qpv50bD2W4OijLWEZgcMsU8C/kGUkAASt5JLYw4o+eI5XDQ1GiYWLnpuE8 1lVirCju8g07h+m3cpXlb7I/q7L151qZDjTC8noPkIbXgaFPF9AOBNbdB+MCg04gA+ zRFg+xrPA3yc+3fIWNgUtS3u8dUs3tglTa13BzYE= Received: from smtp1.mail.yandex.net (localhost [127.0.0.1]) by smtp1.mail.yandex.net (Yandex) with ESMTP id AAAECAA00B7; Wed, 25 Apr 2012 07:03:01 +0400 (MSK) Received: from 98-87.nwlink.spb.ru (98-87.nwlink.spb.ru [178.252.98.87]) by smtp1.mail.yandex.net (nwsmtp/Yandex) with ESMTP id 30R4J2Td-30RGtQ2K; Wed, 25 Apr 2012 07:03:01 +0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1335322981; bh=NTPhww19X/PtVLGbOThJvpBfM460KwRQtXGmyMySxfo=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type; b=CeYLEp+zFPtI/UAkYF/FfSM/0YYl+SjQTtFTSFf7ufR0fFP1czTboqvKfQBh2n1dr V1hTXcpnVgKANDbemq9MtX4yDBzrj3VgJ0nGH/hU/Q5Xc1HYUF7rnP0w0crwpTZ0hQ 6XDe7TxACaZI/9HlyH5q+dlIo9tEV9TjUvZwzf9Q= Message-ID: <4F976969.6050804@yandex.ru> Date: Wed, 25 Apr 2012 07:03:05 +0400 From: Dmitry Gutov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0) Gecko/20120327 Thunderbird/11.0.1 MIME-Version: 1.0 References: <8739ammd8l.fsf@yandex.ru> <87k43vecyt.fsf@yandex.ru> <87ehu3mga2.fsf@yandex.ru> In-Reply-To: Content-Type: multipart/mixed; boundary="------------010000040107060300060006" X-Spam-Score: -1.9 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.9 (-) This is a multi-part message in MIME format. --------------010000040107060300060006 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit So, the patch. On 24.04.2012 21:09, Stefan Monnier wrote: > Here is what it does: > - Split large regexp into more manageable chunks. I didn't think to use match groups this way. Very nice. > - During the split I saw that gsub/sub/split/scan were matched (for > regexp) without regards to what precedes them, so "asub / a + bsub / b" > was taken for a regexp. This fix has uncovered another problem: "gsub", "gsub!", "sub", "sub!", "scan", "split", and "split!" are not special tokens, those are all methods on class String: http://www.ruby-doc.org/core-1.9.3/String.html The original author just collected the methods most often used with regexps. And now this is broken: "abcdec".split /[be]/ One might argue that this isn't the most important use case, and that methods with arity > 1 are covered by the second rule (comma after), but 5 of these 7 methods can be called with just 1 argument. So that would mean backward incompatibility. > - I found a problem in your approach to handling Cucumber code. I'm assuming you mean this: x = toto / foo if /do bar/ =~ "dobar" # shortened version We can add a constraint that "do" is followed by (optionally) |a, d, c| (block arguments), and then EOL, since do ... end syntax isn't usually used with one-liner blocks, especially not after a regexp argument. Or we can revert the change and do it the original way. I looked into how other editors deal with regular expressions in Ruby. Vim is whitespace-sensitive. In the example above, the highlighting depends on whether you put space before "foo" (so it highlights one or the other regexp-looking expression). Textmate favors the whitelisting approach, like ruby-mode had pre-patch: http://www.ruby-forum.com/topic/170852 It has one benefit in that when you've typed the regexp, it's already highlighted, before you type the block keyword. Might feel more natural. In this approach, we'd move the "hardcoded" list of special method names to a variable, so that users might customize it, per project. What do you think? And here's a patch for another issue (attached). -- Dmitry --------------010000040107060300060006 Content-Type: text/plain; charset=windows-1251; name="0001-ruby-mode-Don-t-propertize-percent-literals-inside-s.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0001-ruby-mode-Don-t-propertize-percent-literals-inside-s.pa"; filename*1="tch" >From 05d10742e01cc3dceb4f465695daee7cc42215d6 Mon Sep 17 00:00:00 2001 From: Dmitry Gutov Date: Wed, 25 Apr 2012 06:55:50 +0400 Subject: [PATCH] ruby-mode: Don't propertize percent literals inside strings --- lisp/progmodes/ruby-mode.el | 58 ++++++++++++++++++++++++------------------- test/indent/ruby.rb | 3 +++ 2 files changed, 35 insertions(+), 26 deletions(-) diff --git a/lisp/progmodes/ruby-mode.el b/lisp/progmodes/ruby-mode.el index 5d79437..9ed3879 100644 --- a/lisp/progmodes/ruby-mode.el +++ b/lisp/progmodes/ruby-mode.el @@ -1162,7 +1162,7 @@ See `add-log-current-defun-function'." (7 (prog1 "\"" (ruby-syntax-propertize-heredoc end)))) ;; Handle percent literals: %w(), %q{}, etc. ("\\(?:^\\|[[ \t\n<+(,=]\\)\\(%\\)[qQrswWx]?\\([[:punct:]]\\)" - (1 (prog1 "|" (ruby-syntax-propertize-general-delimiters end))))) + (1 (ruby-syntax-propertize-general-delimiters end)))) (point) end)) (defun ruby-syntax-propertize-heredoc (limit) @@ -1198,31 +1198,37 @@ See `add-log-current-defun-function'." (beginning-of-line)))) (defun ruby-syntax-propertize-general-delimiters (limit) - (goto-char (match-beginning 2)) - (let* ((op (char-after)) - (ops (char-to-string op)) - (cl (or (cdr (aref (syntax-table) op)) - (cdr (assoc op '((?< . ?>)))))) - parse-sexp-lookup-properties) - (ignore-errors - (if cl - (progn ; Paired delimiters. - ;; Delimiter pairs of the same kind can be nested - ;; inside the literal, as long as they are balanced. - ;; Create syntax table that ignores other characters. - (with-syntax-table (make-char-table 'syntax-table nil) - (modify-syntax-entry op (concat "(" (char-to-string cl))) - (modify-syntax-entry cl (concat ")" ops)) - (modify-syntax-entry ?\\ "\\") - (save-restriction - (narrow-to-region (point) limit) - (forward-list)))) ; skip to the paired character - ;; Single character delimiter. - (re-search-forward (concat "[^\\]\\(?:\\\\\\\\\\)*" - (regexp-quote ops)) limit nil)) - ;; If we reached here, the closing delimiter was found. - (put-text-property (1- (point)) (point) - 'syntax-table (string-to-syntax "|"))))) + (goto-char (match-beginning 1)) ; When multiline, the beginning + (let ((state (syntax-ppss)) ; may already be propertized. + (syntax-value (string-to-syntax "|"))) + ;; Move forward either way, to escape inf loop. + (goto-char (match-beginning 2)) + (unless (nth 3 state) ; not inside a string + (let* ((op (char-after)) + (ops (char-to-string op)) + (cl (or (cdr (aref (syntax-table) op)) + (cdr (assoc op '((?< . ?>)))))) + parse-sexp-lookup-properties) + (ignore-errors + (if cl + (progn ; Paired delimiters. + ;; Delimiter pairs of the same kind can be nested + ;; inside the literal, as long as they are balanced. + ;; Create syntax table that ignores other characters. + (with-syntax-table (make-char-table 'syntax-table nil) + (modify-syntax-entry op (concat "(" (char-to-string cl))) + (modify-syntax-entry cl (concat ")" ops)) + (modify-syntax-entry ?\\ "\\") + (save-restriction + (narrow-to-region (point) limit) + (forward-list)))) ; skip to the paired character + ;; Single character delimiter. + (re-search-forward (concat "[^\\]\\(?:\\\\\\\\\\)*" + (regexp-quote ops)) limit nil)) + ;; If we reached here, the closing delimiter was found. + (put-text-property (1- (point)) (point) 'syntax-table + syntax-value))) + syntax-value))) ) ;; For Emacsen where syntax-propertize-rules is not (yet) available, diff --git a/test/indent/ruby.rb b/test/indent/ruby.rb index c4a747a..fe1986a 100644 --- a/test/indent/ruby.rb +++ b/test/indent/ruby.rb @@ -7,6 +7,9 @@ c = %w(foo baz) d = %!hello! +# Don't propertize percent literals inside strings. +"(%s, %s)" % [123, 456] + # A "do" after a slash means that slash is not a division, but it doesn't imply # it's a regexp-ender, since it can be a regexp-starter instead! x = toto / foo; if /do bar/ then -- 1.7.10.msysgit.1 --------------010000040107060300060006-- From unknown Thu Aug 14 18:40:31 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6286: General delimited literals in ruby-mode patch Resent-From: Stefan Monnier Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 25 Apr 2012 14:17:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6286 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch To: Dmitry Gutov Cc: 6286@debbugs.gnu.org Received: via spool by 6286-submit@debbugs.gnu.org id=B6286.13353633852036 (code B ref 6286); Wed, 25 Apr 2012 14:17:02 +0000 Received: (at 6286) by debbugs.gnu.org; 25 Apr 2012 14:16:25 +0000 Received: from localhost ([127.0.0.1]:52296 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SN316-0000Wm-DU for submit@debbugs.gnu.org; Wed, 25 Apr 2012 10:16:24 -0400 Received: from ironport-out.teksavvy.com ([206.248.143.162]:30595) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SN313-0000WX-JM for 6286@debbugs.gnu.org; Wed, 25 Apr 2012 10:16:22 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApYIACxOgk/O+LN8/2dsb2JhbABDuCMDgQyBCIIJAQEEAVYjBQsLDiYSFBgNJIgcBQu2J4thhHkEln2NSIFdgwM X-IronPort-AV: E=Sophos;i="4.75,391,1330923600"; d="scan'208";a="176620472" Received: from 206-248-179-124.dsl.teksavvy.com (HELO pastel.home) ([206.248.179.124]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 25 Apr 2012 10:15:22 -0400 Received: by pastel.home (Postfix, from userid 20848) id C916058FE4; Wed, 25 Apr 2012 10:15:21 -0400 (EDT) From: Stefan Monnier Message-ID: References: <8739ammd8l.fsf@yandex.ru> <87k43vecyt.fsf@yandex.ru> <4F973B73.1000808@yandex.ru> Date: Wed, 25 Apr 2012 10:15:21 -0400 In-Reply-To: <4F973B73.1000808@yandex.ru> (Dmitry Gutov's message of "Wed, 25 Apr 2012 03:46:59 +0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.94 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -1.9 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.9 (-) > To be precise, when you load it in 23.3, it complains about prog-mode's > function definition being void. Ah, that should be easy to fix. > I guess that means we don't need to worry about maintaining the "else" > branch when implementing something that requires `syntax-propertize-rules'? We don't have to improve that "else branch", no, but we do want to preserve its functionality. So, what you did in your patch (move the old font-lock-keywords pattern to the "else branch") was just right. >>> - (or (not (eq ?/ c)) >>> - (null (nth 0 (ruby-parse-region (or begin parse-start) (point))))) >>> + ;; not a regexp or general delimited literal >>> + (null (nth 0 (ruby-parse-region (or begin parse-start) (point)))) >> >> Could you explain this part of your patch? > That's a fix for indentation after percent literals delimited with operator > characters: %r-abc-. So could it be refined so as to check for a "%" char? I.e. instead of removing the old (not (eq ?/ c)), replace it with (not (memq c '(?/ ?%)))? > But you seem to have already worked that out. I just assumed the hunk was related to the rest of the patch ;-) >> BTW, is it really true that "%Q(hello (my) world)" is correct? >> That web-page doesn't clearly mention such nesting. > Yes, it seems to be one of the more obscure features: > irb(main):002:0> %Q(hello [(my]) world) > => "hello [(my]) world" > It's mentioned here: http://phrogz.net/programmingruby/language.html OK, good, thanks. >> - During the split I saw that gsub/sub/split/scan were matched (for >> regexp) without regards to what precedes them, so "asub / a + bsub / b" >> was taken for a regexp. > This fix has uncovered another problem: "gsub", "gsub!", "sub", "sub!", > "scan", "split", and "split!" are not special tokens, those are all methods > on class String: http://www.ruby-doc.org/core-1.9.3/String.html Aha! > The original author just collected the methods most often used with > regexps. And now this is broken: "abcdec".split /[be]/ Oops. > One might argue that this isn't the most important use case, and that > methods with arity > 1 are covered by the second rule (comma after), but > 5 of these 7 methods can be called with just 1 argument. So that would mean > backward incompatibility. And as we've seen the "check for a comma or a do-block afterwards" is not a reliable method. >> - I found a problem in your approach to handling Cucumber code. > I'm assuming you mean this: > x = toto / foo if /do bar/ =~ "dobar" # Shortened version. Yes. > We can add a constraint that "do" is followed by (optionally) |a, d, c| > (block arguments), and then EOL, since do ... end syntax isn't usually used > with one-liner blocks, especially not after a regexp argument. But that just reduces the likelihood of a problem without eliminating it: x = toto / foo if /do bar/ =~ "dobar" # Shortened version. still has the exact same problem. > I looked into how other editors deal with regular expressions in Ruby. > Vim is whitespace-sensitive. In the example above, the highlighting > depends on whether you put space before "foo" (so it highlights one or > the other regexp-looking expression). That sounds like a bad idea: if the / is a division, it's OK because you can easily decide to add/remove the space as needed, but if that / is for a regexp it's not as easy because adding/removing that space changes the regexp. Or is it also linked to the presence of a preceding space? That might not be so bad, then. E.g. " / " is division but "/ ", " /", and "/" is regexp. > Textmate favors the whitelisting approach, like ruby-mode had pre-patch: > http://www.ruby-forum.com/topic/170852 They mention a good point: since you can always use %r/.../ to make it clear you're using a regexp, it's better to err on the side of division when in doubt. > It has one benefit in that when you've typed the regexp, it's already > highlighted, before you type the block keyword. Might feel more natural. Yes, it's a nice side-advantage. > In this approach, we'd move the "hardcoded" list of special method names > to a variable, so that users might customize it, per project. > What do you think? Sounds good. > And here's a patch for another issue (attached). Regarding that patch, I think that you should do it differently: - nitpick: rather than goto-char+syntax-ppss, you can just do (syntax-ppss (match-beginning 1)). - leave the (prog1 "|" (ruby-syntax-propertize-general-delimiters end)) as is. - instead change ruby-syntax-propertize-general-delimiters so that it first checks syntax-ppss to make sure it's inside a general delimiter. This way, if the %Q appeared within a string (or a comment, BTW), you'll handle it right: even if you've put a "|" syntax on the character, that syntax has no effect on syntax-ppss if it appears within a string/comment. - Once you've done that, you can get rid of ruby-syntax-general-delimiters-goto-beg and call ruby-syntax-propertize-general-delimiters instead. You'll notice that ruby-syntax-propertize-heredoc follows that model. Stefan From unknown Thu Aug 14 18:40:31 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6286: General delimited literals in ruby-mode patch Resent-From: Dmitry Gutov Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 28 Apr 2012 20:22:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6286 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch To: Stefan Monnier Cc: 6286@debbugs.gnu.org Received: via spool by 6286-submit@debbugs.gnu.org id=B6286.133564451029125 (code B ref 6286); Sat, 28 Apr 2012 20:22:01 +0000 Received: (at 6286) by debbugs.gnu.org; 28 Apr 2012 20:21:50 +0000 Received: from localhost ([127.0.0.1]:57106 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SOE9M-0007Zf-Dj for submit@debbugs.gnu.org; Sat, 28 Apr 2012 16:21:49 -0400 Received: from forward11.mail.yandex.net ([95.108.130.93]:49674) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SOE9F-0007ZM-VP for 6286@debbugs.gnu.org; Sat, 28 Apr 2012 16:21:46 -0400 Received: from smtp14.mail.yandex.net (smtp14.mail.yandex.net [95.108.131.192]) by forward11.mail.yandex.net (Yandex) with ESMTP id B68CBE8128D; Sun, 29 Apr 2012 00:20:23 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1335644423; bh=Z+aq2hUGufxucNL+fFyDKgfzIz488A5rD88Kg8moPXk=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type; b=HrEq6VeQR9xDheJmHpEKJVRSVKNMLmwzsnvEPTHOf6hT10d0Yjf5Ovqi3uFVfDFHv uwwczXUig61VxBwWEFykNkdBTunuCPf9WUmpef593qkrwG9DQ+0x27WYTtTXR2iQ2Q 7vWkko8mEOOKyQoUkbP5vtMhus3BY5OiXl0rX2X0= Received: from smtp14.mail.yandex.net (localhost [127.0.0.1]) by smtp14.mail.yandex.net (Yandex) with ESMTP id 974D01B604FC; Sun, 29 Apr 2012 00:20:23 +0400 (MSK) Received: from 98-87.nwlink.spb.ru (98-87.nwlink.spb.ru [178.252.98.87]) by smtp14.mail.yandex.net (nwsmtp/Yandex) with ESMTP id KM2acHWC-KM20ghLN; Sun, 29 Apr 2012 00:20:22 +0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1335644423; bh=Z+aq2hUGufxucNL+fFyDKgfzIz488A5rD88Kg8moPXk=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type; b=oUtDhxLriCr4K9zZL1gz1O8svjIr23YISV7XePOmUe0ELOqXEONg7U0S584sGcv0K WkHsIO+f8w9uKUYAwZgJe0Zm8lGUCEELgZC7ZfLBSCub/UkbDRg9LVE5NjNkCcYvt2 AdeFi/fKkT/nmc/4osPrxoMY+XxiP8USbWUZHI/w= Message-ID: <4F9C5108.8090008@yandex.ru> Date: Sun, 29 Apr 2012 00:20:24 +0400 From: Dmitry Gutov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120420 Thunderbird/12.0 MIME-Version: 1.0 References: <8739ammd8l.fsf@yandex.ru> <87k43vecyt.fsf@yandex.ru> <4F973B73.1000808@yandex.ru> <4F981DEA.6060700@yandex.ru> In-Reply-To: Content-Type: multipart/mixed; boundary="------------000701090806070205070304" X-Spam-Score: -1.9 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.9 (-) This is a multi-part message in MIME format. --------------000701090806070205070304 Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit On 25.04.2012 22:00, Stefan Monnier wrote: >>> So could it be refined so as to check for a "%" char? I.e. instead of >>> removing the old (not (eq ?/ c)), replace it with (not (memq c '(?/ ?%)))? >> No, 'c' here is the char at the end of the previous line (see comment above >> "let"), and the closing delimiter can be any punctuation char. > > OK, then I don't understand what the code used to do. That's OK, I have > plenty of better things to do anyway ;-) AFAIU, it was just an optimization. '/' was the only binary operator to have double syntactic meaning. If c is not it, we definitely have continued expression, and can skip checking parse status. >>> x = toto / foo if /do >>> bar/ =~ "dobar" # Shortened version. >>> still has the exact same problem. >> Yes. But let's face it, seeing this code in the wild is not very likely. > > You might be right. > >> And it's as easy to change as // -> %r//. > > I'm not sure what change you're referring to: how should I change the > above code to make it unambiguous that the first / is a division rather > than a regexp? You would move the whole regexp to the next line, or (if the constraint I suggested is in place) replace the line break with "\n": x == toto / foo if /do\nbar/ =~ "dobar" Anyway, the second patch does away this this approach. >>> Or is it also linked to the presence of a preceding space? That might >>> not be so bad, then. E.g. " / " is division but "/ ", " /", and "/" >>> is regexp. >> The lack of preceding space can turn a regexp into division, but not the >> other way around. > > So you're saying that " / ", "/", and "/ " are for division, whereas " /" > is for regexps? Yes, though not without exceptions. "if / boo /" and "if/boo/" are both considered to contain regexps. As well as "/ boo /" when not after a word. The algorithm is likely more complex, I'm just describing how it works with specific examples. >> That's one more reason to do this just like TextMate, though. > > You mean because the problem can be fixed on the user's side without > changing the Ruby code? Yes, that's a clear advantage, especially when > browsing other people's code. If TextMate bundles are easily editable by user, I don't know about that (they might be). I just meant that having the same level of syntax support across editors is a good thing. So that if you have resort to %r//, it's the same for the other team members. And vice versa. I'm speaking mostly hypothetically, but the percent literals patch is in fact indirect result of my colleague using this syntax in TextMate, with unclosed paren inside. >> I'm not exactly sure why, but percent literal starting within a comment is >> already (not) propertized as expected. > > Not in all cases: > > x #= "tot %q/to"; = > /toto; > > now add&remove a space on the second line; then do the same on the > first line. > The approach I suggested is based on years of experience ;-) I wasn't arguing against the approach, just making an observation. :) But it's always good to have a counter-example. I did something close to what you described (patch 0001), but it didn't seem proper to call the same function in two different situations (before search and after search when we already have match data), so there's still two of them. --------------000701090806070205070304 Content-Type: text/plain; charset=windows-1251; name="0001-lisp-progmodes-ruby-mode.el-Don-t-propertize-percent.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0001-lisp-progmodes-ruby-mode.el-Don-t-propertize-percent.pa"; filename*1="tch" >>From bebf7f22ef746e1d20a5cdd4312684e02f0222f9 Mon Sep 17 00:00:00 2001 From: Dmitry Gutov Date: Sat, 28 Apr 2012 19:12:13 +0400 Subject: [PATCH 1/2] * lisp/progmodes/ruby-mode.el: Don't propertize percent literals in strings or comments (bug#6286) Change all names and references to "general delimited literals" to "percent literals", seems to be a shorter and more popular term. ruby-percent-literal-beg-re: New constant. (ruby-propertize-containing-percent-literal): Check the type of literal more carefully. Only propertize it but not other syntax above the point. (ruby-syntax-propertize-percent-literal): Only propertize when not inside basic string or comment. When the literal is unclosed, leave the text after it unpropertized. --- lisp/progmodes/ruby-mode.el | 84 ++++++++++++++++++++++++------------------- test/indent/ruby.rb | 7 ++++ 2 files changed, 54 insertions(+), 37 deletions(-) diff --git a/lisp/progmodes/ruby-mode.el b/lisp/progmodes/ruby-mode.el index 5d79437..432680f 100644 --- a/lisp/progmodes/ruby-mode.el +++ b/lisp/progmodes/ruby-mode.el @@ -794,7 +794,7 @@ and `\\' when preceded by `?'." ;; (not (or (eolp) (looking-at "#") ;; (and (eq (car (nth 1 state)) ?{) ;; (looking-at "|")))))) - ;; Not a regexp or general delimited literal. + ;; Not a regexp or percent literal. (null (nth 0 (ruby-parse-region (or begin parse-start) (point)))) (or (not (eq ?| (char-after (point)))) @@ -1111,17 +1111,21 @@ See `add-log-current-defun-function'." mlist))))) (declare-function ruby-syntax-propertize-heredoc "ruby-mode" (limit)) -(declare-function ruby-syntax-general-delimiters-goto-beg "ruby-mode" ()) -(declare-function ruby-syntax-propertize-general-delimiters "ruby-mode" (limit)) +(declare-function ruby-propertize-containing-percent-literal "ruby-mode" (limit)) +(declare-function ruby-syntax-propertize-percent-literal "ruby-mode" (limit)) (if (eval-when-compile (fboundp #'syntax-propertize-rules)) ;; New code that works independently from font-lock. (progn + (defconst ruby-percent-literal-beg-re + "\\(%\\)[qQrswWx]?\\([[:punct:]]\\)" + "Regexp to match the beginning of percent literal.") + (defun ruby-syntax-propertize-function (start end) "Syntactic keywords for Ruby mode. See `syntax-propertize-function'." (goto-char start) (ruby-syntax-propertize-heredoc end) - (ruby-syntax-general-delimiters-goto-beg) + (ruby-propertize-containing-percent-literal end) (funcall (syntax-propertize-rules ;; #{ }, #$hoge, #@foo are not comments. @@ -1161,8 +1165,8 @@ See `add-log-current-defun-function'." ((concat ruby-here-doc-beg-re ".*\\(\n\\)") (7 (prog1 "\"" (ruby-syntax-propertize-heredoc end)))) ;; Handle percent literals: %w(), %q{}, etc. - ("\\(?:^\\|[[ \t\n<+(,=]\\)\\(%\\)[qQrswWx]?\\([[:punct:]]\\)" - (1 (prog1 "|" (ruby-syntax-propertize-general-delimiters end))))) + ((concat "\\(?:^\\|[[ \t\n<+(,=]\\)" ruby-percent-literal-beg-re) + (1 (prog1 "|" (ruby-syntax-propertize-percent-literal end))))) (point) end)) (defun ruby-syntax-propertize-heredoc (limit) @@ -1189,40 +1193,46 @@ See `add-log-current-defun-function'." ;; inf-loop. (if (< (point) start) (goto-char start)))))) - (defun ruby-syntax-general-delimiters-goto-beg () - (let ((state (syntax-ppss))) - ;; Move to the start of the literal, in case it's multiline. - ;; TODO: determine the literal type more reliably here? + (defun ruby-propertize-containing-percent-literal (limit) + (let ((state (syntax-ppss)) + (start (point))) + ;; When already inside percent literal, re-propertize it. (when (eq t (nth 3 state)) (goto-char (nth 8 state)) - (beginning-of-line)))) + (when (looking-at ruby-percent-literal-beg-re) + (ruby-syntax-propertize-percent-literal limit)) + (when (< (point) start) (goto-char start))))) - (defun ruby-syntax-propertize-general-delimiters (limit) + (defun ruby-syntax-propertize-percent-literal (limit) (goto-char (match-beginning 2)) - (let* ((op (char-after)) - (ops (char-to-string op)) - (cl (or (cdr (aref (syntax-table) op)) - (cdr (assoc op '((?< . ?>)))))) - parse-sexp-lookup-properties) - (ignore-errors - (if cl - (progn ; Paired delimiters. - ;; Delimiter pairs of the same kind can be nested - ;; inside the literal, as long as they are balanced. - ;; Create syntax table that ignores other characters. - (with-syntax-table (make-char-table 'syntax-table nil) - (modify-syntax-entry op (concat "(" (char-to-string cl))) - (modify-syntax-entry cl (concat ")" ops)) - (modify-syntax-entry ?\\ "\\") - (save-restriction - (narrow-to-region (point) limit) - (forward-list)))) ; skip to the paired character - ;; Single character delimiter. - (re-search-forward (concat "[^\\]\\(?:\\\\\\\\\\)*" - (regexp-quote ops)) limit nil)) - ;; If we reached here, the closing delimiter was found. - (put-text-property (1- (point)) (point) - 'syntax-table (string-to-syntax "|"))))) + ;; Not inside a simple string or comment. + (when (eq t (nth 3 (syntax-ppss))) + (let* ((op (char-after)) + (ops (char-to-string op)) + (cl (or (cdr (aref (syntax-table) op)) + (cdr (assoc op '((?< . ?>)))))) + parse-sexp-lookup-properties) + (condition-case nil + (progn + (if cl ; Paired delimiters. + ;; Delimiter pairs of the same kind can be nested + ;; inside the literal, as long as they are balanced. + ;; Create syntax table that ignores other characters. + (with-syntax-table (make-char-table 'syntax-table nil) + (modify-syntax-entry op (concat "(" (char-to-string cl))) + (modify-syntax-entry cl (concat ")" ops)) + (modify-syntax-entry ?\\ "\\") + (save-restriction + (narrow-to-region (point) limit) + (forward-list))) ; skip to the paired character + ;; Single character delimiter. + (re-search-forward (concat "[^\\]\\(?:\\\\\\\\\\)*" + (regexp-quote ops)) limit nil)) + ;; Found the closing delimiter. + (put-text-property (1- (point)) (point) 'syntax-table + (string-to-syntax "|"))) + ;; Unclosed literal, leave the following text unpropertized. + ((scan-error search-failed) (goto-char limit)))))) ) ;; For Emacsen where syntax-propertize-rules is not (yet) available, @@ -1267,7 +1277,7 @@ This should only be called after matching against `ruby-here-doc-end-re'." (4 (7 . ?/)) (6 (7 . ?/))) ("^=en\\(d\\)\\_>" 1 "!") - ;; General delimited string. + ;; Percent literal. ("\\(^\\|[[ \t\n<+(,=]\\)\\(%[xrqQwW]?\\([^<[{(a-zA-Z0-9 \n]\\)[^\n\\\\]*\\(\\\\.[^\n\\\\]*\\)*\\(\\3\\)\\)" (3 "\"") (5 "\"")) diff --git a/test/indent/ruby.rb b/test/indent/ruby.rb index c4a747a..7a9d123 100644 --- a/test/indent/ruby.rb +++ b/test/indent/ruby.rb @@ -7,6 +7,13 @@ c = %w(foo baz) d = %!hello! +# Don't propertize percent literals inside strings. +"(%s, %s)" % [123, 456] + +# Nor inside comments. +x = # "tot %q/to"; = +y = 2 / 3 + # A "do" after a slash means that slash is not a division, but it doesn't imply # it's a regexp-ender, since it can be a regexp-starter instead! x = toto / foo; if /do bar/ then -- 1.7.10.msysgit.1 --------------000701090806070205070304 Content-Type: text/plain; charset=windows-1251; name="0002-lisp-progmodes-ruby-mode.el-Go-back-to-method-whitel.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0002-lisp-progmodes-ruby-mode.el-Go-back-to-method-whitel.pa"; filename*1="tch" >>From ac047bcebe08a8f850e3a3400156316088fd9520 Mon Sep 17 00:00:00 2001 From: Dmitry Gutov Date: Sat, 28 Apr 2012 22:11:00 +0400 Subject: [PATCH 2/2] * lisp/progmodes/ruby-mode.el: Go back to method whitelisting for regexps. ruby-syntax-methods-before-regexp: New variable. (ruby-syntax-propertize-function): Use it to recognize regexps. Don't look at the text after regexp, just use the old approach. * test/indent/ruby.rb: Update examples, add a new one. --- lisp/progmodes/ruby-mode.el | 38 ++++++++++++++++++++------------------ test/indent/ruby.rb | 25 +++++++++++++------------ 2 files changed, 33 insertions(+), 30 deletions(-) diff --git a/lisp/progmodes/ruby-mode.el b/lisp/progmodes/ruby-mode.el index 432680f..14ce846 100644 --- a/lisp/progmodes/ruby-mode.el +++ b/lisp/progmodes/ruby-mode.el @@ -226,6 +226,13 @@ Also ignores spaces after parenthesis when 'space." "Use `ruby-encoding-map' to set encoding magic comment if this is non-nil." :type 'boolean :group 'ruby) +(defcustom ruby-syntax-methods-before-regexp + '("gsub" "gsub!" "sub" "sub!" "scan" "split" "split!" "index" "match" + "assert_match" "Given" "Then" "When") + "Methods that can take regexp as the first argument. +It will be properly highlighted even when the call omits parens." + :group 'ruby) + ;; Safe file variables (put 'ruby-indent-tabs-mode 'safe-local-variable 'booleanp) (put 'ruby-indent-level 'safe-local-variable 'integerp) @@ -1137,28 +1144,23 @@ See `add-log-current-defun-function'." ;; Not within a string. (nth 3 (syntax-ppss (match-beginning 0)))) (string-to-syntax "\\")))) - ;; Regexps: regexps are distinguished from division either because - ;; of the keyword/symbol before them, or because of the code - ;; following them. + ;; Regexps: regexps are distinguished from division because + ;; of the keyword, symbol, or method name before them. ((concat ;; Special tokens that can't be followed by a division operator. - "\\(?:\\(^\\|[[=(,~?:;<>]\\|\\(?:^\\|\\s \\)" + "\\(^\\|[[=(,~?:;<>]" + ;; Control flow keywords and operators following bol or whitespace. + "\\|\\(?:^\\|\\s \\)" (regexp-opt '("if" "elsif" "unless" "while" "until" "when" "and" - "or" "&&" "||" - "gsub" "gsub!" "sub" "sub!" "scan" "split" "split!")) - "\\)\\s *\\)?" + "or" "not" "&&" "||")) + ;; Method name from the list. + "\\|\\_<" + (regexp-opt ruby-syntax-methods-before-regexp) + "\\)\\s *" ;; The regular expression itself. - "\\(/\\)[^/\n\\\\]*\\(?:\\\\.[^/\n\\\\]*\\)*\\(/\\)" - ;; Special code that cannot follow a division operator. - ;; FIXME: Just because the second slash of "/foo/ do bar" can't - ;; be a division, doesn't mean it can't *start* a regexp, as in - ;; "x = toto/foo; if /do bar/". - "\\([imxo]*\\s *\\(?:,\\|\\_\\)\\)?") - (2 (when (or (match-beginning 1) (match-beginning 4)) - (string-to-syntax "\"/"))) - (3 (if (or (match-beginning 1) (match-beginning 4)) - (string-to-syntax "\"/") - (goto-char (match-end 2))))) + "\\(/\\)[^/\n\\\\]*\\(?:\\\\.[^/\n\\\\]*\\)*\\(/\\)") + (2 (string-to-syntax "\"/")) + (3 (string-to-syntax "\"/"))) ("^=en\\(d\\)\\_>" (1 "!")) ("^\\(=\\)begin\\_>" (1 "!")) ;; Handle here documents. diff --git a/test/indent/ruby.rb b/test/indent/ruby.rb index 7a9d123..4f2e9e6 100644 --- a/test/indent/ruby.rb +++ b/test/indent/ruby.rb @@ -1,24 +1,25 @@ -# Don't mis-match "sub" at the end of words. -a = asub / aslb + bsub / bslb; - +# Percent literals. b = %Q{This is a "string"} -c = %w(foo +c = %w!foo bar - baz) -d = %!hello! + baz! +d = %(hello (nested) world) # Don't propertize percent literals inside strings. "(%s, %s)" % [123, 456] -# Nor inside comments. +# Or inside comments. x = # "tot %q/to"; = y = 2 / 3 -# A "do" after a slash means that slash is not a division, but it doesn't imply -# it's a regexp-ender, since it can be a regexp-starter instead! -x = toto / foo; if /do bar/ then - toto = 1 - end +# Regexp after whitelisted method. +"abc".sub /b/, 'd' + +# Don't mis-match "sub" at the end of words. +a = asub / aslb + bsub / bslb; + +# Highlight the regexp after "if". +x = toto / foo if /do bar/ =~ "dobar" # Some Cucumber code: Given /toto/ do -- 1.7.10.msysgit.1 --------------000701090806070205070304-- From unknown Thu Aug 14 18:40:31 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6286: General delimited literals in ruby-mode patch Resent-From: Dmitry Gutov Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 03 May 2012 05:42:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6286 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch To: Stefan Monnier Cc: 6286@debbugs.gnu.org Received: via spool by 6286-submit@debbugs.gnu.org id=B6286.133602369610922 (code B ref 6286); Thu, 03 May 2012 05:42:02 +0000 Received: (at 6286) by debbugs.gnu.org; 3 May 2012 05:41:36 +0000 Received: from localhost ([127.0.0.1]:34395 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SPonH-0002q6-Ng for submit@debbugs.gnu.org; Thu, 03 May 2012 01:41:36 -0400 Received: from forward20.mail.yandex.net ([95.108.253.145]:45761) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SPonE-0002pm-Au for 6286@debbugs.gnu.org; Thu, 03 May 2012 01:41:34 -0400 Received: from smtp19.mail.yandex.net (smtp19.mail.yandex.net [95.108.252.19]) by forward20.mail.yandex.net (Yandex) with ESMTP id 44019104179C; Thu, 3 May 2012 09:39:49 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1336023589; bh=XiMn5mDjUSQB5yvFum/COm0hiYDqI1bwloQessfFqDs=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type; b=B2XctxpQdSQ15diW01/kH5PxnTaagmCfqeMwpMo0r4+zQUSnfgPuhGvMs6MwX95aR tVSRkdkidZqv2Q1vm+yqn/arL+EWi/dOaMh+Dza5Ls66e+QkBxaMx7VZmcpJuopkQB d8MXnAjVcxdciIff1aYYdfbFcq8d03lrnJRB5VFQ= Received: from smtp19.mail.yandex.net (localhost [127.0.0.1]) by smtp19.mail.yandex.net (Yandex) with ESMTP id 203BEBE0193; Thu, 3 May 2012 09:39:49 +0400 (MSK) Received: from 98-87.nwlink.spb.ru (98-87.nwlink.spb.ru [178.252.98.87]) by smtp19.mail.yandex.net (nwsmtp/Yandex) with ESMTP id daRmE2Lb-daRCmWIh; Thu, 3 May 2012 09:39:36 +0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1336023589; bh=XiMn5mDjUSQB5yvFum/COm0hiYDqI1bwloQessfFqDs=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type; b=FKiXfzy0XNOr91iSipVE+OPp7lKJsDjD1Jn1sKw7pX34AOkX1eMsRSlCLS3kyEORJ Dh+j6z2FtQjmXokC85ZSqpvPMXfofb/FhLIw26bwNmAPaH6KbdQSDQKLGWl8J2TFsv qeDyWBOk8xGuovLw/ME+yU0krawDX+htjQLFOJBY= Message-ID: <4FA21A1C.8040907@yandex.ru> Date: Thu, 03 May 2012 09:39:40 +0400 From: Dmitry Gutov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 References: <8739ammd8l.fsf@yandex.ru> <87k43vecyt.fsf@yandex.ru> <4F973B73.1000808@yandex.ru> <4F981DEA.6060700@yandex.ru> In-Reply-To: Content-Type: multipart/mixed; boundary="------------030107000209060203020408" X-Spam-Score: -1.9 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.9 (-) This is a multi-part message in MIME format. --------------030107000209060203020408 Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit Hi again, I missed the byte compilation error, arising from the fact that `syntax-propertize-rules` evaluates regular expressions at compile time. Options: 1) Remove customizability (see attached patch). 2) Regexp matches any method, then we check against the whitelist in a new function. -- Dmitry --------------030107000209060203020408 Content-Type: text/plain; charset=windows-1251; name="0003-lisp-progmodes-ruby-mode.el-Fix-byte-compilation-err.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0003-lisp-progmodes-ruby-mode.el-Fix-byte-compilation-err.pa"; filename*1="tch" >From bb9ffe72bcb874aec06ca06b62576af8eca5f9f2 Mon Sep 17 00:00:00 2001 From: Dmitry Gutov Date: Thu, 3 May 2012 09:11:20 +0400 Subject: [PATCH 3/3] * lisp/progmodes/ruby-mode.el: Fix byte compilation error ruby-syntax-methods-before-regexp: make constant ruby-syntax-methods-before-regexp, ruby-percent-literal-beg-re: wrap together in (eval-and-compile) --- lisp/progmodes/ruby-mode.el | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/lisp/progmodes/ruby-mode.el b/lisp/progmodes/ruby-mode.el index 14ce846..15c8246 100644 --- a/lisp/progmodes/ruby-mode.el +++ b/lisp/progmodes/ruby-mode.el @@ -226,13 +226,6 @@ Also ignores spaces after parenthesis when 'space." "Use `ruby-encoding-map' to set encoding magic comment if this is non-nil." :type 'boolean :group 'ruby) -(defcustom ruby-syntax-methods-before-regexp - '("gsub" "gsub!" "sub" "sub!" "scan" "split" "split!" "index" "match" - "assert_match" "Given" "Then" "When") - "Methods that can take regexp as the first argument. -It will be properly highlighted even when the call omits parens." - :group 'ruby) - ;; Safe file variables (put 'ruby-indent-tabs-mode 'safe-local-variable 'booleanp) (put 'ruby-indent-level 'safe-local-variable 'integerp) @@ -1124,9 +1117,16 @@ See `add-log-current-defun-function'." (if (eval-when-compile (fboundp #'syntax-propertize-rules)) ;; New code that works independently from font-lock. (progn - (defconst ruby-percent-literal-beg-re - "\\(%\\)[qQrswWx]?\\([[:punct:]]\\)" - "Regexp to match the beginning of percent literal.") + (eval-and-compile + (defconst ruby-percent-literal-beg-re + "\\(%\\)[qQrswWx]?\\([[:punct:]]\\)" + "Regexp to match the beginning of percent literal.") + + (defconst ruby-syntax-methods-before-regexp + '("gsub" "gsub!" "sub" "sub!" "scan" "split" "split!" "index" "match" + "assert_match" "Given" "Then" "When") + "Methods that can take regexp as the first argument. +It will be properly highlighted even when the call omits parens.")) (defun ruby-syntax-propertize-function (start end) "Syntactic keywords for Ruby mode. See `syntax-propertize-function'." -- 1.7.10.msysgit.1 --------------030107000209060203020408-- From unknown Thu Aug 14 18:40:31 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6286: General delimited literals in ruby-mode patch Resent-From: Dmitry Gutov Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 14 Aug 2012 04:06:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6286 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch To: Stefan Monnier Cc: 6286@debbugs.gnu.org Received: via spool by 6286-submit@debbugs.gnu.org id=B6286.13449171235738 (code B ref 6286); Tue, 14 Aug 2012 04:06:01 +0000 Received: (at 6286) by debbugs.gnu.org; 14 Aug 2012 04:05:23 +0000 Received: from localhost ([127.0.0.1]:54634 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T18Ne-0001UT-CD for submit@debbugs.gnu.org; Tue, 14 Aug 2012 00:05:23 -0400 Received: from forward3.mail.yandex.net ([77.88.46.8]:34314) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T18NZ-0001UI-A4 for 6286@debbugs.gnu.org; Tue, 14 Aug 2012 00:05:20 -0400 Received: from smtp2.mail.yandex.net (smtp2.mail.yandex.net [77.88.46.102]) by forward3.mail.yandex.net (Yandex) with ESMTP id 88C6EB40557; Tue, 14 Aug 2012 07:56:38 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1344916600; bh=0BOLvQAcJou/x+mKe1A827a9QUNsZaF3EVHZtDIYn1s=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type; b=wJ57jR9q3AkKTezAfu0fvb5pYdrcTW3vvHhH/5lsexRD8xPzqiz1/4Bdp6LIjpIZV 99ak7HL7qEhXjYj7fBYViQrTelLHeLRiBi91mlA5zJG3Ti9IA5AMZB6F4bf6VicubL f2cX4Qi4JQbOMMpWEVLDqKNo/w71CJ0jtP07BpmA= Received: from smtp2.mail.yandex.net (localhost [127.0.0.1]) by smtp2.mail.yandex.net (Yandex) with ESMTP id 668ACE205C4; Tue, 14 Aug 2012 07:56:38 +0400 (MSK) Received: from dynamicip-188-187-113-24.pppoe.volgograd.ertelecom.ru (dynamicip-188-187-113-24.pppoe.volgograd.ertelecom.ru [188.187.113.24]) by smtp2.mail.yandex.net (nwsmtp/Yandex) with ESMTP id ubJepRk4-ubJ4PPRG; Tue, 14 Aug 2012 07:56:37 +0400 X-Yandex-Rcpt-Suid: monnier@iro.umontreal.ca X-Yandex-Rcpt-Suid: 6286@debbugs.gnu.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1344916598; bh=0BOLvQAcJou/x+mKe1A827a9QUNsZaF3EVHZtDIYn1s=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type; b=OtvooF48msxIeCe99It/HAoJe3D980VbR15XJ9RW6Q+OXHxX2M3jewP6g3F5l2ew6 kCCO/HkeCFDIlgPUfpBJtlhgl5vio73MtxNPL3oV81j71tV35iwfHKF2a9g7FmBj3Z I4u7RPvWsjqzYq0fUl8oIt4Vpn+aotPZ7N0wjPxk= Message-ID: <5029CC7A.4060001@yandex.ru> Date: Tue, 14 Aug 2012 07:56:42 +0400 From: Dmitry Gutov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 References: <8739ammd8l.fsf@yandex.ru> <87k43vecyt.fsf@yandex.ru> <4F973B73.1000808@yandex.ru> <4F981DEA.6060700@yandex.ru> <4FA21A1C.8040907@yandex.ru> In-Reply-To: <4FA21A1C.8040907@yandex.ru> Content-Type: multipart/mixed; boundary="------------060300020203050609040106" X-Spam-Score: -2.6 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.6 (--) This is a multi-part message in MIME format. --------------060300020203050609040106 Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit Here are the 3 previously submitted patches as 2 files, slightly reworded, and with proper ChangeLog entries. I think the first one is quite important, and I've had a user emailing me about it. The second one deals with a fairly narrow edge case, so it's kinda optional. --Dmitry --------------060300020203050609040106 Content-Type: text/plain; charset=windows-1251; name="01-6286-percent-literals.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="01-6286-percent-literals.diff" diff --git a/lisp/ChangeLog b/lisp/ChangeLog index cedf854..4f52796 100644 --- a/lisp/ChangeLog +++ b/lisp/ChangeLog @@ -1,3 +1,15 @@ +2012-08-14 Dmitry Gutov + + * progmodes/ruby-mode.el (ruby-percent-literal-beg-re): New constant. + (ruby-syntax-general-delimiters-goto-beg): Rename to + `ruby-syntax-enclosing-percent-literal', improve literal type check. + (ruby-syntax-propertize-general-delimiters): Rename to + `ruby-syntax-propertize-percent-literal', it's a shorter and more + popular term. Adjust comments everywhere. + (ruby-syntax-propertize-percent-literal): Only propertize when not + inside a simple string or comment. When the literal is unclosed, + leave the text after it unpropertized. + 2012-08-13 Andreas Schwab * files.el (hack-local-variables-filter): Remove useless eval. diff --git a/lisp/progmodes/ruby-mode.el b/lisp/progmodes/ruby-mode.el index 744dd43..42e1ac7 100644 --- a/lisp/progmodes/ruby-mode.el +++ b/lisp/progmodes/ruby-mode.el @@ -800,7 +800,7 @@ and `\\' when preceded by `?'." ;; (not (or (eolp) (looking-at "#") ;; (and (eq (car (nth 1 state)) ?{) ;; (looking-at "|")))))) - ;; Not a regexp or general delimited literal. + ;; Not a regexp or percent literal. (null (nth 0 (ruby-parse-region (or begin parse-start) (point)))) (or (not (eq ?| (char-after (point)))) @@ -1169,17 +1169,22 @@ See `add-log-current-defun-function'." (ruby-do-end-to-brace))) (declare-function ruby-syntax-propertize-heredoc "ruby-mode" (limit)) -(declare-function ruby-syntax-general-delimiters-goto-beg "ruby-mode" ()) -(declare-function ruby-syntax-propertize-general-delimiters "ruby-mode" (limit)) +(declare-function ruby-syntax-enclosing-percent-literal "ruby-mode" (limit)) +(declare-function ruby-syntax-propertize-percent-literal "ruby-mode" (limit)) (if (eval-when-compile (fboundp #'syntax-propertize-rules)) ;; New code that works independently from font-lock. (progn + (eval-and-compile + (defconst ruby-percent-literal-beg-re + "\\(%\\)[qQrswWx]?\\([[:punct:]]\\)" + "Regexp to match the beginning of percent literal.")) + (defun ruby-syntax-propertize-function (start end) "Syntactic keywords for Ruby mode. See `syntax-propertize-function'." (goto-char start) (ruby-syntax-propertize-heredoc end) - (ruby-syntax-general-delimiters-goto-beg) + (ruby-syntax-enclosing-percent-literal end) (funcall (syntax-propertize-rules ;; #{ }, #$hoge, #@foo are not comments. @@ -1222,8 +1227,8 @@ See `add-log-current-defun-function'." 'syntax-table (string-to-syntax "\"")) (ruby-syntax-propertize-heredoc end)))) ;; Handle percent literals: %w(), %q{}, etc. - ("\\(?:^\\|[[ \t\n<+(,=]\\)\\(%\\)[qQrswWx]?\\([[:punct:]]\\)" - (1 (prog1 "|" (ruby-syntax-propertize-general-delimiters end))))) + ((concat "\\(?:^\\|[[ \t\n<+(,=]\\)" ruby-percent-literal-beg-re) + (1 (prog1 "|" (ruby-syntax-propertize-percent-literal end))))) (point) end)) (defun ruby-syntax-propertize-heredoc (limit) @@ -1251,40 +1256,46 @@ See `add-log-current-defun-function'." ;; inf-loop. (if (< (point) start) (goto-char start)))))) - (defun ruby-syntax-general-delimiters-goto-beg () - (let ((state (syntax-ppss))) - ;; Move to the start of the literal, in case it's multiline. - ;; TODO: determine the literal type more reliably here? + (defun ruby-syntax-enclosing-percent-literal (limit) + (let ((state (syntax-ppss)) + (start (point))) + ;; When already inside percent literal, re-propertize it. (when (eq t (nth 3 state)) (goto-char (nth 8 state)) - (beginning-of-line)))) + (when (looking-at ruby-percent-literal-beg-re) + (ruby-syntax-propertize-percent-literal limit)) + (when (< (point) start) (goto-char start))))) - (defun ruby-syntax-propertize-general-delimiters (limit) + (defun ruby-syntax-propertize-percent-literal (limit) (goto-char (match-beginning 2)) - (let* ((op (char-after)) - (ops (char-to-string op)) - (cl (or (cdr (aref (syntax-table) op)) - (cdr (assoc op '((?< . ?>)))))) - parse-sexp-lookup-properties) - (ignore-errors - (if cl - (progn ; Paired delimiters. - ;; Delimiter pairs of the same kind can be nested - ;; inside the literal, as long as they are balanced. - ;; Create syntax table that ignores other characters. - (with-syntax-table (make-char-table 'syntax-table nil) - (modify-syntax-entry op (concat "(" (char-to-string cl))) - (modify-syntax-entry cl (concat ")" ops)) - (modify-syntax-entry ?\\ "\\") - (save-restriction - (narrow-to-region (point) limit) - (forward-list)))) ; skip to the paired character - ;; Single character delimiter. - (re-search-forward (concat "[^\\]\\(?:\\\\\\\\\\)*" - (regexp-quote ops)) limit nil)) - ;; If we reached here, the closing delimiter was found. - (put-text-property (1- (point)) (point) - 'syntax-table (string-to-syntax "|"))))) + ;; Not inside a simple string or comment. + (when (eq t (nth 3 (syntax-ppss))) + (let* ((op (char-after)) + (ops (char-to-string op)) + (cl (or (cdr (aref (syntax-table) op)) + (cdr (assoc op '((?< . ?>)))))) + parse-sexp-lookup-properties) + (condition-case nil + (progn + (if cl ; Paired delimiters. + ;; Delimiter pairs of the same kind can be nested + ;; inside the literal, as long as they are balanced. + ;; Create syntax table that ignores other characters. + (with-syntax-table (make-char-table 'syntax-table nil) + (modify-syntax-entry op (concat "(" (char-to-string cl))) + (modify-syntax-entry cl (concat ")" ops)) + (modify-syntax-entry ?\\ "\\") + (save-restriction + (narrow-to-region (point) limit) + (forward-list))) ; skip to the paired character + ;; Single character delimiter. + (re-search-forward (concat "[^\\]\\(?:\\\\\\\\\\)*" + (regexp-quote ops)) limit nil)) + ;; Found the closing delimiter. + (put-text-property (1- (point)) (point) 'syntax-table + (string-to-syntax "|"))) + ;; Unclosed literal, leave the following text unpropertized. + ((scan-error search-failed) (goto-char limit)))))) ) ;; For Emacsen where syntax-propertize-rules is not (yet) available, @@ -1329,7 +1340,7 @@ This should only be called after matching against `ruby-here-doc-end-re'." (4 (7 . ?/)) (6 (7 . ?/))) ("^=en\\(d\\)\\_>" 1 "!") - ;; General delimited string. + ;; Percent literal. ("\\(^\\|[[ \t\n<+(,=]\\)\\(%[xrqQwW]?\\([^<[{(a-zA-Z0-9 \n]\\)[^\n\\\\]*\\(\\\\.[^\n\\\\]*\\)*\\(\\3\\)\\)" (3 "\"") (5 "\"")) diff --git a/test/ChangeLog b/test/ChangeLog index a0163b5..9dbca3d 100644 --- a/test/ChangeLog +++ b/test/ChangeLog @@ -1,3 +1,7 @@ +2012-08-14 Dmitry Gutov + + * indent/ruby.rb: New examples. + 2012-08-12 Dmitry Gutov * automated/ruby-mode-tests.el (ruby-move-to-block-stops-at-opening) diff --git a/test/indent/ruby.rb b/test/indent/ruby.rb index c4a747a..7a9d123 100644 --- a/test/indent/ruby.rb +++ b/test/indent/ruby.rb @@ -7,6 +7,13 @@ c = %w(foo baz) d = %!hello! +# Don't propertize percent literals inside strings. +"(%s, %s)" % [123, 456] + +# Nor inside comments. +x = # "tot %q/to"; = +y = 2 / 3 + # A "do" after a slash means that slash is not a division, but it doesn't imply # it's a regexp-ender, since it can be a regexp-starter instead! x = toto / foo; if /do bar/ then --------------060300020203050609040106 Content-Type: text/plain; charset=windows-1251; name="02-6286-regexps.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="02-6286-regexps.diff" diff --git a/lisp/ChangeLog b/lisp/ChangeLog index 4f52796..f37c346 100644 --- a/lisp/ChangeLog +++ b/lisp/ChangeLog @@ -9,6 +9,9 @@ (ruby-syntax-propertize-percent-literal): Only propertize when not inside a simple string or comment. When the literal is unclosed, leave the text after it unpropertized. + (ruby-syntax-methods-before-regexp): New constant. + (ruby-syntax-propertize-function): Use it to recognize regexps. + Don't look at the text after regexp, just use the whitelist. 2012-08-13 Andreas Schwab diff --git a/lisp/progmodes/ruby-mode.el b/lisp/progmodes/ruby-mode.el index 42e1ac7..457c7fe 100644 --- a/lisp/progmodes/ruby-mode.el +++ b/lisp/progmodes/ruby-mode.el @@ -1178,7 +1178,13 @@ See `add-log-current-defun-function'." (eval-and-compile (defconst ruby-percent-literal-beg-re "\\(%\\)[qQrswWx]?\\([[:punct:]]\\)" - "Regexp to match the beginning of percent literal.")) + "Regexp to match the beginning of percent literal.") + + (defconst ruby-syntax-methods-before-regexp + '("gsub" "gsub!" "sub" "sub!" "scan" "split" "split!" "index" "match" + "assert_match" "Given" "Then" "When") + "Methods that can take regexp as the first argument. +It will be properly highlighted even when the call omits parens.")) (defun ruby-syntax-propertize-function (start end) "Syntactic keywords for Ruby mode. See `syntax-propertize-function'." @@ -1196,28 +1202,23 @@ See `add-log-current-defun-function'." ;; Not within a string. (nth 3 (syntax-ppss (match-beginning 0)))) (string-to-syntax "\\")))) - ;; Regexps: regexps are distinguished from division either because - ;; of the keyword/symbol before them, or because of the code - ;; following them. + ;; Regexps: regexps are distinguished from division because + ;; of the keyword, symbol, or method name before them. ((concat ;; Special tokens that can't be followed by a division operator. - "\\(?:\\(^\\|[[=(,~?:;<>]\\|\\(?:^\\|\\s \\)" + "\\(^\\|[[=(,~?:;<>]" + ;; Control flow keywords and operators following bol or whitespace. + "\\|\\(?:^\\|\\s \\)" (regexp-opt '("if" "elsif" "unless" "while" "until" "when" "and" - "or" "&&" "||" - "gsub" "gsub!" "sub" "sub!" "scan" "split" "split!")) - "\\)\\s *\\)?" + "or" "not" "&&" "||")) + ;; Method name from the list. + "\\|\\_<" + (regexp-opt ruby-syntax-methods-before-regexp) + "\\)\\s *" ;; The regular expression itself. - "\\(/\\)[^/\n\\\\]*\\(?:\\\\.[^/\n\\\\]*\\)*\\(/\\)" - ;; Special code that cannot follow a division operator. - ;; FIXME: Just because the second slash of "/foo/ do bar" can't - ;; be a division, doesn't mean it can't *start* a regexp, as in - ;; "x = toto/foo; if /do bar/". - "\\([imxo]*\\s *\\(?:,\\|\\_\\)\\)?") - (2 (when (or (match-beginning 1) (match-beginning 4)) - (string-to-syntax "\"/"))) - (3 (if (or (match-beginning 1) (match-beginning 4)) - (string-to-syntax "\"/") - (goto-char (match-end 2))))) + "\\(/\\)[^/\n\\\\]*\\(?:\\\\.[^/\n\\\\]*\\)*\\(/\\)") + (2 (string-to-syntax "\"/")) + (3 (string-to-syntax "\"/"))) ("^=en\\(d\\)\\_>" (1 "!")) ("^\\(=\\)begin\\_>" (1 "!")) ;; Handle here documents. diff --git a/test/ChangeLog b/test/ChangeLog index 9dbca3d..f1bf458 100644 --- a/test/ChangeLog +++ b/test/ChangeLog @@ -1,6 +1,6 @@ 2012-08-14 Dmitry Gutov - * indent/ruby.rb: New examples. + * indent/ruby.rb: Rearrange examples, add new ones. 2012-08-12 Dmitry Gutov diff --git a/test/indent/ruby.rb b/test/indent/ruby.rb index 7a9d123..4f2e9e6 100644 --- a/test/indent/ruby.rb +++ b/test/indent/ruby.rb @@ -1,24 +1,25 @@ -# Don't mis-match "sub" at the end of words. -a = asub / aslb + bsub / bslb; - +# Percent literals. b = %Q{This is a "string"} -c = %w(foo +c = %w!foo bar - baz) -d = %!hello! + baz! +d = %(hello (nested) world) # Don't propertize percent literals inside strings. "(%s, %s)" % [123, 456] -# Nor inside comments. +# Or inside comments. x = # "tot %q/to"; = y = 2 / 3 -# A "do" after a slash means that slash is not a division, but it doesn't imply -# it's a regexp-ender, since it can be a regexp-starter instead! -x = toto / foo; if /do bar/ then - toto = 1 - end +# Regexp after whitelisted method. +"abc".sub /b/, 'd' + +# Don't mis-match "sub" at the end of words. +a = asub / aslb + bsub / bslb; + +# Highlight the regexp after "if". +x = toto / foo if /do bar/ =~ "dobar" # Some Cucumber code: Given /toto/ do --------------060300020203050609040106-- From unknown Thu Aug 14 18:40:31 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6286: General delimited literals in ruby-mode patch Resent-From: Stefan Monnier Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 14 Aug 2012 12:49:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6286 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch To: Dmitry Gutov Cc: 6286@debbugs.gnu.org Received: via spool by 6286-submit@debbugs.gnu.org id=B6286.134494852611250 (code B ref 6286); Tue, 14 Aug 2012 12:49:02 +0000 Received: (at 6286) by debbugs.gnu.org; 14 Aug 2012 12:48:46 +0000 Received: from localhost ([127.0.0.1]:55406 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T1GYA-0002vP-0l for submit@debbugs.gnu.org; Tue, 14 Aug 2012 08:48:46 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:64178) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T1GY7-0002vE-Ii for 6286@debbugs.gnu.org; Tue, 14 Aug 2012 08:48:44 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av0EAG6Zu09MCqqR/2dsb2JhbABEtBGBCIIVAQEEAVYjBQsLDh8HEhQYDSQTiAADBgW6CY0agyoDozOBWIMFgUOBOA X-IronPort-AV: E=Sophos;i="4.75,637,1330923600"; d="scan'208";a="195646720" Received: from 76-10-170-145.dsl.teksavvy.com (HELO fmsmemgm.homelinux.net) ([76.10.170.145]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 14 Aug 2012 08:40:04 -0400 Received: by fmsmemgm.homelinux.net (Postfix, from userid 20848) id 4E25FAE2F6; Tue, 14 Aug 2012 08:40:04 -0400 (EDT) From: Stefan Monnier Message-ID: References: <8739ammd8l.fsf@yandex.ru> <87k43vecyt.fsf@yandex.ru> <4F973B73.1000808@yandex.ru> <4F981DEA.6060700@yandex.ru> <4FA21A1C.8040907@yandex.ru> <5029CC7A.4060001@yandex.ru> Date: Tue, 14 Aug 2012 08:40:04 -0400 In-Reply-To: <5029CC7A.4060001@yandex.ru> (Dmitry Gutov's message of "Tue, 14 Aug 2012 07:56:42 +0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -1.9 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.9 (-) > I think the first one is quite important, and I've had a user emailing me > about it. Thanks, installed. > The second one deals with a fairly narrow edge case, so it's kinda optional. Thanks, installed. I kind of remember something about this second case, and remember being bothered by the fact that ruby-syntax-methods-before-regexp would need to be adjusted by the user since it can depend on its locale (especially the Given/Then/When). I also vaguely remember that this problem was related to the reason why I added the "look after the regexp" test. It would probably be better if you could commit those changes yourself. If you want to do that, please request membership in the "emacs" group from your savannah account (which you may have to create beforehand). Stefan > diff --git a/lisp/ChangeLog b/lisp/ChangeLog > index 4f52796..f37c346 100644 > --- a/lisp/ChangeLog > +++ b/lisp/ChangeLog > @@ -9,6 +9,9 @@ > (ruby-syntax-propertize-percent-literal): Only propertize when not > inside a simple string or comment. When the literal is unclosed, > leave the text after it unpropertized. > + (ruby-syntax-methods-before-regexp): New constant. > + (ruby-syntax-propertize-function): Use it to recognize regexps. > + Don't look at the text after regexp, just use the whitelist. > 2012-08-13 Andreas Schwab > diff --git a/lisp/progmodes/ruby-mode.el b/lisp/progmodes/ruby-mode.el > index 42e1ac7..457c7fe 100644 > --- a/lisp/progmodes/ruby-mode.el > +++ b/lisp/progmodes/ruby-mode.el > @@ -1178,7 +1178,13 @@ See `add-log-current-defun-function'." > (eval-and-compile > (defconst ruby-percent-literal-beg-re > "\\(%\\)[qQrswWx]?\\([[:punct:]]\\)" > - "Regexp to match the beginning of percent literal.")) > + "Regexp to match the beginning of percent literal.") > + > + (defconst ruby-syntax-methods-before-regexp > + '("gsub" "gsub!" "sub" "sub!" "scan" "split" "split!" "index" "match" > + "assert_match" "Given" "Then" "When") > + "Methods that can take regexp as the first argument. > +It will be properly highlighted even when the call omits parens.")) > (defun ruby-syntax-propertize-function (start end) > "Syntactic keywords for Ruby mode. See `syntax-propertize-function'." > @@ -1196,28 +1202,23 @@ See `add-log-current-defun-function'." > ;; Not within a string. > (nth 3 (syntax-ppss (match-beginning 0)))) > (string-to-syntax "\\")))) > - ;; Regexps: regexps are distinguished from division either because > - ;; of the keyword/symbol before them, or because of the code > - ;; following them. > + ;; Regexps: regexps are distinguished from division because > + ;; of the keyword, symbol, or method name before them. > ((concat > ;; Special tokens that can't be followed by a division operator. > - "\\(?:\\(^\\|[[=(,~?:;<>]\\|\\(?:^\\|\\s \\)" > + "\\(^\\|[[=(,~?:;<>]" > + ;; Control flow keywords and operators following bol or whitespace. > + "\\|\\(?:^\\|\\s \\)" > (regexp-opt '("if" "elsif" "unless" "while" "until" "when" "and" > - "or" "&&" "||" > - "gsub" "gsub!" "sub" "sub!" "scan" "split" "split!")) > - "\\)\\s *\\)?" > + "or" "not" "&&" "||")) > + ;; Method name from the list. > + "\\|\\_<" > + (regexp-opt ruby-syntax-methods-before-regexp) > + "\\)\\s *" > ;; The regular expression itself. > - "\\(/\\)[^/\n\\\\]*\\(?:\\\\.[^/\n\\\\]*\\)*\\(/\\)" > - ;; Special code that cannot follow a division operator. > - ;; FIXME: Just because the second slash of "/foo/ do bar" can't > - ;; be a division, doesn't mean it can't *start* a regexp, as in > - ;; "x = toto/foo; if /do bar/". > - "\\([imxo]*\\s *\\(?:,\\|\\_\\)\\)?") > - (2 (when (or (match-beginning 1) (match-beginning 4)) > - (string-to-syntax "\"/"))) > - (3 (if (or (match-beginning 1) (match-beginning 4)) > - (string-to-syntax "\"/") > - (goto-char (match-end 2))))) > + "\\(/\\)[^/\n\\\\]*\\(?:\\\\.[^/\n\\\\]*\\)*\\(/\\)") > + (2 (string-to-syntax "\"/")) > + (3 (string-to-syntax "\"/"))) > ("^=en\\(d\\)\\_>" (1 "!")) > ("^\\(=\\)begin\\_>" (1 "!")) > ;; Handle here documents. > diff --git a/test/ChangeLog b/test/ChangeLog > index 9dbca3d..f1bf458 100644 > --- a/test/ChangeLog > +++ b/test/ChangeLog > @@ -1,6 +1,6 @@ > 2012-08-14 Dmitry Gutov > - * indent/ruby.rb: New examples. > + * indent/ruby.rb: Rearrange examples, add new ones. > 2012-08-12 Dmitry Gutov > diff --git a/test/indent/ruby.rb b/test/indent/ruby.rb > index 7a9d123..4f2e9e6 100644 > --- a/test/indent/ruby.rb > +++ b/test/indent/ruby.rb > @@ -1,24 +1,25 @@ > -# Don't mis-match "sub" at the end of words. > -a = asub / aslb + bsub / bslb; > - > +# Percent literals. > b = %Q{This is a "string"} > -c = %w(foo > +c = %w!foo > bar > - baz) > -d = %!hello! > + baz! > +d = %(hello (nested) world) > # Don't propertize percent literals inside strings. > "(%s, %s)" % [123, 456] > -# Nor inside comments. > +# Or inside comments. > x = # "tot %q/to"; = > y = 2 / 3 > -# A "do" after a slash means that slash is not a division, but it doesn't imply > -# it's a regexp-ender, since it can be a regexp-starter instead! > -x = toto / foo; if /do bar/ then > - toto = 1 > - end > +# Regexp after whitelisted method. > +"abc".sub /b/, 'd' > + > +# Don't mis-match "sub" at the end of words. > +a = asub / aslb + bsub / bslb; > + > +# Highlight the regexp after "if". > +x = toto / foo if /do bar/ =~ "dobar" > # Some Cucumber code: > Given /toto/ do From unknown Thu Aug 14 18:40:31 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6286: General delimited literals in ruby-mode patch Resent-From: Dmitry Gutov Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 14 Aug 2012 17:56:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6286 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch To: Stefan Monnier Cc: 6286@debbugs.gnu.org Received: via spool by 6286-submit@debbugs.gnu.org id=B6286.134496693224141 (code B ref 6286); Tue, 14 Aug 2012 17:56:01 +0000 Received: (at 6286) by debbugs.gnu.org; 14 Aug 2012 17:55:32 +0000 Received: from localhost ([127.0.0.1]:56762 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T1LL1-0006HK-Fh for submit@debbugs.gnu.org; Tue, 14 Aug 2012 13:55:32 -0400 Received: from forward10.mail.yandex.net ([77.88.61.49]:42423) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T1LKy-0006HB-OU for 6286@debbugs.gnu.org; Tue, 14 Aug 2012 13:55:30 -0400 Received: from smtp8.mail.yandex.net (smtp8.mail.yandex.net [77.88.61.54]) by forward10.mail.yandex.net (Yandex) with ESMTP id 6E9121022326; Tue, 14 Aug 2012 21:46:48 +0400 (MSK) Received: from smtp8.mail.yandex.net (localhost [127.0.0.1]) by smtp8.mail.yandex.net (Yandex) with ESMTP id 4C2A91B60788; Tue, 14 Aug 2012 21:46:48 +0400 (MSK) Received: from dynamicip-188-187-120-34.pppoe.volgograd.ertelecom.ru (dynamicip-188-187-120-34.pppoe.volgograd.ertelecom.ru [188.187.120.34]) by smtp8.mail.yandex.net (nwsmtp/Yandex) with ESMTP id kl2qufbJ-kl2eofZS; Tue, 14 Aug 2012 21:46:48 +0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1344966408; bh=qReu7/2WGT4AForPt7PtzFcJPUt+qNj0ilBcGXnb40c=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=QtIgLY2W7qdgLAbwbS0TEZ+a3CumrgVdfZHmQTI43zuZPHvb8aCt0m+a2xRo+1a2b g3iTfUe2SQcpZIgMaHdskPca2ePNnyP6vPsv92IulJob3EExSvIRW3wMbY2BxhhLRo 9gwB9WtT9e3P8ZvQQsovcCUucFhG0EGUr1foG5B8= Message-ID: <502A8F08.8010406@yandex.ru> Date: Tue, 14 Aug 2012 21:46:48 +0400 From: Dmitry Gutov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 References: <8739ammd8l.fsf@yandex.ru> <87k43vecyt.fsf@yandex.ru> <4F973B73.1000808@yandex.ru> <4F981DEA.6060700@yandex.ru> <4FA21A1C.8040907@yandex.ru> <5029CC7A.4060001@yandex.ru> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -2.6 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.6 (--) On 14.08.2012 16:40, Stefan Monnier wrote: > I kind of remember something about this second case, and remember being > bothered by the fact that ruby-syntax-methods-before-regexp would need > to be adjusted by the user since it can depend on its locale > (especially the Given/Then/When). While Cucumber features can be localized, ruby-mode should only be used with step definitions [1], and those just contain Ruby code, so the Given/Then/When method names would be the same, AFAIK. A user might want to be able to add a new method to the list, theoretically, but paren-less calling convention is mostly used in DSL-like code (that means, with methods provided by some framework or other), so I say we wait until such feature request actually comes. > I also vaguely remember that this problem was related to the reason why > I added the "look after the regexp" test. That was in the original patch I submitted, mostly because it looked like a clever solution. But the conservative approach should work, too: the canonical (I think) TextMate bundle uses the whitelist [2] without major complaints. > It would probably be better if you could commit those changes yourself. > If you want to do that, please request membership in the "emacs" group > from your savannah account (which you may have to create beforehand). Requested, thanks. I think we're finished with this bug, though, unless you'd like to see some other changes. --Dmitry [1] https://github.com/cucumber/cucumber/wiki/Step-Definitions [2] https://github.com/drnic/ruby-tmbundle/blob/master/Syntaxes/Ruby.plist#L882 From unknown Thu Aug 14 18:40:31 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.428 (Entity 5.428) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Nick Ewing Subject: bug#6286: closed (Re: General delimited literals in ruby-mode patch) Message-ID: References: X-Gnu-PR-Message: they-closed 6286 X-Gnu-PR-Package: emacs X-Gnu-PR-Keywords: patch Reply-To: 6286@debbugs.gnu.org Date: Wed, 15 Aug 2012 02:43:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1344998582-15552-1" This is a multi-part message in MIME format... ------------=_1344998582-15552-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #6286: Ruby Mode Missing Syntax which was filed against the emacs package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 6286@debbugs.gnu.org. --=20 6286: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D6286 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1344998582-15552-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 6286-done) by debbugs.gnu.org; 15 Aug 2012 02:42:39 +0000 Received: from localhost ([127.0.0.1]:57353 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T1TZ9-00042H-Cp for submit@debbugs.gnu.org; Tue, 14 Aug 2012 22:42:39 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:49735) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1T1TZ7-000428-EB for 6286-done@debbugs.gnu.org; Tue, 14 Aug 2012 22:42:37 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av0EAG6Zu09MCqqR/2dsb2JhbABEtBGBCIIVAQEEAVYjBQsLDiYSFBgNJIgcBboJkEQDozOBWIMF X-IronPort-AV: E=Sophos;i="4.75,637,1330923600"; d="scan'208";a="195708140" Received: from 76-10-170-145.dsl.teksavvy.com (HELO fmsmemgm.homelinux.net) ([76.10.170.145]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 14 Aug 2012 22:33:55 -0400 Received: by fmsmemgm.homelinux.net (Postfix, from userid 20848) id 2552BAE2F6; Tue, 14 Aug 2012 22:33:55 -0400 (EDT) From: Stefan Monnier To: Dmitry Gutov Subject: Re: General delimited literals in ruby-mode patch Message-ID: References: <8739ammd8l.fsf@yandex.ru> <87k43vecyt.fsf@yandex.ru> <4F973B73.1000808@yandex.ru> <4F981DEA.6060700@yandex.ru> <4FA21A1C.8040907@yandex.ru> <5029CC7A.4060001@yandex.ru> <502A8F08.8010406@yandex.ru> Date: Tue, 14 Aug 2012 22:33:55 -0400 In-Reply-To: <502A8F08.8010406@yandex.ru> (Dmitry Gutov's message of "Tue, 14 Aug 2012 21:46:48 +0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -1.9 (-) X-Debbugs-Envelope-To: 6286-done Cc: 6286-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -1.9 (-) > I think we're finished with this bug, though, unless you'd like to see some > other changes. Indeed, thanks, Stefan ------------=_1344998582-15552-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 27 May 2010 22:01:53 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OHl9D-00085D-V2 for submit@debbugs.gnu.org; Thu, 27 May 2010 18:01:52 -0400 Received: from mail.gnu.org ([199.232.76.166] helo=mx10.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OHkzc-00080M-Vk for submit@debbugs.gnu.org; Thu, 27 May 2010 17:52:28 -0400 Received: from lists.gnu.org ([199.232.76.165]:40994) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1OHkzH-0001L8-Ak for submit@debbugs.gnu.org; Thu, 27 May 2010 17:51:35 -0400 Received: from [140.186.70.92] (port=50248 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OHkzA-0003yl-UQ for bug-gnu-emacs@gnu.org; Thu, 27 May 2010 17:51:34 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID,T_TO_NO_BRKTS_FREEMAIL autolearn=unavailable version=3.3.1 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OHkwP-0005ZH-BD for bug-gnu-emacs@gnu.org; Thu, 27 May 2010 17:48:38 -0400 Received: from mail-pz0-f192.google.com ([209.85.222.192]:49856) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OHkwP-0005Yr-0W for bug-gnu-emacs@gnu.org; Thu, 27 May 2010 17:48:37 -0400 Received: by pzk30 with SMTP id 30so482472pzk.6 for ; Thu, 27 May 2010 14:48:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=T5f39lQThaa52X7eNFsVFybL9ekdae8udvfLAQ4H/wg=; b=X0mj/uzqhgUszNx2r159higVRV/3SX+zBkVW2GgsiNi7h5GJItwI+fdUPw9xXuHrnO Dl/70XXOP9FMyTzvWf/5jUbBcV0emq4Gzk7kVBfzYtX6R69w+96YuNBUjLLzfv2shSRW 4RbqVMLI0MtVARHHrsZmY78KjFRMcXG6KrrX8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=JGiegHUNH63wWMqo9yDLqnMn6s6CWAXolH/WdApiB2pJ4kPTlZP3nYZRxtqmatb7Hw kXnCLWDEGWcM0hLibMPoVF9Z6ngznVBBN2WV5Zbn/c6m4fjqe/+GmWkOwtTr93hkhsJu oKltgk3aJCP99Xpj+Sh9LIxXlF/FduJMoomPE= MIME-Version: 1.0 Received: by 10.140.252.3 with SMTP id z3mr8488319rvh.285.1274996913846; Thu, 27 May 2010 14:48:33 -0700 (PDT) Received: by 10.141.13.18 with HTTP; Thu, 27 May 2010 14:48:33 -0700 (PDT) Date: Thu, 27 May 2010 14:48:33 -0700 X-Google-Sender-Auth: Sw-ZSHMOklksq-gYU_iMmWebP9I Message-ID: Subject: Ruby Mode Missing Syntax From: Nick Ewing To: bug-gnu-emacs@gnu.org Content-Type: text/plain; charset=ISO-8859-1 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-Spam-Score: -3.3 (---) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Thu, 27 May 2010 18:01:50 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.6 (----) Ruby mode does not fully support general delimited strings or regular expressions in many cases. General delimited strings using parentheses, brackets, or curly braces are not supported at all by Ruby mode, therefore the following valid code is not correctly highlighted: %Q{This is a string} %w(foo bar) etc... This problem is most apparent when there is a single quote or a double quote within the general delimited string. In such a case, the quote starts highlighting everything following it as a string. Example: %Q{foo 'bar} Ruby mode does support general delimited strings while using a single character delimiter as such: %Q|This is a string| However, if there is a quote contained within this syntax, as follows, the highlighting breaks: %Q|foo 'bar| Also, general delimited strings are restricted to one line by Ruby mode, however Ruby allows them to be multiline. Therefore the following valid Ruby code is not highlighted by Ruby mode: %w| element1 element2 element3 | Finally, regular expressions do not work in all cases where they should. For example, the following Cucumber step definition (using valid Ruby syntax) will cause the regular expression contained in it to not be highlighted. Given /A user is logged in/ It seems that Ruby mode highlights Given as a constant, which is understandable, but fails to recognize the regular expression as valid when after a constant. I attempted to fix some of these problems myself, but was unable. Hopefully a more experienced emacs user can provide some help. Thank you! In GNU Emacs 23.2.1 (x86_64-apple-darwin, NS apple-appkit-1038.29) of 2010-05-08 on black.local Windowing system distributor `Apple', version 10.3.949 configured using `configure '--host=x86_64-apple-darwin' '--build=i686-apple-darwin' '--with-ns' 'build_alias=i686-apple-darwin' 'host_alias=x86_64-apple-darwin' 'CC=gcc -mmacosx-version-min=10.5'' Important settings: value of $LC_ALL: nil value of $LC_COLLATE: nil value of $LC_CTYPE: nil value of $LC_MESSAGES: nil value of $LC_MONETARY: nil value of $LC_NUMERIC: nil value of $LC_TIME: nil value of $LANG: nil value of $XMODIFIERS: nil locale-coding-system: nil default enable-multibyte-characters: t Major mode: Debugger Minor modes in effect: global-whitespace-mode: t autopair-mode: t autopair-global-mode: t show-paren-mode: t ido-everywhere: t textmate-mode: t global-hl-line-mode: t yas/global-mode: t nxhtml-menu-mode: t tooltip-mode: t mouse-wheel-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Recent input: x x x r e p o r t Recent messages: [yas] warning: won't overwrite keybinding "k e y b i n d i n g" for snippet "module" in `ruby-mode-map' [yas] warning: won't overwrite keybinding "k e y b i n d i n g" for snippet "if" in `ruby-mode-map' [yas] warning: won't overwrite keybinding "k e y b i n d i n g" for snippet "for" in `ruby-mode-map' [yas] warning: won't overwrite keybinding "k e y b i n d i n g" for snippet "do" in `ruby-mode-map' [yas] warning: won't overwrite keybinding "k e y b i n d i n g" for snippet "class" in `ruby-mode-map' Ido mode enabled [2 times] For information about GNU Emacs and the GNU system, type C-h C-a. Entering debugger... goto-history-element: Beginning of history; no preceding item [2 times] call-interactively: Text is read-only [2 times] Load-path shadows: /Users/nick/.emacs.d/vendor/nxhtml/related/php-mode hides ~/.emacs.d/vendor/php-mode ~/.emacs.d/vendor/whitespace hides /Applications/Emacs.app/Contents/Resources/lisp/whitespace /Users/nick/.emacs.d/vendor/cedet-1.0pre7/speedbar/speedbar hides /Applications/Emacs.app/Contents/Resources/lisp/speedbar /Users/nick/.emacs.d/vendor/cedet-1.0pre7/speedbar/sb-image hides /Applications/Emacs.app/Contents/Resources/lisp/sb-image ~/.emacs.d/vendor/linum hides /Applications/Emacs.app/Contents/Resources/lisp/linum /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/ezimage hides /Applications/Emacs.app/Contents/Resources/lisp/ezimage /Users/nick/.emacs.d/vendor/cedet-1.0pre7/speedbar/dframe hides /Applications/Emacs.app/Contents/Resources/lisp/dframe /Users/nick/.emacs.d/elpa/ruby-mode-1.1/ruby-mode hides /Applications/Emacs.app/Contents/Resources/lisp/progmodes/ruby-mode /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio-speedbar hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-speedbar /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio-opt hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-opt /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio-datadebug hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-datadebug /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio-custom hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-custom /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio-comp hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-comp /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/eieio-base hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-base /Users/nick/.emacs.d/vendor/cedet-1.0pre7/eieio/chart hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/chart /Users/nick/.emacs.d/vendor/cedet-1.0pre7/srecode/srecode hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/srecode /Users/nick/.emacs.d/vendor/cedet-1.0pre7/semantic/semantic hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/semantic /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/pulse hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/pulse /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/mode-local hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/mode-local /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/inversion hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/inversion /Users/nick/.emacs.d/vendor/cedet-1.0pre7/ede/ede hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/ede /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/data-debug hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/data-debug /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/cedet hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/cedet /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/cedet-idutils hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/cedet-idutils /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/cedet-global hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/cedet-global /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/cedet-files hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/cedet-files /Users/nick/.emacs.d/vendor/cedet-1.0pre7/common/cedet-cscope hides /Applications/Emacs.app/Contents/Resources/lisp/cedet/cedet-cscope Features: (shadow sort mail-extr message ecomplete rfc822 mml mml-sec password-cache mm-decode mm-bodies mm-encode mailcap mail-parse rfc2231 rfc2047 rfc2045 qp ietf-drums mailabbrev nnheader gnus-util netrc time-date mm-util mail-prsvr gmm-utils mailheader canlock sha1 hex-util hashcash mail-utils emacsbug cedet-edebug debug semantic-el semantic-bovine bovine-debug semantic-debug highlight-parentheses warnings erlang-start whitespace autopair paren cus-edit cus-start cus-load linum ido textmate hl-line yasnippet dropdown-list mumamo-fun mumamo nxhtml-autostart nxhtml-autoload majmodpri rinari easy-mmode jump inflections findr ruby-compilation which-func imenu pcomplete ansi-color inf-ruby ruby-mode nxhtml-menu web-autoload nxhtml-base ecb-layout-defs ecb ecb-symboldef ecb-analyse ecb-compatibility ecb-winman-support ecb-autogen autoload ecb-tod ecb-cycle ecb-eshell ecb-help ecb-jde ecb-method-browser hideshow ecb-file-browser ecb-layout compile comint ecb-create-layout ecb-compilation ecb-speedbar ecb-common-browser ecb-cedet-wrapper semanticdb-mode semanticdb-find semanticdb-ref ecb-navigate ecb-mode-line ecb-face tree-buffer ecb-upgrade ecb-util ring thingatpt cedet cedet-contrib-load cogre-load cogre-srecode semantic-edit srecode-template-mode srecode-template srecode-template-wy semantic-wisent wisent srecode-map srecode-dictionary speedbar-load sb-info ede-load ede-speedbar ede-files ede ede-base ede-auto eieio-speedbar semantic-ia-sb semantic-analyze semantic-scope semantic-analyze-fcn semantic-sort semanticdb-el semanticdb semantic-ctxt semantic-format semantic-util-modes semantic-util semantic semantic-lex semantic-tag working fame speedbar sb-image ezimage dframe assoc eieio-custom ede-source eieio-base srecode-load srecode semantic-load semantic-fw mode-local find-func derived eieio-load cedet-load cedet-compat eieio byte-opt bytecomp byte-compile inversion blank-mode-autoloads gist-autoloads rspec-mode-autoloads ruby-test-mode-autoloads sass-mode-autoloads advice help-fns advice-preload info wrap-region-autoloads yaml-mode-autoloads package color-theme edmacro kmacro wid-edit cl cl-19 sendmail regexp-opt reporter tooltip ediff-hook vc-hooks lisp-float-type mwheel ns-win easymenu tool-bar dnd fontset image fringe lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar mldrag mouse jit-lock font-lock syntax facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev loaddefs button minibuffer faces cus-face files text-properties overlay md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote make-network-process ns multi-tty emacs) ------------=_1344998582-15552-1--