GNU bug report logs - #5624
23.1; etags elisp and scheme "=" in names

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 5624 in the body.
You can then email your comments to 5624 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Kevin Ryde <user42 <at> zip.com.au>
To: bug-gnu-emacs <at> gnu.org
Subject: 23.1; etags elisp and scheme "=" in names
Date: Tue, 23 Feb 2010 09:09:59 +1100

If an elisp defun has an "=" char in its name, etags stops at that point
and the TAGS file doesn't have the whole name.  Eg. a foo.el containing

    (defun foo==bar () (message "hi"))

M-. foo==bar gives

    No tags containing foo==bar

The same seems to be true of scheme, eg. a foo.scm containing a similar

    (define foo==bar 123)

The way the tag line comes out means a single "=" at the end of a name,
or any number of them at the start of a name, ends up working, but I
believe it's legal (if unusual) to have them in the middle of a name
too.



In GNU Emacs 23.1.1 (i486-pc-linux-gnu, GTK+ Version 2.16.5)
 of 2009-09-14 on raven, modified by Debian
configured using `configure  '--build=i486-linux-gnu' '--host=i486-linux-gnu' '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib' '--localstatedir=/var/lib' '--infodir=/usr/share/info' '--mandir=/usr/share/man' '--with-pop=yes' '--enable-locallisppath=/etc/emacs23:/etc/emacs:/usr/local/share/emacs/23.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/23.1/site-lisp:/usr/share/emacs/site-lisp:/usr/share/emacs/23.1/leim' '--with-x=yes' '--with-x-toolkit=gtk' '--with-toolkit-scroll-bars' 'build_alias=i486-linux-gnu' 'host_alias=i486-linux-gnu' 'CFLAGS=-DDEBIAN -g -O2' 'LDFLAGS=-g' 'CPPFLAGS=''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_AU
  value of $XMODIFIERS: nil
  locale-coding-system: iso-latin-1-unix
  default-enable-multibyte-characters: t

Message #10 received at 5624 <at> debbugs.gnu.org (full text, mbox):

From: Alex <agrambot <at> gmail.com>
To: Kevin Ryde <user42 <at> zip.com.au>
Cc: 5624 <at> debbugs.gnu.org
Subject: Re: bug#5624: 23.1; etags elisp and scheme "=" in names
Date: Sat, 10 Jun 2017 19:37:49 -0600

Kevin Ryde <user42 <at> zip.com.au> writes:

> If an elisp defun has an "=" char in its name, etags stops at that point
> and the TAGS file doesn't have the whole name.  Eg. a foo.el containing
>
>     (defun foo==bar () (message "hi"))
>
> M-. foo==bar gives
>
>     No tags containing foo==bar
>
> The same seems to be true of scheme, eg. a foo.scm containing a similar
>
>     (define foo==bar 123)
>
> The way the tag line comes out means a single "=" at the end of a name,
> or any number of them at the start of a name, ends up working, but I
> believe it's legal (if unusual) to have them in the middle of a name
> too.

This appears to be because in etags.c, all languages use the same
`notinname' procedure to determine a valid identifier.

Shouldn't each language (optionally) use a different procedure to
determine the bounds of an identifier? Lisp and Scheme, for instance,
would not have '=' in their respective `notinname' procedures.

Message #13 received at 5624 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alex <agrambot <at> gmail.com>
Cc: 5624 <at> debbugs.gnu.org, user42 <at> zip.com.au
Subject: Re: bug#5624: 23.1; etags elisp and scheme "=" in names
Date: Sun, 11 Jun 2017 17:52:48 +0300

> From: Alex <agrambot <at> gmail.com>
> Date: Sat, 10 Jun 2017 19:37:49 -0600
> Cc: 5624 <at> debbugs.gnu.org
> 
> Kevin Ryde <user42 <at> zip.com.au> writes:
> 
> > If an elisp defun has an "=" char in its name, etags stops at that point
> > and the TAGS file doesn't have the whole name.  Eg. a foo.el containing
> >
> >     (defun foo==bar () (message "hi"))
> >
> > M-. foo==bar gives
> >
> >     No tags containing foo==bar
> >
> > The same seems to be true of scheme, eg. a foo.scm containing a similar
> >
> >     (define foo==bar 123)
> >
> > The way the tag line comes out means a single "=" at the end of a name,
> > or any number of them at the start of a name, ends up working, but I
> > believe it's legal (if unusual) to have them in the middle of a name
> > too.
> 
> This appears to be because in etags.c, all languages use the same
> `notinname' procedure to determine a valid identifier.

Yes.  But there are exceptions from this rule, where that is deemed
necessary.  For example, Ruby_functions has this:

	 /* Ruby method names can end in a '='.  Also, operator overloading can
	    define operators whose names include '='.  */
	  while (!notinname (*cp) || *cp == '=')
	    cp++;

> Shouldn't each language (optionally) use a different procedure to
> determine the bounds of an identifier? Lisp and Scheme, for instance,
> would not have '=' in their respective `notinname' procedures.

Feel free to work on this, if you want to scratch that particular
itch.  Alternatively, just augment notinname in language-specific
support code, like some of the supported languages already do.

TIA.

Message #16 received at 5624 <at> debbugs.gnu.org (full text, mbox):

From: Alex <agrambot <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 5624 <at> debbugs.gnu.org, user42 <at> zip.com.au
Subject: Re: bug#5624: 23.1; etags elisp and scheme "=" in names
Date: Sun, 11 Jun 2017 20:44:29 -0600

[Message part 1 (text/plain, inline)]

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Alex <agrambot <at> gmail.com>
>> Date: Sat, 10 Jun 2017 19:37:49 -0600
>> Cc: 5624 <at> debbugs.gnu.org

>> This appears to be because in etags.c, all languages use the same
>> `notinname' procedure to determine a valid identifier.
>
> Yes.  But there are exceptions from this rule, where that is deemed
> necessary.  For example, Ruby_functions has this:
>
> 	 /* Ruby method names can end in a '='.  Also, operator overloading can
> 	    define operators whose names include '='.  */
> 	  while (!notinname (*cp) || *cp == '=')
> 	    cp++;
>
>> Shouldn't each language (optionally) use a different procedure to
>> determine the bounds of an identifier? Lisp and Scheme, for instance,
>> would not have '=' in their respective `notinname' procedures.
>
> Feel free to work on this, if you want to scratch that particular
> itch.  Alternatively, just augment notinname in language-specific
> support code, like some of the supported languages already do.
>
> TIA.

One issue is that the Lisp and Scheme functions use get_tag, which uses
notinname. What do you think about the following diff? It adds a wrapper
to notinname which get_tag uses. I had to add an additional parameter to
get_tag.

[etags.diff (text/x-diff, inline)]

diff --git a/lib-src/etags.c b/lib-src/etags.c
index 6f280d8ab4..e354325ad5 100644
--- a/lib-src/etags.c
+++ b/lib-src/etags.c
@@ -181,6 +181,20 @@ notinname (unsigned char c)
   return table[c];
 }
 
+/* C is not in a name in language LANG. */
+static bool
+notinname_lang (unsigned char c, char const *lang)
+{
+  if (lang == NULL)
+    return notinname (c);
+
+  else if (streq ("lisp", lang) || streq ("scheme", lang))
+    if (c == '=')
+      return false;
+
+  return notinname (c);
+}
+
 /* C can start a token.  */
 static bool
 begtoken (unsigned char c)
@@ -371,7 +385,7 @@ static language *get_language_from_langname (const char *);
 static void readline (linebuffer *, FILE *);
 static long readline_internal (linebuffer *, FILE *, char const *);
 static bool nocase_tail (const char *);
-static void get_tag (char *, char **);
+static void get_tag (char *, char **, char const *);
 
 static void analyze_regex (char *);
 static void free_regexps (void);
@@ -4715,7 +4729,7 @@ Perl_functions (FILE *inf)
       if (LOOKING_AT (cp, "package"))
 	{
 	  free (package);
-	  get_tag (cp, &package);
+	  get_tag (cp, &package, NULL);
 	}
       else if (LOOKING_AT (cp, "sub"))
 	{
@@ -5346,7 +5360,7 @@ L_getit (void)
       /* Ok, then skip "(" before name in (defstruct (foo)) */
       dbp = skip_spaces (dbp);
   }
-  get_tag (dbp, NULL);
+  get_tag (dbp, NULL, "lisp");
 }
 
 static void
@@ -5426,7 +5440,7 @@ Lua_functions (FILE *inf)
 	{
 	  char *tag_name, *tp_dot, *tp_colon;
 
-	  get_tag (bp, &tag_name);
+	  get_tag (bp, &tag_name, NULL);
 	  /* If the tag ends with ".foo" or ":foo", make an additional tag for
 	     "foo".  */
 	  tp_dot = strrchr (tag_name, '.');
@@ -5436,7 +5450,7 @@ Lua_functions (FILE *inf)
 	      char *p = tp_dot > tp_colon ? tp_dot : tp_colon;
 	      int len_add = p - tag_name + 1;
 
-	      get_tag (bp + len_add, NULL);
+	      get_tag (bp + len_add, NULL, NULL);
 	    }
 	}
     }
@@ -5468,7 +5482,7 @@ PS_functions (FILE *inf)
 		    lb.buffer, ep - lb.buffer + 1, lineno, linecharno);
 	}
       else if (LOOKING_AT (bp, "defineps"))
-	get_tag (bp, NULL);
+	get_tag (bp, NULL, NULL);
     }
 }
 
@@ -5550,12 +5564,12 @@ Scheme_functions (FILE *inf)
 	  bp = skip_non_spaces (bp+4);
 	  /* Skip over open parens and white space.  Don't continue past
 	     '\0'. */
-	  while (*bp && notinname (*bp))
+	  while (*bp && notinname_lang (*bp, "scheme"))
 	    bp++;
-	  get_tag (bp, NULL);
+	  get_tag (bp, NULL, "scheme");
 	}
       if (LOOKING_AT (bp, "(SET!") || LOOKING_AT (bp, "(set!"))
-	get_tag (bp, NULL);
+	get_tag (bp, NULL, "scheme");
     }
 }
 
@@ -6573,14 +6587,14 @@ nocase_tail (const char *cp)
 }
 
 static void
-get_tag (register char *bp, char **namepp)
+get_tag (register char *bp, char **namepp, char const *lang)
 {
   register char *cp = bp;
 
   if (*bp != '\0')
     {
       /* Go till you get to white space or a syntactic break */
-      for (cp = bp + 1; !notinname (*cp); cp++)
+      for (cp = bp + 1; !notinname_lang (*cp, lang); cp++)
 	continue;
       make_tag (bp, cp - bp, true,
 		lb.buffer, cp - lb.buffer + 1, lineno, linecharno);

Message #19 received at 5624 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alex <agrambot <at> gmail.com>
Cc: 5624 <at> debbugs.gnu.org, user42 <at> zip.com.au
Subject: Re: bug#5624: 23.1; etags elisp and scheme "=" in names
Date: Mon, 12 Jun 2017 17:13:17 +0300

> From: Alex <agrambot <at> gmail.com>
> Cc: user42 <at> zip.com.au,  5624 <at> debbugs.gnu.org
> Date: Sun, 11 Jun 2017 20:44:29 -0600
> 
> One issue is that the Lisp and Scheme functions use get_tag, which uses
> notinname. What do you think about the following diff? It adds a wrapper
> to notinname which get_tag uses.

I think it would be much cleaner not to use get_tag for these
languages, but instead either call make_tag directly or write a
get_lispy_tag function which will DTRT for Lisp-like languages.

Thanks.

Message #22 received at 5624 <at> debbugs.gnu.org (full text, mbox):

From: Alex <agrambot <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 5624 <at> debbugs.gnu.org, user42 <at> zip.com.au
Subject: Re: bug#5624: 23.1; etags elisp and scheme "=" in names
Date: Mon, 12 Jun 2017 21:31:57 -0600

[Message part 1 (text/plain, inline)]

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Alex <agrambot <at> gmail.com>
>> Cc: user42 <at> zip.com.au,  5624 <at> debbugs.gnu.org
>> Date: Sun, 11 Jun 2017 20:44:29 -0600
>> 
>> One issue is that the Lisp and Scheme functions use get_tag, which uses
>> notinname. What do you think about the following diff? It adds a wrapper
>> to notinname which get_tag uses.
>
> I think it would be much cleaner not to use get_tag for these
> languages, but instead either call make_tag directly or write a
> get_lispy_tag function which will DTRT for Lisp-like languages.
>
> Thanks.

I wanted to reduce duplicated code, but I suppose it is cleaner that
way.

I've attached a patch below.

[0001-Include-in-Scheme-and-Lisp-tags-in-etags.patch (text/x-diff, attachment)]

Message #25 received at 5624 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alex <agrambot <at> gmail.com>
Cc: 5624 <at> debbugs.gnu.org, user42 <at> zip.com.au
Subject: Re: bug#5624: 23.1; etags elisp and scheme "=" in names
Date: Tue, 13 Jun 2017 17:28:51 +0300

> From: Alex <agrambot <at> gmail.com>
> Cc: user42 <at> zip.com.au,  5624 <at> debbugs.gnu.org
> Date: Mon, 12 Jun 2017 21:31:57 -0600
> 
> > I think it would be much cleaner not to use get_tag for these
> > languages, but instead either call make_tag directly or write a
> > get_lispy_tag function which will DTRT for Lisp-like languages.
> >
> > Thanks.
> 
> I wanted to reduce duplicated code, but I suppose it is cleaner that
> way.
> 
> I've attached a patch below.

Thanks.  This looks OK to me, but please also add a test for this
problem, and make sure the previous tests still succeed.  (The etags
test suite is in test/manual/etags/.)

> +/* Similar to get_tag, but include '=' as part of the tag. */
> +static void
> +get_lispy_tag (register char *bp, char **namepp)
> +{
> +  register char *cp = bp;
> +
> +  if (*bp != '\0')
> +    {
> +      /* Go till you get to white space or a syntactic break */
> +      for (cp = bp + 1; !notinname (*cp) || *cp == '='; cp++)
> +	continue;
> +      make_tag (bp, cp - bp, true,
> +		lb.buffer, cp - lb.buffer + 1, lineno, linecharno);
> +    }
> +
> +  if (namepp != NULL)
> +    *namepp = savenstr (bp, cp - bp);
> +}

It looks like none of the callers uses a non-NULL 2nd arg, so perhaps
just remove it, and its supporting code.

Message #28 received at 5624 <at> debbugs.gnu.org (full text, mbox):

From: Alex <agrambot <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 5624 <at> debbugs.gnu.org, user42 <at> zip.com.au
Subject: Re: bug#5624: 23.1; etags elisp and scheme "=" in names
Date: Wed, 14 Jun 2017 16:10:37 -0600

[Message part 1 (text/plain, inline)]

Eli Zaretskii <eliz <at> gnu.org> writes:

>> From: Alex <agrambot <at> gmail.com>
>> Cc: user42 <at> zip.com.au,  5624 <at> debbugs.gnu.org
>> Date: Mon, 12 Jun 2017 21:31:57 -0600
>> 
>> > I think it would be much cleaner not to use get_tag for these
>> > languages, but instead either call make_tag directly or write a
>> > get_lispy_tag function which will DTRT for Lisp-like languages.
>> >
>> > Thanks.
>> 
>> I wanted to reduce duplicated code, but I suppose it is cleaner that
>> way.
>> 
>> I've attached a patch below.
>
> Thanks.  This looks OK to me, but please also add a test for this
> problem, and make sure the previous tests still succeed.  (The etags
> test suite is in test/manual/etags/.)
>
>> +/* Similar to get_tag, but include '=' as part of the tag. */
>> +static void
>> +get_lispy_tag (register char *bp, char **namepp)
>> +{
>> +  register char *cp = bp;
>> +
>> +  if (*bp != '\0')
>> +    {
>> +      /* Go till you get to white space or a syntactic break */
>> +      for (cp = bp + 1; !notinname (*cp) || *cp == '='; cp++)
>> +	continue;
>> +      make_tag (bp, cp - bp, true,
>> +		lb.buffer, cp - lb.buffer + 1, lineno, linecharno);
>> +    }
>> +
>> +  if (namepp != NULL)
>> +    *namepp = savenstr (bp, cp - bp);
>> +}
>
> It looks like none of the callers uses a non-NULL 2nd arg, so perhaps
> just remove it, and its supporting code.

Alright, I did both.

[0001-Include-in-Scheme-and-Lisp-tags-in-etags.patch (text/x-diff, attachment)]

Message #33 received at 5624-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Alex <agrambot <at> gmail.com>
Cc: 5624-done <at> debbugs.gnu.org, user42 <at> zip.com.au
Subject: Re: bug#5624: 23.1; etags elisp and scheme "=" in names
Date: Sat, 08 Jul 2017 11:28:02 +0300

> From: Alex <agrambot <at> gmail.com>
> Cc: user42 <at> zip.com.au,  5624 <at> debbugs.gnu.org
> Date: Wed, 14 Jun 2017 16:10:37 -0600
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> From: Alex <agrambot <at> gmail.com>
> >> Cc: user42 <at> zip.com.au,  5624 <at> debbugs.gnu.org
> >> Date: Mon, 12 Jun 2017 21:31:57 -0600
> >> 
> >> > I think it would be much cleaner not to use get_tag for these
> >> > languages, but instead either call make_tag directly or write a
> >> > get_lispy_tag function which will DTRT for Lisp-like languages.
> >> >
> >> > Thanks.
> >> 
> >> I wanted to reduce duplicated code, but I suppose it is cleaner that
> >> way.
> >> 
> >> I've attached a patch below.
> >
> > Thanks.  This looks OK to me, but please also add a test for this
> > problem, and make sure the previous tests still succeed.  (The etags
> > test suite is in test/manual/etags/.)
> >
> >> +/* Similar to get_tag, but include '=' as part of the tag. */
> >> +static void
> >> +get_lispy_tag (register char *bp, char **namepp)
> >> +{
> >> +  register char *cp = bp;
> >> +
> >> +  if (*bp != '\0')
> >> +    {
> >> +      /* Go till you get to white space or a syntactic break */
> >> +      for (cp = bp + 1; !notinname (*cp) || *cp == '='; cp++)
> >> +	continue;
> >> +      make_tag (bp, cp - bp, true,
> >> +		lb.buffer, cp - lb.buffer + 1, lineno, linecharno);
> >> +    }
> >> +
> >> +  if (namepp != NULL)
> >> +    *namepp = savenstr (bp, cp - bp);
> >> +}
> >
> > It looks like none of the callers uses a non-NULL 2nd arg, so perhaps
> > just remove it, and its supporting code.
> 
> Alright, I did both.

Thanks, pushed to master.

Sorry for such a long delay in pushing.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.