GNU bug report logs - #76322
Make ctags a thin wrapper around etags

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 76322 in the body.
You can then email your comments to 76322 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Emacs bug reports and feature requests <bug-gnu-emacs <at> gnu.org>
Subject: Make ctags a thin wrapper around etags
Date: Sat, 15 Feb 2025 21:21:00 -0800

[Message part 1 (text/plain, inline)]

ctags and etags are both built from the same source file, with a 
different compile-time flag. This takes quite a while on my old desktop 
when I configure with --enable-gcc-warnings as is my usual practice. 
Also, it means two very similar executables are floating around where 
one would do. This has long been a FIXME and I finally got annoyed by 
the long compiles enough to do something about it. Patch attached.

[0001-Make-ctags-a-thin-wrapper-around-etags.patch (text/x-patch, attachment)]

Message #10 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 76322 <at> debbugs.gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Sun, 16 Feb 2025 09:12:26 +0200

> Date: Sat, 15 Feb 2025 21:21:00 -0800
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> 
> ctags and etags are both built from the same source file, with a 
> different compile-time flag. This takes quite a while on my old desktop 
> when I configure with --enable-gcc-warnings as is my usual practice. 
> Also, it means two very similar executables are floating around where 
> one would do. This has long been a FIXME and I finally got annoyed by 
> the long compiles enough to do something about it. Patch attached.

Sorry, that won't fly as suggested because Unix shell scripts don't
work on Windows.  So building Emacs on Windows and DOS will still need
to compile the same source twice, and the patch needs to be changed to
do what you suggest only on Posix platforms, leaving ctags a .exe
program on DOS/Windows.  (And I'd like to avoid making ctags a batch
file on Windows, for various obscure reasons.)

Message #13 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 76322 <at> debbugs.gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Sun, 16 Feb 2025 00:26:02 -0800

On 2025-02-15 23:12, Eli Zaretskii wrote:
> So building Emacs on Windows and DOS will still need
> to compile the same source twice,

How about if we compile a little .exe file that merely execs etags with 
the --ctags option? That would be the equivalent of the script.

Message #16 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 76322 <at> debbugs.gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Sun, 16 Feb 2025 11:31:19 +0200

> Date: Sun, 16 Feb 2025 00:26:02 -0800
> Cc: 76322 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> 
> On 2025-02-15 23:12, Eli Zaretskii wrote:
> > So building Emacs on Windows and DOS will still need
> > to compile the same source twice,
> 
> How about if we compile a little .exe file that merely execs etags with 
> the --ctags option? That would be the equivalent of the script.

That'd be fine, thanks.  But we will need to invoke etags via
w32_execvp because of the problems explained in emacsclient.c where
that function is defined (which will need to be moved to ntlib.c).

Message #19 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 76322 <at> debbugs.gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Mon, 17 Feb 2025 00:16:58 -0800

[Message part 1 (text/plain, inline)]

On 2025-02-16 01:31, Eli Zaretskii wrote:
>> Date: Sun, 16 Feb 2025 00:26:02 -0800
>> From: Paul Eggert <eggert <at> cs.ucla.edu>
>> How about if we compile a little .exe file that merely execs etags with
>> the --ctags option? That would be the equivalent of the script.
> 
> That'd be fine, thanks.  But we will need to invoke etags via
> w32_execvp because of the problems explained in emacsclient.c where
> that function is defined (which will need to be moved to ntlib.c).

I took at shot at doing something along those lines; see revised patch 
(attached).

Since I don't use MS-Windows or compile or test anything on that 
platform, I quailed a bit at doing nontrivial surgery on ntlib.c. Also, 
emacsclient.c's w32_execvp has some alternate-editor code specific to 
emacsclient, and it uses alloca in a potentially-unsafe way, and it 
mishandles strings longer than INT_MAX bytes, and it appears to 
mishandle '\t', '\\' and '"' if Microsoft's documentation is to be 
believed. So the attached patch defines its own w32_execvp_arg_repr 
function that attempts to be standalone and free of these issues. If 
needed later, someone with MS-Windows expertise could merge this into 
ntlib.c and combine it with what's in emacsclient.c.

[0001-Make-ctags-a-thin-wrapper-around-etags.patch (text/x-patch, attachment)]

Message #24 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 76322 <at> debbugs.gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Thu, 20 Feb 2025 11:23:12 +0200

> Date: Mon, 17 Feb 2025 00:16:58 -0800
> Cc: 76322 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> 
> On 2025-02-16 01:31, Eli Zaretskii wrote:
> >> Date: Sun, 16 Feb 2025 00:26:02 -0800
> >> From: Paul Eggert <eggert <at> cs.ucla.edu>
> >> How about if we compile a little .exe file that merely execs etags with
> >> the --ctags option? That would be the equivalent of the script.
> > 
> > That'd be fine, thanks.  But we will need to invoke etags via
> > w32_execvp because of the problems explained in emacsclient.c where
> > that function is defined (which will need to be moved to ntlib.c).
> 
> I took at shot at doing something along those lines; see revised patch 
> (attached).
> 
> Since I don't use MS-Windows or compile or test anything on that 
> platform, I quailed a bit at doing nontrivial surgery on ntlib.c. Also, 
> emacsclient.c's w32_execvp has some alternate-editor code specific to 
> emacsclient, and it uses alloca in a potentially-unsafe way, and it 
> mishandles strings longer than INT_MAX bytes, and it appears to 
> mishandle '\t', '\\' and '"' if Microsoft's documentation is to be 
> believed. So the attached patch defines its own w32_execvp_arg_repr 
> function that attempts to be standalone and free of these issues. If 
> needed later, someone with MS-Windows expertise could merge this into 
> ntlib.c and combine it with what's in emacsclient.c.

Thanks, a few minor comments below.

> +INSTALLABLE_EXES = etags${EXEEXT} $(if $(NTDIR), ctags$(EXEEXT)) \
> +  emacsclient${EXEEXT} $(CLIENTW) ebrowse${EXEEXT}
> +INSTALLABLE_SCRIPTS = $(if $(NTDIR), , ctags)

The dependence on NTDIR is sub-optimal, IMO.  For starters, it means
that one cannot reliably invoke "make" in lib-src, because (AFAIU)
NTDIR is defined in the top-level Makefile.  It is better to depend on
the value of NTLIB, which is defined in lib-src/Makefile.

Or maybe we should just depend on the value of EXEEXT?  I think that
might be preferable, because it will also work for the MS-DOS build of
Emacs, where we have the same problem with invoking Unix shell
scripts.

> +/* Return the number of bytes used to represent the string S as an
> +   MS-Windows _execvp argument, including the trailing null byte.
> +   If D is nonnull, also store the representation into D.
> +   Due to Microsoft C runtime constraints, S cannot contain '\n',
> +   and if S contains '"' the representation cannot be used for argument 0.  */
> +static size_t
> +w32_execvp_arg_repr (char const *restrict s, char *restrict d)
> +{
> +  /* See <https://learn.microsoft.com/en-us/cpp/c-language/parsing-c-command-line-arguments>.

This part should be under #ifdef WINDOWSNT, since it's specific to
Windows.  The MS-DOS build doesn't have these problems with execvp.

> +int
> +main (int argc, char **argv)
> +{
> +  char const *progname = argv[0];
> +  static char const option_prefix[] = "--ctags=";
> +
> +  /* Number of bytes needed to represent all etags args.  */
> +  size_t nargbytes = sizeof option_prefix - 1;
> +  for (int i = 0; i < argc; i++)
> +    nargbytes += w32_execvp_arg_repr (argv[i], NULL);
> +
> +  char const **etags_argv = malloc ((argc + 2) * sizeof *argv + nargbytes);
> +  if (etags_argv)
> +    {
> +      char *d = (char *) (etags_argv + argc + 2);
> +      etags_argv[0] = "etags";
> +      etags_argv[1] = memcpy (d, option_prefix, sizeof option_prefix - 1);
> +      d += sizeof option_prefix - 1;
> +      d += w32_execvp_arg_repr (progname, d);
> +
> +      for (int i = 1; i < argc; i++)
> +	{
> +	  etags_argv[i + 1] = d;
> +	  d += w32_execvp_arg_repr (argv[i], d);
> +	}

Likewise here: the calls to w32_execvp_arg_repr should only be done on
WINDOWSNT.

Also, this:

> +      etags_argv[0] = "etags";

basically means the etags to be invoked is the one found on PATH,
whereas I think we want to invoke etags from the same place from where
ctags was invoked.  So I would suggest keeping any leading directories
in argv[0] intact, and just replacing "ctags" with "etags" to generate
the new argv[0].

Caveat: I didn't try to actually build this stuff on Windows, let
alone run the test suite; that will need to wait till later, maybe
even till after you install this on master.

Thanks again.

Message #27 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 76322 <at> debbugs.gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Tue, 4 Mar 2025 10:37:05 -0800

[Message part 1 (text/plain, inline)]

Thanks for the review. Revised patch attached, which I think addresses 
all the comments.

[0001-Make-ctags-a-thin-wrapper-around-etags.patch (text/x-patch, attachment)]

Message #30 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 76322 <at> debbugs.gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Thu, 06 Mar 2025 15:31:11 +0200

> Date: Tue, 4 Mar 2025 10:37:05 -0800
> Cc: 76322 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> 
> Thanks for the review. Revised patch attached, which I think addresses 
> all the comments.

Thanks.  This mostly worked, but I needed a few minor changes (below).
We should use _spawnvp, not _execvp, since the latter doesn't return,
so we cannot return the exit status of etags to the caller.  More
importantly, the counting of bytes to allocate missed the PROGRAM part
of --ctags=PROGRAM, so ctags would crash in weird ways.  Finally, I
need to include stdio.h to get the declaration of perror.  Here's the
patch:


diff --git a/lib-src/ctags.c b/lib-src/ctags.c
index 82324f5..2a2a7d2 100644
--- a/lib-src/ctags.c
+++ b/lib-src/ctags.c
@@ -23,13 +23,14 @@ #define DEFER_MS_W32_H
 #include <filename.h>
 
 #include <malloc.h>
+#include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 
 /* Declare _execvp by hand, since MS-Windows <process.h> collides with
    ../src/process.h.  */
 #include <stdint.h> /* for intptr_t */
-extern intptr_t _execvp (char const *, char const *const *);
+extern intptr_t _spawnvp (int, char const *, char const *const *);
 
 #ifdef WINDOWSNT
 /* Store into D[I] through D[I + N - 1] the byte C, and return I + N.
@@ -100,7 +101,10 @@ main (int argc, char **argv)
   for (int i = 0; i < argc; i++)
     nargbytes += w32_execvp_arg_repr (argv[i], NULL);
   nargbytes -= progname_baselen;
-  nargbytes += sizeof etags_basename - 1 + sizeof option_prefix - 1;
+  nargbytes += sizeof etags_basename - 1;
+  nargbytes += w32_execvp_arg_repr (option_prefix, NULL);
+  nargbytes += w32_execvp_arg_repr (argv[0], NULL)
+	       - progname_baselen + sizeof etags_basename - 1;
 
   char const **etags_argv = malloc ((argc + 2) * sizeof *argv + nargbytes);
   if (etags_argv)
@@ -122,7 +126,7 @@ main (int argc, char **argv)
 
       etags_argv[argc + 1] = NULL;
       progname = etags_argv[0];
-      _execvp (progname, etags_argv);
+      return _spawnvp (0, progname, etags_argv);
     }
 
   perror (progname);

Message #33 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 76322 <at> debbugs.gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Sun, 9 Mar 2025 00:13:51 -0800

[Message part 1 (text/plain, inline)]

On 2025-03-06 05:31, Eli Zaretskii wrote:
>> Date: Tue, 4 Mar 2025 10:37:05 -0800
>> Cc: 76322 <at> debbugs.gnu.org
>> From: Paul Eggert <eggert <at> cs.ucla.edu>
>>
>> Thanks for the review. Revised patch attached, which I think addresses
>> all the comments.
> 
> Thanks.  This mostly worked, but I needed a few minor changes (below).
> We should use _spawnvp, not _execvp, since the latter doesn't return,
> so we cannot return the exit status of etags to the caller.  More
> importantly, the counting of bytes to allocate missed the PROGRAM part
> of --ctags=PROGRAM, so ctags would crash in weird ways.  Finally, I
> need to include stdio.h to get the declaration of perror.  Here's the
> patch:

Thanks, I incorporated all that into the proposed patch. A couple of 
things, though. First, when _spawnvp returns -1 I assume ctags should 
call perror and exit with status 127 rather than simply try to exit with 
status -1. Second and more important, my earlier proposal violated the 
GNU Coding Standards because its behavior depended on its name.[1]

Fixing the second problem simplified the C and shell code a bit. Revised 
patch attached.

[1]: 
https://www.gnu.org/prep/standards/html_node/User-Interfaces.html#index-program-name-and-its-behavior

[0001-Make-ctags-a-thin-wrapper-around-etags.patch (text/x-patch, attachment)]

Message #36 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 76322 <at> debbugs.gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Sun, 09 Mar 2025 14:15:51 +0200

> Date: Sun, 9 Mar 2025 00:13:51 -0800
> Cc: 76322 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> 
> > Thanks.  This mostly worked, but I needed a few minor changes (below).
> > We should use _spawnvp, not _execvp, since the latter doesn't return,
> > so we cannot return the exit status of etags to the caller.  More
> > importantly, the counting of bytes to allocate missed the PROGRAM part
> > of --ctags=PROGRAM, so ctags would crash in weird ways.  Finally, I
> > need to include stdio.h to get the declaration of perror.  Here's the
> > patch:
> 
> Thanks, I incorporated all that into the proposed patch. A couple of 
> things, though. First, when _spawnvp returns -1 I assume ctags should 
> call perror and exit with status 127 rather than simply try to exit with 
> status -1. Second and more important, my earlier proposal violated the 
> GNU Coding Standards because its behavior depended on its name.[1]
> 
> Fixing the second problem simplified the C and shell code a bit. Revised 
> patch attached.

Thanks, but this doesn't quite work.  The reason is here:

> +  static char const etags_program[] = ETAGS_PROGRAM;

ETAGS_PROGRAM is computed by lib-src/Makefile, which sets it to the
absolute file name of etags in the installation tree.  So if the etags
there doesn't exist (e.g., Emacs was not yet installed), or is not the
new etags which understands the --ctags= option, ctags will fail.  So
I cannot even run the etags test suite after building the patched
tree, because all ctags tests failed.

Why did you drop the previous approach, which just replaced "ctags"
with "etags" in the original argv[0]?  That did work, was reliable,
and allowed to move the installed programs to any other directory
(something that happens a lot on Windows).  It also made sure the pair
of programs came from the same Emacs version, which is also important.
Was there any problem with that approach that required you to abandon
it?

Thanks.

Message #39 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Pip Cet <pipcet <at> protonmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 76322 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Sun, 09 Mar 2025 12:23:33 +0000

"Paul Eggert" <eggert <at> cs.ucla.edu> writes:

> On 2025-03-06 05:31, Eli Zaretskii wrote:
>>> Date: Tue, 4 Mar 2025 10:37:05 -0800
>>> Cc: 76322 <at> debbugs.gnu.org
>>> From: Paul Eggert <eggert <at> cs.ucla.edu>
>>>
>>> Thanks for the review. Revised patch attached, which I think addresses
>>> all the comments.
>>
>> Thanks.  This mostly worked, but I needed a few minor changes (below).
>> We should use _spawnvp, not _execvp, since the latter doesn't return,
>> so we cannot return the exit status of etags to the caller.  More
>> importantly, the counting of bytes to allocate missed the PROGRAM part
>> of --ctags=PROGRAM, so ctags would crash in weird ways.  Finally, I
>> need to include stdio.h to get the declaration of perror.  Here's the
>> patch:
>
> Thanks, I incorporated all that into the proposed patch. A couple of
> things, though. First, when _spawnvp returns -1 I assume ctags should
> call perror and exit with status 127 rather than simply try to exit with
> status -1. Second and more important, my earlier proposal violated the
> GNU Coding Standards because its behavior depended on its name.[1]

If you're referring to the previous behavior of looking for etags in the
right directory, rather than a hard-coded directory as the current patch
does, I'm not sure how the GNU coding standards prohibit that.

> Fixing the second problem simplified the C and shell code a bit. Revised
> patch attached.

But that means we hard-code the installation path for "etags" in the
"ctags" shell script/executable, which seems worse than simply depending
on the name (or going back to building two binaries) to me.
Non-relocatable builds are a pain.

It's also different from what other GNU utilities do: gunzip looks for
gzip in the PATH, for example.

> diff --git a/lib-src/ctags.sh b/lib-src/ctags.sh
> new file mode 100755
> index 00000000000..13a4320d9e9
> --- /dev/null
> +++ b/lib-src/ctags.sh
> @@ -0,0 +1,2 @@
> +#!/bin/sh
> +exec '@ETAGS_PROGRAM@' --ctags="$0" "$@"

I think this file should not be executable (and I'm not sure why it's
called "ctags.sh" rather than "ctags.in" or something similar, which
would indicate it needs to be processed to yield an executable file).

> +  for (; *s; s++)
> +    {
> +      if (*s == '"')
> +        store (d, i, bscount + 1, '\\');

This sholud be i = store (d, i , ...), so we don't lose the backslashes.

Pip

Message #42 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Pip Cet <pipcet <at> protonmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 76322 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Sun, 09 Mar 2025 13:08:32 +0000

Pip Cet <pipcet <at> protonmail.com> writes:

> "Paul Eggert" <eggert <at> cs.ucla.edu> writes:
>
>> +  for (; *s; s++)
>> +    {
>> +      if (*s == '"')
>> +        store (d, i, bscount + 1, '\\');
>
> This sholud be i = store (d, i , ...), so we don't lose the
> backslashes.

Also (but much more harmless, no buffer overflow here):

  size_t nargbytes = sizeof etags_program + sizeof option_prefix - 1;

should be

  size_t nargbytes = sizeof option_prefix - 1;

because etags_program is no longer copied into the newly allocated area.

Pip

Message #45 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Richard Stallman <rms <at> gnu.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 76322 <at> debbugs.gnu.org, eggert <at> cs.ucla.edu
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Mon, 10 Mar 2025 00:18:20 -0400

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

i wrote etags very early in development of GNU, starting from ctags
which I did not write.  At that time, I considered it unclean for a
program to behave differently depending on the name it is invoked
with.  The idea was that a user could give the program any name,
and it would still behave the same.

Back then, there was a reason to have ctags and etags, making their
different formats.  etags output had rough source locations which sped
up finding a definition in the source.

Nowadays that speedup may be insignificant in practice.

Is it still beneficial?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Message #48 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: rms <at> gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Cc: 76322 <at> debbugs.gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Sun, 9 Mar 2025 23:33:03 -0700

On 2025-03-09 21:18, Richard Stallman wrote:
> The idea was that a user could give the program any name,
> and it would still behave the same.

Yes, the idea is that a user can copy /usr/bin/ctags to (for example) 
/usr/bin/ceetags or /home/eggert/bin/sitags and the command will still 
work under its new name.

If ctags invokes etags by replacing $0's basename with "etags", copying 
/usr/bin/ctags to /usr/bin/ceetags just works, whereas copying 
/usr/bin/ctags to /home/eggert/bin/sitags will require that the user 
also copy or link /usr/bin/etags to /home/eggert/bin/etags. I assume 
this requirement is OK (as I understand it, Eli says it's OK).


> Back then, there was a reason to have ctags and etags, making their
> different formats.  etags output had rough source locations which sped
> up finding a definition in the source.
> 
> Nowadays that speedup may be insignificant in practice.
> 
> Is it still beneficial?

Dropping support for ctags output would address the problem by removing 
the need for two programs. And since there is a GPL'ed ctags[1] that is 
better than what's shipped with Emacs, one option is for Emacs to stop 
installing ctags, and to advise users who want a ctags program to use 
the other ctags instead. That's the default on many distros nowadays anyway.

More radically, Emacs could also stop installing etags, and advise 
people to run ctags -e instead, where this is the other ctags.

As a practical matter, ctags and etags both have problems with languages 
like C++ and Python where the same name can mean many different things 
depending on context. My impression is that nowadays many (most?) people 
who want tags-like capabilities use Language Server Protocol[2] clients 
like lsp-mode[3] instead, as LSP can handle these more-complicated 
naming conventions. Ctags and etags are mostly needed for old-fashioned 
users.

[1]: https://github.com/universal-ctags/ctags
[2]: https://en.wikipedia.org/wiki/Language_Server_Protocol
[3]: https://emacs-lsp.github.io/lsp-mode/

Message #51 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: rms <at> gnu.org
Cc: 76322 <at> debbugs.gnu.org, eggert <at> cs.ucla.edu
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Mon, 10 Mar 2025 17:40:51 +0200

> From: Richard Stallman <rms <at> gnu.org>
> Cc: eggert <at> cs.ucla.edu, 76322 <at> debbugs.gnu.org
> Date: Mon, 10 Mar 2025 00:18:20 -0400
> 
> i wrote etags very early in development of GNU, starting from ctags
> which I did not write.  At that time, I considered it unclean for a
> program to behave differently depending on the name it is invoked
> with.  The idea was that a user could give the program any name,
> and it would still behave the same.
> 
> Back then, there was a reason to have ctags and etags, making their
> different formats.  etags output had rough source locations which sped
> up finding a definition in the source.
> 
> Nowadays that speedup may be insignificant in practice.
> 
> Is it still beneficial?

Emacs doesn't use the output of ctags.  All our tags-related commands
use on the TAGS files produced by etags.  So dropping ctags will lose
Emacs nothing.

AFAIR, the ctags output is used by other editors (Vim, I think?).
Whether that, or the fact that we shipped ctags since forever, is a
significant enough reason to keep it in the tree and in the tarballs,
is another question.  Maybe we should ask downstream distros how they
feel about dropping ctags.

Message #54 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 76322 <at> debbugs.gnu.org, rms <at> gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Mon, 10 Mar 2025 18:05:38 +0200

> Date: Sun, 9 Mar 2025 23:33:03 -0700
> Cc: 76322 <at> debbugs.gnu.org
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> 
> On 2025-03-09 21:18, Richard Stallman wrote:
> > The idea was that a user could give the program any name,
> > and it would still behave the same.
> 
> Yes, the idea is that a user can copy /usr/bin/ctags to (for example) 
> /usr/bin/ceetags or /home/eggert/bin/sitags and the command will still 
> work under its new name.

If we must support arbitrary renames of programs that are not
synchronized across the entire Emacs installation, i.e. not bounded by
what we have in --program-transform-name=PROGRAM and --program-prefix
etc., then no wrapper script can ever work reliably to cause etags
work as ctags, because the script will be unable to find the correct
etags to run.

> If ctags invokes etags by replacing $0's basename with "etags", copying 
> /usr/bin/ctags to /usr/bin/ceetags just works, whereas copying 
> /usr/bin/ctags to /home/eggert/bin/sitags will require that the user 
> also copy or link /usr/bin/etags to /home/eggert/bin/etags. I assume 
> this requirement is OK (as I understand it, Eli says it's OK).

This requirement will be OK if the programs were renamed using the
facilities mentioned above, or at least the most common uses fog them.

> Dropping support for ctags output would address the problem by removing 
> the need for two programs. And since there is a GPL'ed ctags[1] that is 
> better than what's shipped with Emacs, one option is for Emacs to stop 
> installing ctags, and to advise users who want a ctags program to use 
> the other ctags instead. That's the default on many distros nowadays anyway.

We can do something less drastic: tell them to use "etags --ctags"
instead.

> More radically, Emacs could also stop installing etags, and advise 
> people to run ctags -e instead, where this is the other ctags.

To this I object, since the master source of the format of the TAGS
files must be in Emacs, and what better to document this format than
to maintain and distribute etags the program.

> As a practical matter, ctags and etags both have problems with languages 
> like C++ and Python where the same name can mean many different things 
> depending on context. My impression is that nowadays many (most?) people 
> who want tags-like capabilities use Language Server Protocol[2] clients 
> like lsp-mode[3] instead, as LSP can handle these more-complicated 
> naming conventions. Ctags and etags are mostly needed for old-fashioned 
> users.

Given that there still is no GCC-based LSP server for C and C++, using
LSP for C/C++ development means one needs to install clang in addition
to GCC, and that is a nuisance.

(I urged the GCC developers several years ago to develop LSP
capabilities, but AFAIK nothing practical came out if that.)

In any case, etags is a much more lightweight solution for what it
provides (which us only a small part of what an LSP server can
provide), so removing it for that reason would be unwise, IMO.

Message #57 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 76322 <at> debbugs.gnu.org, rms <at> gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Mon, 10 Mar 2025 16:14:13 -0400

Eli Zaretskii <eliz <at> gnu.org> writes:

>> More radically, Emacs could also stop installing etags, and advise
>> people to run ctags -e instead, where this is the other ctags.
>
> To this I object, since the master source of the format of the TAGS
> files must be in Emacs, and what better to document this format than
> to maintain and distribute etags the program.

As a counter-point, we distribute parsers for quite a few formats where
the master source of the format is not in emacs.git.  They don't usually
have the same legacy as ctags, admittedly.

How about adding a --without-ctags flag, for those of us that never have
a need for it?

Message #60 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: 76322 <at> debbugs.gnu.org, Eli Zaretskii <eliz <at> gnu.org>, rms <at> gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Mon, 10 Mar 2025 16:42:12 -0700

On 3/10/25 13:14, Stefan Kangas wrote:
> How about adding a --without-ctags flag, for those of us that never have
> a need for it?

I assume you mean a --disable-ctags option for 'configure'. Yes, that's 
doable. We could also have a --disable-etags if we want people to be 
able to disable ctags without disabling etags. The reverse wouldn't be 
possible (or needed).

This is an independent issue of course.

Message #63 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Eli Zaretskii <eliz <at> gnu.org>, rms <at> gnu.org
Cc: 76322 <at> debbugs.gnu.org, eggert <at> cs.ucla.edu
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Tue, 11 Mar 2025 04:34:55 +0200

On 10/03/2025 17:40, Eli Zaretskii wrote:
> AFAIR, the ctags output is used by other editors (Vim, I think?).
> Whether that, or the fact that we shipped ctags since forever, is a
> significant enough reason to keep it in the tree and in the tarballs,
> is another question.  Maybe we should ask downstream distros how they
> feel about dropping ctags.

From what I'm reading, a lot of distros have replaced their 'ctags' 
package with universal-ctags:

https://repology.org/project/universal-ctags/versions

It's the main ctags fork out there, and a pretty active one.

Message #66 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: 76322 <at> debbugs.gnu.org, eggert <at> cs.ucla.edu, rms <at> gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Tue, 11 Mar 2025 14:11:11 +0200

> From: Stefan Kangas <stefankangas <at> gmail.com>
> Date: Mon, 10 Mar 2025 16:14:13 -0400
> Cc: 76322 <at> debbugs.gnu.org, rms <at> gnu.org
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> More radically, Emacs could also stop installing etags, and advise
> >> people to run ctags -e instead, where this is the other ctags.
> >
> > To this I object, since the master source of the format of the TAGS
> > files must be in Emacs, and what better to document this format than
> > to maintain and distribute etags the program.
> 
> As a counter-point, we distribute parsers for quite a few formats where
> the master source of the format is not in emacs.git.  They don't usually
> have the same legacy as ctags, admittedly.
> 
> How about adding a --without-ctags flag, for those of us that never have
> a need for it?

Is that worth the extra maintenance headaches?

If we think our ctags is not being used (I suggested to query a few
distros about that), I'd rather drop it entirely, first by adding the
--ctags option to etags, and later maybe removing the ctags-only code
from etags.c.

Message #69 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 76322 <at> debbugs.gnu.org, eggert <at> cs.ucla.edu, rms <at> gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Tue, 11 Mar 2025 14:45:53 +0200

> Date: Tue, 11 Mar 2025 04:34:55 +0200
> Cc: 76322 <at> debbugs.gnu.org, eggert <at> cs.ucla.edu
> From: Dmitry Gutov <dmitry <at> gutov.dev>
> 
> On 10/03/2025 17:40, Eli Zaretskii wrote:
> > AFAIR, the ctags output is used by other editors (Vim, I think?).
> > Whether that, or the fact that we shipped ctags since forever, is a
> > significant enough reason to keep it in the tree and in the tarballs,
> > is another question.  Maybe we should ask downstream distros how they
> > feel about dropping ctags.
> 
>  From what I'm reading, a lot of distros have replaced their 'ctags' 
> package with universal-ctags:
> 
> https://repology.org/project/universal-ctags/versions
> 
> It's the main ctags fork out there, and a pretty active one.

Yes, that's what I had in mind.

Message #72 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 76322 <at> debbugs.gnu.org, rms <at> gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Tue, 11 Mar 2025 10:01:38 -0700

[Message part 1 (text/plain, inline)]

On 2025-03-11 05:11, Eli Zaretskii wrote:
> If we think our ctags is not being used (I suggested to query a few
> distros about that), I'd rather drop it entirely, first by adding the
> --ctags option to etags, and later maybe removing the ctags-only code
> from etags.c.

That would be simpler, yes. Proposed patch attached.

My impression is that distros generally prefer Universal Ctags these 
days, though you can use Exuberant Ctags if you're old-fashioned or 
Emacs ctags if your even older-fashioned. On Fedora, 'ctags' is 
Universal Ctags; on Ubuntu all three variants are available as options 
and /usr/bin/ctags is a symlink to /etc/alternatives/ctags which in turn 
can link to /usr/bin/ctags-universal or /usr/bin/ctags-exuberant or 
/usr/bin/ctags.emacs.

To some extent it's a chicken-and-egg problem: if we keep shipping ctags 
some distros will keep making it available as an option. But there's no 
real need for us to keep maintaining the third (and oldest) variant.

[0001-Remove-ctags-program.patch (text/x-patch, attachment)]

Message #75 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Eli Zaretskii <eliz <at> gnu.org>
Cc: 76322 <at> debbugs.gnu.org, rms <at> gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Tue, 11 Mar 2025 14:02:43 -0400

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> On 2025-03-11 05:11, Eli Zaretskii wrote:
>> If we think our ctags is not being used (I suggested to query a few
>> distros about that), I'd rather drop it entirely, first by adding the
>> --ctags option to etags, and later maybe removing the ctags-only code
>> from etags.c.
>
> That would be simpler, yes. Proposed patch attached.
>
> My impression is that distros generally prefer Universal Ctags these
> days, though you can use Exuberant Ctags if you're old-fashioned or
> Emacs ctags if your even older-fashioned. On Fedora, 'ctags' is
> Universal Ctags; on Ubuntu all three variants are available as options
> and /usr/bin/ctags is a symlink to /etc/alternatives/ctags which in turn
> can link to /usr/bin/ctags-universal or /usr/bin/ctags-exuberant or
> /usr/bin/ctags.emacs.
>
> To some extent it's a chicken-and-egg problem: if we keep shipping ctags
> some distros will keep making it available as an option. But there's no
> real need for us to keep maintaining the third (and oldest) variant.

Makes sense to me.

Message #78 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Richard Stallman <rms <at> gnu.org>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 76322 <at> debbugs.gnu.org, eliz <at> gnu.org, eggert <at> cs.ucla.edu
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Wed, 12 Mar 2025 00:26:58 -0400

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >  From what I'm reading, a lot of distros have replaced their 'ctags' 
  > package with universal-ctags:

  > https://repology.org/project/universal-ctags/versions

Can you summarize the important differences between this and our ctags?

Can you summarize the important differences between this and etags?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Message #81 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: rms <at> gnu.org, Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 76322 <at> debbugs.gnu.org, eliz <at> gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Wed, 12 Mar 2025 00:18:11 -0700

On 2025-03-11 21:26, Richard Stallman wrote:
> Can you summarize the important differences between this and our ctags?
> Can you summarize the important differences between this and etags?

No. And as I haven't used ctags or etags (of any flavor) for decades, I 
would not be a good choice to investigate details.

That being said, by all reports Universal Ctags is better than what's 
shipped with GNU Emacs. Of the recent discussions I found on this topic, 
nobody recommended Emacs ctags; everybody wanted Universal Ctags, or 
something fancier like GNU Global or a language server, and some users 
were frustrated when Emacs ctags was installed by mistake.

Apparently Emacs ctags is a revenant that costs us more in maintenance 
effort than it's worth, which is why I suggested removing it.

Message #84 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, rms <at> gnu.org,
 Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 76322 <at> debbugs.gnu.org, eliz <at> gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Wed, 12 Mar 2025 01:48:07 -0700

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> On 2025-03-11 21:26, Richard Stallman wrote:
>> Can you summarize the important differences between this and our ctags?
>> Can you summarize the important differences between this and etags?
>
> No. And as I haven't used ctags or etags (of any flavor) for decades, I
> would not be a good choice to investigate details.

Two important differences:
- Exuberant Ctags supports 160 languages, while our ctags supports 30.
- Exuberant Ctags has much better support for new language features.

Message #87 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 76322 <at> debbugs.gnu.org, rms <at> gnu.org, stefankangas <at> gmail.com
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Wed, 12 Mar 2025 14:15:25 +0200

> Date: Tue, 11 Mar 2025 10:01:38 -0700
> Cc: 76322 <at> debbugs.gnu.org, rms <at> gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
> From: Paul Eggert <eggert <at> cs.ucla.edu>
> 
> On 2025-03-11 05:11, Eli Zaretskii wrote:
> > If we think our ctags is not being used (I suggested to query a few
> > distros about that), I'd rather drop it entirely, first by adding the
> > --ctags option to etags, and later maybe removing the ctags-only code
> > from etags.c.
> 
> That would be simpler, yes. Proposed patch attached.

Thanks, but please also update the test suite in test/manual/etags/.
(For now, I'd prefer to leave the ctags tests intact, and just update
the makefile to invoke "etags --ctags".)

> My impression is that distros generally prefer Universal Ctags these 
> days, though you can use Exuberant Ctags if you're old-fashioned or 
> Emacs ctags if your even older-fashioned. On Fedora, 'ctags' is 
> Universal Ctags; on Ubuntu all three variants are available as options 
> and /usr/bin/ctags is a symlink to /etc/alternatives/ctags which in turn 
> can link to /usr/bin/ctags-universal or /usr/bin/ctags-exuberant or 
> /usr/bin/ctags.emacs.
> 
> To some extent it's a chicken-and-egg problem: if we keep shipping ctags 
> some distros will keep making it available as an option. But there's no 
> real need for us to keep maintaining the third (and oldest) variant.

OK, let's wait with installing the patch until this discussion reaches
its conclusion, perhaps after more people chime in.

Message #90 received at 76322 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: dmitry <at> gutov.dev, eggert <at> cs.ucla.edu, rms <at> gnu.org, 76322 <at> debbugs.gnu.org
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Thu, 13 Mar 2025 11:37:09 +0200

> From: Stefan Kangas <stefankangas <at> gmail.com>
> Date: Wed, 12 Mar 2025 01:48:07 -0700
> Cc: 76322 <at> debbugs.gnu.org, eliz <at> gnu.org
> 
> Paul Eggert <eggert <at> cs.ucla.edu> writes:
> 
> > On 2025-03-11 21:26, Richard Stallman wrote:
> >> Can you summarize the important differences between this and our ctags?
> >> Can you summarize the important differences between this and etags?
> >
> > No. And as I haven't used ctags or etags (of any flavor) for decades, I
> > would not be a good choice to investigate details.
> 
> Two important differences:
> - Exuberant Ctags supports 160 languages, while our ctags supports 30.
> - Exuberant Ctags has much better support for new language features.

This sounds rather dismissive and lacking important details.

Our etags are a subprogram of Emacs, and thus need support mainly the
languages that Emacs supports.  How many of the languages for which
Emacs has a major mode are not supported by our etags?

As for support for "new language features", is that relevant for the
Emacs commands that use TAGS tables?  If so, can you tell the details?
which language features do we lack?

On the flip side, our etags seems to be about 10 times faster, at
least when running it on lib/, src/, and lisp/ directories of the
Emacs source tree (~1.5 sec vs ~16 sec on my system).

Anyway, not sure how this is relevant to the issue at hand.  I don't
think anyone is suggesting that we drop etags (and if someone does,
then I object), and whether we should drop ctags as a separate program
doesn't depend on how many languages it supports, because the main
arguments for dropping our ctags are that (a) Emacs doesn't use the
ctags format of the tags tables, and (b) that most distros package
Universal ctags as their preferred version.

Message #95 received at 76322-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 76322-done <at> debbugs.gnu.org, rms <at> gnu.org, stefankangas <at> gmail.com
Subject: Re: bug#76322: Make ctags a thin wrapper around etags
Date: Sat, 22 Mar 2025 11:59:18 -0700

On 2025-03-12 05:15, Eli Zaretskii wrote:
> OK, let's wait with installing the patch until this discussion reaches
> its conclusion, perhaps after more people chime in.

Seems like a long enough wait, so I installed the patch on master and am 
closing the bug report. Thanks for the review.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.