Subject: etags.el xref-location-marker does not handle TAGS references to .el.gz files
From: Pierre Rouleau <[email-masked]>
To: [email-masked]
Date: Fri, 6 Nov 2020 18:22:46 -0500
List-Id: <debbugs-submit.debbugs.gnu.org>

This problem was detected in emacs 26.3, but is also present in emacs 27.1,
according to the code posted inside
https://github.com/emacs-mirror/emacs/blob/master/test/manual/etags/el-src/emacs/lisp/progmodes/etags.el#L2139

Problem description follows:

----------------------------BUG DESCRIPTION [ ------------------------------

The current version of etags distributed with emacs 27.0 is capable of
parsing compressed files like something.el.gz but unfortunately the
reference inside the generated TAGS file uses the name "something.el"
instead of the complete file name "something.el.gz".  The same is true
for the other compressed files (.bz2, .xz and .lzma).

When trying to use the etags xref back-end to move point to a definition
inside such a file, the xref-find-definitions command will fail because the
TAGS file identifies the .el file instead of the .el.gz file.

xref-find-definitions fails because it only checks if the uncompressed
file exists, it does not try to see if the compressed file exists.

The issue is inside etags.el xref-location-marker method.

The current code is:

(cl-defmethod xref-location-marker ((l xref-etags-location))
  (with-slots (tag-info file) l
    (let ((buffer (find-file-noselect file)))
      (with-current-buffer buffer
        (save-excursion
          (etags-goto-tag-location tag-info)
          (point-marker))))))

One could consider that the issue is inside the etags utility.

An alternative would be to provide Emacs with the ability to check
for the presence of the .el file and if it is not present look for the
equivalent compressed files.

Here's a proposal for a solution:

(defun etags-file-or-compressed-file-for (fname)
  "Return the valid file name for FNAME.
Check if FNAME is an existing file name, if not
try FNAME appended with the following compression extensions:
- \".gz\", the extension of compressed files created by gzip
- \".bz2\", the extension for compressed files created by bzip2
- \".xz\", the extension for compressed files created by xz
- \".lzma\", the extension for compressed files created by xz.

Return the file that exists or nil if nothing found."
  (let ((fpath nil))
    (cl-dolist (ext '(""
                      ".gz"
                      ".bz2"
                      ".xz"
                      ".lzma"))
      (setq fpath (concat fname ext))
      (when (file-exists-p fpath)
        (cl-return fpath)))))

(cl-defmethod xref-location-marker ((l xref-etags-location))
  (with-slots (tag-info file) l
    (let (buffer
          (fname (pel-file-or-compressed-file-for file)))
      (if fname
          (setq buffer (find-file-noselect fname))
        (user-error "file %s (or .gz, .bz2, .xz, .lzma) does not exist"
file))
      (with-current-buffer buffer
        (save-excursion
          (etags-goto-tag-location tag-info)
          (point-marker))))))

------------------------------ END OF DESCRIPTION ]
--------------------------------

In GNU Emacs 26.3 (build 1, x86_64-apple-darwin18.6.0)
 of 2019-08-30 built on Mojave.local
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --disable-dependency-tracking --disable-silent-rules
 --enable-locallisppath=/usr/local/share/emacs/site-lisp
 --infodir=/usr/local/Cellar/emacs/26.3/share/info/emacs
 --prefix=/usr/local/Cellar/emacs/26.3 --with-gnutls --without-x
 --with-xml2 --without-dbus --with-modules --without-ns
 --without-imagemagick'

Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB MODULES THREADS

Important settings:
  value of $LANG: en_CA.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail tool-bar rmail-loaddefs mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
regexp-opt rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
term/xterm xterm time-date elec-pair mule-util tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads kqueue
multi-tty make-network-process emacs)

Memory information:
((conses 16 96177 5831)
 (symbols 48 19813 1)
 (miscs 40 33 96)
 (strings 32 28253 1011)
 (string-bytes 1 748198)
 (vectors 16 11977)
 (vector-slots 8 455846 6474)
 (floats 8 48 566)
 (intervals 56 197 0)
 (buffers 992 11))

;; ---------------------------

Thank you,

/Pierre Rouleau