Subject: bug#73581: 29.4; Gnus: Error doing a search on nnmaildir with gnus-search-find-grep

From: Visuwesh <[email-masked]>
Date: Thu, 10 Apr 2025 18:15:31 +0530

[வியாழன் ஏப்ரல் 10, 2025] Daniel Cerqueira wrote:

> I am coming back to this issue.  It still is unresolved.
>
> I have made some progress, but still I get an error.  Here is what I
> have done:
>
>
> My search configuration:
>
> (setq gnus-search-default-engines
>       '((nnimap . gnus-search-imap)
>         ;; (nnmaildir . gnus-search-find-grep)
>         ;; (nnselect . gnus-search-nnselect)
>         (nnselect . gnus-search-find-grep)))
>
> (setq gnus-select-method
>       `(nnmaildir "email"
>                   (directory ,(expand-file-name "~/Mail"))
>                   (remove-prefix ,(expand-file-name "~/Mail"))
>                   (nnmaildir-directory ,(expand-file-name "~/Mail"))
>                   (gnus-search-engine gnus-search-find-grep)))
>
>
> My error message:
>
> nnselect-generate-artlist: gnus-search-run-query on ((search-query-spec (query . parabéns) (raw)) (search-group-spec (nnmaildir:email friends))) gave error (wrong-type-argument stringp nil)
> nnselect-generate-artlist: nil on nil gave error (void-function nil) [2 times]
> gnus-request-group: Wrong type argument: stringp, nil

There are a number of problems with gnus-search-find-grep and its
interaction with a nnmaildir virtual server:

  1. It assumes that every mail server uses the server option
     SERVER-directory when nnmaildir uses only directory.
  2. It might not be obvious but your query should be grep:parabéns
     IIUC.
  3. However, (2) won't solve the issue because of the problematic find
     command that I talked about in the last mail.  You can work around
     this.
  4. Its conversion of nnmaildir mail filename to article number is not
     correct.

With all that accounted for, I ended up with the (ugly?) patch below.

diff --git a/lisp/gnus/gnus-search.el b/lisp/gnus/gnus-search.el
index 41915a0e3c0..fa5eaf936b7 100644
--- a/lisp/gnus/gnus-search.el
+++ b/lisp/gnus/gnus-search.el
@@ -1980,8 +1980,11 @@ gnus-search-run-search
 				      &optional groups)
   "Run find and grep to obtain matching articles."
   (let* ((method (gnus-server-to-method server))
-	 (sym (intern
-	       (concat (symbol-name (car method)) "-directory")))
+         ;; `nnmaildir' backends use the 'directory' setting.
+	 (sym (if (eq (car method) 'nnmaildir)
+                  'directory
+                (intern
+	         (concat (symbol-name (car method)) "-directory"))))
 	 (directory (cadr (assoc sym (cddr method))))
 	 (regexp (alist-get 'grep query))
 	 (grep-options (slot-value engine 'grep-options))
@@ -1996,6 +1999,11 @@ gnus-search-run-search
      #'vconcat
      (mapcar (lambda (x)
 	       (let ((group x)
+                     (nnmaildir-strip-regexp
+                      (concat (regexp-quote (gnus-group-real-name x))
+                              "/\\(?:new\\|cur\\)/"))
+                     (nnmaildir-strip-repl
+                      (concat (gnus-group-real-name x) "/"))
 		     artlist)
 		 (gnus-message 7 "Searching %s using find-grep..."
 			       (or group server))
@@ -2003,7 +2011,7 @@ gnus-search-run-search
 		   (set-buffer buffer)
 		   (if (> gnus-verbose 6)
 		       (pop-to-buffer (current-buffer)))
-		   (cd directory)    ; Using relative paths simplifies
+		   (cd directory)      ; Using relative paths simplifies
 					; postprocessing.
 		   (let ((group
 			  (if (not group)
@@ -2029,28 +2037,49 @@ gnus-search-run-search
 		     (save-excursion
 		       (apply
 			#'call-process "find" nil t
-			"find" group "-maxdepth" "1" "-type" "f"
-			"-name" "[0-9]*" "-exec"
-			(slot-value engine 'grep-program)
-			`("-l" ,@(and grep-options
-				      (split-string grep-options "\\s-" t))
-			  "-e" ,regexp "{}" "+"))))
+			"find"
+                        (delq
+                         nil   ; nil can be present if GREP-OPTIONS=nil.
+                         `(,@(if (eq (car method) 'nnmaildir)
+                                 (list (concat group "/cur")
+                                       (concat group "/new"))
+                               group)
+                           "-maxdepth" "1" "-type" "f"
+			   "-name" "[0-9]*" "-exec"
+			   ,(slot-value engine 'grep-program)
+			   "-l" ,@(and grep-options
+				       (split-string grep-options "\\s-" t))
+			   "-e" ,regexp "{}" "+")))))
 
 		   ;; Translate relative paths to group names.
 		   (while (not (eobp))
-		     (let* ((path (split-string
-				   (buffer-substring
-				    (point)
-				    (line-end-position))
-				   "/" t))
-			    (art (string-to-number (car (last path)))))
+		     (let* ((path (buffer-substring
+				   (point)
+				   (line-end-position)))
+                            (path (split-string (if (eq (car method) 'nnmaildir)
+                                                    (replace-regexp-in-string
+                                                     nnmaildir-strip-regexp
+                                                     nnmaildir-strip-repl
+                                                     path)
+                                                  path)
+                                                "/" t))
+			    (art (car (last path))))
 		       (while (string= "." (car path))
 			 (setq path (cdr path)))
 		       (let ((group (mapconcat #'identity
 					       (cl-subseq path 0 -1)
 					       ".")))
 			 (push
-			  (vector (gnus-group-full-name group server) art 0)
+			  (vector (gnus-group-full-name group server)
+                                  ;; FIXME: Code duplication.  This is
+                                  ;; copied over from
+                                  ;; `gnus-search-indexed-parse-output'.
+                                  (if (string-match-p "\\`[[:digit:]]+\\'" art)
+		                      (string-to-number art)
+		                    (nnmaildir-base-name-to-article-number
+		                     (substring art 0 (string-search ":" art))
+		                     group (string-remove-prefix "nnmaildir:" server)))
+                                  0)
 			  artlist))
 		       (forward-line 1)))
 		   (gnus-message 7 "Searching %s using find-grep...done"