From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 21 03:11:06 2023 Received: (at submit) by debbugs.gnu.org; 21 Jun 2023 07:11:06 +0000 Received: from localhost ([127.0.0.1]:60516 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qBrzc-0000Ji-SS for submit@debbugs.gnu.org; Wed, 21 Jun 2023 03:11:06 -0400 Received: from lists.gnu.org ([209.51.188.17]:34584) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qBrza-0000JZ-3a for submit@debbugs.gnu.org; Wed, 21 Jun 2023 03:11:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qBrzZ-00035U-I9 for bug-gnu-emacs@gnu.org; Wed, 21 Jun 2023 03:11:01 -0400 Received: from dsemy.com ([46.23.89.208]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qBrzV-0008Im-O3 for bug-gnu-emacs@gnu.org; Wed, 21 Jun 2023 03:11:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=ezg91LrMacUhe /jQzo23W6OPmsE/c7PZbydfT7cBCBU=; h=date:subject:to:from; d=dsemy.com; b=fmV1b2whqs18xabo/hpO9sef3A7kzsifVqIJ+YxRF/tvweZ3MyqCsqzmrTK60P7hkszK +75wRgBBuzG+0ZfIIWRem1tIRvVQl2WQUr8FGhED7K9Htthp66NXzst1wyuiixlLSPX4PJ 2A4OnTK6Ca19E0DB3I5cx434U6x3aJpTj1s4VmRACtjh2IZiR+8rVCSwPY9/q3o9On2Gjv HTnqUHA4+GajuOQCc6CQJKQBSMbDrAubwQ7yCGSKN2oPu0R/iWK3Msm8hoZ427GEyCcEC8 2AUcuLv+JXurbjyDGtDQnhpyWF5zqrt/84ecRrAmsnYHBxqkQpo2oaxc5hSTp7Ig== Received: from coldharbour.home.arpa (bzq-109-64-87-185.red.bezeqint.net [109.64.87.185]) by dsemy.com (OpenSMTPD) with ESMTPSA id 64b8f1d6 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 21 Jun 2023 09:10:48 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id 0c848fa6 for ; Wed, 21 Jun 2023 07:08:51 +0000 (UTC) From: Daniel Semyonov To: bug-gnu-emacs@gnu.org Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) Date: Wed, 21 Jun 2023 10:08:51 +0300 Message-ID: <87v8fhmgvw.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=46.23.89.208; envelope-from=daniel@dsemy.com; helo=dsemy.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_FILL_THIS_FORM_SHORT=0.01, T_SCC_BODY_TEXT_LINE=-0.01, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) --=-=-= Content-Type: text/plain Tags: patch This patch adds a back end for Atom feeds to Gnus, and documents it. In GNU Emacs 30.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.24.38, cairo version 1.16.0) of 2023-06-21 built on coldharbour Repository revision: 0273cb5e6dbd6c5111f3352b7777b4efc769ba14 Repository branch: master Windowing system distributor 'The X.Org Foundation', version 11.0.12101008 System Description: Void Linux Configured using: 'configure -C --prefix=/opt/Emacs --with-x --with-x-toolkit=gtk3 --without-gsettings --without-dbus --with-xinput2 --with-small-ja-dic --with-native-compilation' --=-=-= Content-Type: text/patch Content-Disposition: attachment; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch >From 6085ee8139cc3d815a5028babb4daf438df9d06b Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnatom.el: New file. * doc/misc/gnus.texi: * etc/NEWS: Document nnatom --- doc/misc/gnus.texi | 42 +++ etc/NEWS | 7 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 705 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 755 insertions(+) create mode 100644 lisp/gnus/nnatom.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 8d25e868c8a..95eba21c4dd 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -17250,6 +17251,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17494,6 +17496,46 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@xref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds +as Gnus servers (with a single group), as you would with any other +method, by supplying the location of the feed as the server address. +Note, however, that the server address shouldn't be prefixed with +@code{http://} or @code{https://}. + +The @code{nnatom} back end saves a server data file in the +@code{atom} sub-directory of @code{gnus-directory} for each feed. + +The @code{nnatom} back end generates an article part for both the +summary and content of each article in the feed. + +@code{nnatom} has been designed to be very modular, and theoretically +supports many features which aren't available in the Atom Syndication +Format, in an effort to reduce the work required to build back ends +for other web feed formats. + +Every parsing step is handled by a function stored in a server +variable; these are all called @code{nnatom-read-*-function}, and +their requirements are detailed in their docstrings. + +The function responsible for printing the content of each article part +is stored in a server variable (@code{nnatom-print-content-function}). +The default function only handles (X)HTML and plain text content. + +@defmac nnatom-define-basic-backend-interface backend +Define server variables expected by @code{nnatom} and import its back +end functions for BACKEND. This macro (or code similar to it) should +be used by any inheriting back end. +@end defmac + + @node Other Sources @section Other Sources diff --git a/etc/NEWS b/etc/NEWS index 77ca749ccc3..46e6aeb34dc 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -324,6 +324,13 @@ distracting and easily confused with actual code, or a significant early aid that relieves you from moving the buffer or reaching for the mouse to consult an error message. +** Gnus + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + ** Python Mode --- diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index fc8518512ee..d35d709c448 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1343,6 +1343,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..3da4644de20 --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,705 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format , or any +;; other type of feed with customized parsing functions. + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'gv) + (require 'subr-x)) + +(require 'gnus) +(require 'nnheader) +(require 'nnoo) +(require 'gnus-group) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'gnus) + +(nnoo-declare nnatom) + +(defvoo nnatom-backend 'nnatom + "Symbol which identifies this backend.") + +(defvoo nnatom-status-string nil + "Last status message reported by this backend.") + +(defsubst nnatom--backend-prefix (backend) + (concat (symbol-name backend) ":")) + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "^https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-readable-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (if (libxml-available-p) + (libxml-parse-xml-region (point-min) (point-max)) + (require 'xml) + (car (xml-parse-region (point-min) (point-max))))))))) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) + (setq data (dom-by-tag data 'entry))) + (and data `(,(car data) . , (setq data (cdr data))))) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) + +(defun nnatom--read-article-or-group-author (article-or-group) + "Return the author of ARTICLE-OR-GROUP, or nil." + (let* ((author (dom-child-by-tag article-or-group 'author)) + (name (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (or (and name email (format "%s <%s>" name email)) + name email))) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((rel (make-vector 5 0))) ; [ALTERNATE RELATED SELF ENCLOSURE VIA] + (mapcan + (lambda (link) + (when-let + (((and (consp link) (eq (car link) 'link))) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf (aref rel 1))) + (format " %s" (aref rel 1))))) + ("self" + (concat "More" + (and (< 1 (cl-incf (aref rel 2))) + (format " %s" (aref rel 2))))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf (aref rel 3))) + (format " %s" (aref rel 3))))) + ("via" + (concat "Source" + (and (< 1 (cl-incf (aref rel 4))) + (format " %s" (aref rel 4))))) + (_ (if-let ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat "Link" + (and (< 1 (cl-incf (aref rel 0))) + (format " %s" (aref rel 0)))))))))) + `(,(vector (cdr (assq 'href attrs)) + (concat label ":") (concat "[" label "]"))))) + article))) + +(defsubst nnatom--read-part (part type links) + `(,part + ("Content-Type" . ,(concat "text/" (or type "plain"))) + ,(and (or (string= type "html") (string= type "xhtml")) + 'html) + ,(and links 'links))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (dom-attr summary 'type)) + (summary (dom-text summary)) + (summary (unless (string-blank-p summary) summary)) + (content (dom-child-by-tag article 'content)) + (ctype (dom-attr content 'type)) + (content (dom-text content)) + (content (unless (string-blank-p content) content)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push (nnatom--read-part summary stype nil) parts)) + (push (nnatom--read-part content ctype t) parts) + (or st (push (nnatom--read-part summary stype nil) parts)) + parts) + ((setq parts (or summary content)) + `(,(nnatom--read-part parts (if content ctype stype) t))) + (t '((nil ("Content-Type" . "text/html") html links)))))) + +;;;; Feed I/O: + +(defvoo nnatom-read-feed-function #'nnatom--read-feed + "Function returning a Lisp object representing a feed (or part of it). +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnatom-read-group-function').") + +(defvoo nnatom-read-group-function #'nnatom--read-group + "Function returning a cons cell of a group and remaining data from a feed.") + +(defvoo nnatom-read-article-function #'nnatom--read-article + "Function returning a cons cell of an article and remaining data from a group. +It should accept a two arguments, a Lisp object representing a feed, +and a flag indicating whether the last article was stale (not new or updated).") + +(defvoo nnatom-read-title-function #'nnatom--read-title + "Function returning the title of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-description-function #'nnatom--read-description + "Function returning the description of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-group-author-function #'nnatom--read-article-or-group-author + "Function returning the author of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-id-function #'nnatom--read-id + "Function returning the ID of an article. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-subject-function #'nnatom--read-subject + "Function returning the subject of an article (a string). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + "Function returning the publish date of an article (a time value). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-update-date-function #'nnatom--read-update + "Function returning the update date of an article (a time value). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-author + "Function returning the author of an article (a string). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-links-function #'nnatom--read-links + "Function returning all links contained in an article. +For the default `nnatom-print-content-function', it should return a +list of links, where each link is a vector of the form +[LINK LABEL HTML-LABEL], where: +- LINK is the link. +- LABEL is a label used for LINK (printed \"LABEL: LINK\"). +- HTML-LABEL is a label used for LINK, but only if the type of the + part in which LINK is printed is \"html\" or \"xhtml\" (printed \"[LABEL]\"). +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-parts-function #'nnatom--read-parts + "Function returning an alist associating parts of an article to their headers. +For the default `nnatom-print-content-function', each part should be +a string. Otherwise, it can be any Lisp object. +The \"headers\" of each part should be a list where each element is +either a cons of a MIME header (a string) and its value (a string) or +any other Lisp object. MIME headers will be printed, the rest will be +passed on to `nnatom-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +- `html', if present, will format the part as HTML. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnatom-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnatom-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnatom--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnatom-backend) + "nn") + (gnus-newsgroup-savable-name server)) + gnus-directory)) + +(defun nnatom--read-server (server) + "Read SERVER's information from `nnatom-directory'." + (if-let ((f (nnatom--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnatom-servers)) + (nnheader-report nnatom-backend "Can't read %s" server))) + +(defun nnatom--write-server (server) + "Write SERVER's information to `nnatom-directory'." + (if-let ((f (nnatom--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnatom-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n" + (prin1-to-string s)) + t) + t) + (nnheader-report nnatom-backend "Can't write %s" f))) + +(defsubst nnatom--server-address (server) + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnatom-backend 'address) + (cddr (gnus-server-to-method + (concat (nnatom--backend-prefix + nnatom-backend) + server))))) + server)) + server)) + +(defun nnatom--parse-feed (feed &optional group) + "Parse FEED into a new or existing server. +Optionally, only parse GROUP." + (let* ((feed (nnatom--server-address feed)) + (prefix (nnatom--backend-prefix nnatom-backend)) + (s (or (gethash feed nnatom-servers) + (nnatom--read-server feed) t)) + (g (or (and (hash-table-p s) (gethash group s)) `[,group])) + (name group) ; (Maybe) fake name + (group (or (aref g 0) name)) ; Real name + data) + (when (setq data (funcall nnatom-read-feed-function feed group) + s (or (gethash feed nnatom-servers) + (make-hash-table :test 'equal))) + (while-let ((cg (or (and name `(,data)) + (funcall nnatom-read-group-function data))) + ((progn (setq data (cdr cg)) t)) + (cg (car cg))) + (let* ((name (funcall nnatom-read-title-function cg)) ; Real name + (group (gethash name nnatom-group-names name)) ; (Maybe) fake name + (info (gnus-get-info (concat prefix group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnatom-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnatom-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnatom-read-article-function cg stale)) + ((progn (setq cg (cdr article)) t)) + (article (car article))) + (when-let ((id (funcall nnatom-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnatom-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnatom-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (cl-incf max)))) + (publish (funcall nnatom-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnatom-read-author-function article) + group-author group) + ,(or (funcall nnatom-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnatom-read-links-function article) + ,(funcall nnatom-read-parts-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnatom-servers)))) + +;;;; Gnus backend functions: + +(defvoo nnatom-group nil + "Name of the current group.") + +(defvoo nnatom-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnatom-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers.") + +(defvoo nnatom-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnatom-group-article-min-num 1 + "Minimum article number for the current group.") + +(defvar nnatom-date-format "%F %X" + "Format of displayed dates.") + +(nnoo-define-basics nnatom) + +(defun nnatom--group-data (group &optional server) + (let ((s (gethash server nnatom-servers)) c) + (or (and (hash-table-p s) (gethash group s)) + (and (setq c (nnoo-current-server nnatom-backend)) + (setq s (gethash c nnatom-servers)) + (hash-table-p s) (gethash group s)) + (catch :stop (maphash (lambda (n s) + (or (string= n server) + (string= n c) + (when-let (((hash-table-p s)) + (g (gethash group s))) + (throw :stop g)))) + nnatom-servers))))) + +(defun nnatom-retrieve-article (article group) + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string nnatom-date-format (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnatom-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnoo-current-server nnatom-backend))) + (g (or (nnatom--group-data group server) + `[ nil ,nnatom-group-article-ids ,nnatom-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnatom-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnatom-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnatom-backend "Group %s not found" (or group "")))) + +(deffoo nnatom-open-server (server &optional defs backend) + (let ((backend (or backend 'nnatom)) + (a (nnatom--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (nnatom--read-server a)) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash group g nnatom-group-names))) + s)) + (or s (file-writable-p (nnatom--server-file a)) + (and (nnoo-close-server (or backend 'nnatom) server) + (nnheader-report + nnatom-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnatom-request-close () + (maphash (lambda (server _) + (nnatom--write-server + (nnatom--server-address server))) + nnatom-servers) + (setq nnatom-servers (make-hash-table :test 'equal) + nnatom-status-string nil) + t) + +(defun nnatom--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, possibly with LINKS added." + (let ((html (memq 'html attributes)) + (links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and html "") + (and content (format "%s\n\n" content)) + (and links html "

") + (and links + (if html + (mapconcat + (lambda (link) + (format "%s" (aref link 0) (aref link 2))) + links " ") + (mapconcat + (lambda (link) + (format "%s %s\n" (aref link 1) (aref link 0))) + links "\n"))) + (and links html "

") + (and html ""))))) + +(defvoo nnatom-print-content-function #'nnatom--print-content + "Function returning a single piece of content (a string). +It should accept three arguments, a part and its attributes (as returned +by `nnatom-read-parts-function'), and links.") + +(defsubst nnatom--print-part (content headers mime links) + (concat + (mapconcat (lambda (header) + (when-let (((consp header)) + (m (car header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnatom-print-content-function content headers links))) + +(deffoo nnatom-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnoo-current-server nnatom-backend))) + (g (or (nnatom--group-data group server) + (and (setq group nnatom-group) + `[ nil ,nnatom-group-article-ids + ,nnatom-group-articles + ,nnatom-group-article-max-num + ,nnatom-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((boundary (format "-_%s_-" nnatom-backend)) + (links (aref a 5)) + (parts (aref a 6)) + (multi (length> parts 1)) + (mime '( "Content-Type" "Content-Disposition" + "Content-Transfer-Encoding"))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnatom-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + "MIME-Version: 1.0\n" + (if multi + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary) + (prog1 (nnatom--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (let ((headers (cdr part))) + (format "--%s\n%s\n" boundary + (nnatom--print-part + (car part) headers mime links)))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnatom-backend "No such article"))) + +(deffoo nnatom-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnoo-current-server nnatom-backend))) + (g (or (if fast (nnatom--group-data group server) + (setq server (nnatom--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnatom-group group + nnatom-group-article-ids (aref g 1) + nnatom-group-articles (aref g 2) + nnatom-group-article-max-num (aref g 3) + nnatom-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnatom-group-article-ids) + nnatom-group-article-min-num + nnatom-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnatom-backend "Group %s not found" group)))) + +(deffoo nnatom-close-group (group &optional server) + (and (string= group nnatom-group) + (setq nnatom-group nil + nnatom-group-article-ids (make-hash-table :test 'equal) + nnatom-group-articles (make-hash-table :test 'eql) + nnatom-group-article-max-num 0 + nnatom-group-article-min-num 1)) + (setq server (or server (nnoo-current-server nnatom-backend))) + (nnatom--write-server server)) + +(deffoo nnatom-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnatom--parse-feed + (or server (nnoo-current-server nnatom-backend)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnatom-request-post (&optional _server) + (nnheader-report nnatom-backend "%s is a read only backend" nnatom-backend)) + +;;;; Optional back end functions: + +(deffoo nnatom-retrieve-groups (_groups &optional server) + (nnatom-request-list (or server (nnoo-current-server nnatom-backend))) + 'active) + +(deffoo nnatom-request-type (_group &optional _article) + 'unknown) + +(deffoo nnatom-request-group-description (group &optional server) + (when-let ((server (or server (nnoo-current-server nnatom-backend))) + (g (nnatom--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnatom-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnoo-current-server nnatom-backend))) + (s (gethash server nnatom-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnatom-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnoo-current-server nnatom-backend))) + (s (gethash server nnatom-servers)) + (g (or (nnatom--group-data group server) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnatom-group-names) + (remhash group s) + (and (string= group nnatom-group) + (setq nnatom-group new-name)) + t)) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +;;;; Utilities: + +(defmacro nnatom-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnatom-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnatom-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnatom-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnatom-servers) + (defvoo ,(nnoo-symbol backend 'group-names) + (make-hash-table :test 'equal) nil nnatom-group-names) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnatom-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnatom-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnatom-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnatom-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnatom-open-server server defs ',backend)) + (nnoo-import ,backend (nnatom)))) + +(provide 'nnatom) + +;;; nnatom.el ends here -- 2.41.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 21 05:52:54 2023 Received: (at 64202) by debbugs.gnu.org; 21 Jun 2023 09:52:54 +0000 Received: from localhost ([127.0.0.1]:60683 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qBuWE-0004iY-4h for submit@debbugs.gnu.org; Wed, 21 Jun 2023 05:52:54 -0400 Received: from dsemy.com ([46.23.89.208]:35203) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qBuWA-0004iE-2g for 64202@debbugs.gnu.org; Wed, 21 Jun 2023 05:52:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=/nP0HlV6tECzv 7/IfSoaQBTwF+The7V2coyRtYcOlsM=; h=date:references:in-reply-to: subject:to:from; d=dsemy.com; b=U3DVx9gRzmx6N5rBb6KbVfkcap+0crjnE5x7ws pqKwgYOZi2mNtOOPNulFeiEelo7vu0AtnzsCJHq44JniWhctweX4Qi4sVCEV/b/rxNbLBN aNTnR9ciJLMn2+dCTKIWv2PdGc3G3HQGtWoKpJeR/KsAAYUHQaFbjsYNJHjL1yWSGQ+PWW MG/JqZho8Vf/iTPKJRpre/dAiVAV1hmw9tJHZSsoAnnT4Abw/4bw2mdcuj9JW4zxycfu75 bpYOl9JGn83FQQjdXdjzZ+aEI9zw6uEJQoETkrncAPMpDXcFsm2sz/jHz+4E4suWPPt8mu hJSB4pqTO04nWtejo2i1+60A== Received: from coldharbour.home.arpa (bzq-109-64-87-185.red.bezeqint.net [109.64.87.185]) by dsemy.com (OpenSMTPD) with ESMTPSA id 2c005eb1 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for <64202@debbugs.gnu.org>; Wed, 21 Jun 2023 11:52:43 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id 49483744 for <64202@debbugs.gnu.org>; Wed, 21 Jun 2023 09:50:48 +0000 (UTC) From: Daniel Semyonov To: 64202@debbugs.gnu.org Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87v8fhmgvw.fsf@dsemy.com> (Daniel Semyonov's message of "Wed, 21 Jun 2023 10:08:51 +0300") References: <87v8fhmgvw.fsf@dsemy.com> Date: Wed, 21 Jun 2023 12:50:48 +0300 Message-ID: <87r0q5m9dz.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain I realize I never wrote down much about the internals of the back end in detail, so here are some notes: - This backend supports all features of Gnus which (I think) make sense. - It is separated into three main parts - a set of parsing functions specific to Atom, a fairly generic implementation of server data storage (since most Atom feeds only contain the few most recent articles), and a Gnus back end interface which uses the former two parts to get information for Gnus. The intention is to allow creating inheriting back ends for different types of feeds by only changing the first part (and all of its functions are therefore stored in server variables). This also means a user can customize the parser by customizing the select method, which is pretty cool. - The function responsible for printing the content of each part is also stored in a server variable ('nnatom-print-content-function). This means that 'nnatom-read-links-function' can return any Lisp object and 'nnatom-read-parts-function' can return any Lisp object as its car and any Lisp object in its cdr (a list) with a custom value of 'nnatom-print-content-function'. The reason it is customizable is that Atom feeds only support plain text and (X)HTML feeds, and they already require some extra work to display nicely; I suspect if other web feed formats support other types of content they would also require something like that. And of course it can also be customized through the select method. - A macro is provided which eliminates most of the boilerplate code required to define an inheriting back end. I've also attached a simple inheriting back end for youtube channel feeds - they are very slightly different from normal Atom feeds so it's a good way to know whether issues come from the back end itself or from the inheritance mechanism. Regards, Daniel --=-=-= Content-Type: application/emacs-lisp Content-Disposition: inline; filename=nnyt.el Content-Transfer-Encoding: quoted-printable Content-Description: nnyt ;;; nnyt.el --- yt backend for Gnus -*- lexical-binding: t -*- ;; Copyright (C) 2023 Daniel Semyonov. ;; Author: Daniel Semyonov ;; Package-Version: 0.1 ;; This file is not part of GNU Emacs. ;; nnyt is free software: you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. ;; nnyt is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; You should have received a copy of the GNU General Public License ;; along with nnyt. If not, see . ;;; Commentary: ;;; Code: (require 'nnatom) (nnoo-declare nnyt nnatom) (nnatom-define-basic-backend-interface nnyt) ;;;; YouTube feed parser: ;; All channels have the same prefix, so an address can be just a channel i= d. (defvoo nnyt-read-channel-function (lambda (feed group) (nnatom--read-feed (concat "youtube.com/feeds/videos.xml?channel_id=3D" feed) group)) nil nnatom-read-feed-function) ;; YouTube feeds don't contain a summary or content, only a "description". (defvoo nnyt-read-description-function (lambda (article) `(,(nnatom--read-part (dom-text (if-let ((d (dom-child-by-tag article 'group))) (dom-child-by-tag d 'description) (dom-child-by-tag (dom-child-by-tag article 'media:gr= oup) 'media:description))) "plain" t))) nil nnatom-read-parts-function) (gnus-declare-backend (symbol-name nnyt-backend) 'address) (provide 'nnyt) ;;; nnyt.el ends here --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 21 09:02:57 2023 Received: (at 64202) by debbugs.gnu.org; 21 Jun 2023 13:02:57 +0000 Received: from localhost ([127.0.0.1]:60866 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qBxU9-0006bq-6x for submit@debbugs.gnu.org; Wed, 21 Jun 2023 09:02:57 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33886) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qBxU6-0006bb-Sy for 64202@debbugs.gnu.org; Wed, 21 Jun 2023 09:02:55 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qBxTv-0004gv-Ol; Wed, 21 Jun 2023 09:02:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=cD6v4UrY6bn7WE0VjvJuFCTVmY3cgTWtN12llsvuZB4=; b=bUWffXLC47bm ew3jy+p2l1WpMHYdEDu4sb1zXnyqsuG3+Bd3LRJmbDBIldaJR1Ujduq4RuI1NEa7RwWwo8VMV8DCc kIjAkaL88UI5Lpt5vxRcnAkR6pBSt0XfkPnzlATo4YnTae6FQwwmAGwHx5EXSkZyyycJmHlD/Xc5Y CMlkcKBvsBYh1PYyhj9gE4Tb/AHz2jl7+W1g+XAZlnIZFd9MHyqgzASL/Ddqeb4O1I+6kShksx3Gu jarJbIk+l9SpGzHsBFVbdmstUZJmsYpJJXsvEiJN/8A9cMMUPgQI6Nq63yqaZpEbbt464zQpGyTtq Z/uOhPodBSb8YNiGn5B57g==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qBxTX-0004Ku-N1; Wed, 21 Jun 2023 09:02:27 -0400 Date: Wed, 21 Jun 2023 16:02:27 +0300 Message-Id: <83y1kd3r4s.fsf@gnu.org> From: Eli Zaretskii To: Daniel Semyonov , Andrew Cohen , Eric Abrahamsen , Stefan Monnier , Lars Ingebrigtsen In-Reply-To: <87v8fhmgvw.fsf@dsemy.com> (bug-gnu-emacs@gnu.org) Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) References: <87v8fhmgvw.fsf@dsemy.com> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Wed, 21 Jun 2023 10:08:51 +0300 > From: Daniel Semyonov via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" > > This patch adds a back end for Atom feeds to Gnus, and documents it. Thanks for working on this. I don't use Gnus, so my comments will be mostly to the documentation parts. I expect others (CC'ed) to review the code more thoroughly. > +Some web sites provide an Atom Syndication Format feed. Atom is a web > +feed format similar in function to RDF Site Summary (@xref{RSS}). ^^^^^^^^^^ This should be @pxref, not @xref. > +Note, however, that the server address shouldn't be prefixed with > +@code{http://} or @code{https://}. These should use @file, not @code, unless most or all of the rest of the Gnus manual uses @code for URLs. > +The @code{nnatom} back end saves a server data file in the > +@code{atom} sub-directory of @code{gnus-directory} for each feed. These should definitely be @file, not @code, as these are file names. > +@defmac nnatom-define-basic-backend-interface backend > +Define server variables expected by @code{nnatom} and import its back > +end functions for BACKEND. This macro (or code similar to it) should ^^^^^^^ This should be @var{backend}. From debbugs-submit-bounces@debbugs.gnu.org Wed Jun 21 10:28:32 2023 Received: (at 64202) by debbugs.gnu.org; 21 Jun 2023 14:28:32 +0000 Received: from localhost ([127.0.0.1]:33880 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qByow-0005gL-4y for submit@debbugs.gnu.org; Wed, 21 Jun 2023 10:28:32 -0400 Received: from dsemy.com ([46.23.89.208]:9603) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qByot-0005g6-C8 for 64202@debbugs.gnu.org; Wed, 21 Jun 2023 10:28:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=32kyhCKUij70e CowyrEwTMYqCpx0neE6qMvJcHQNA20=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=RXmBfhzDFhkgAB58LzP5GHhg1X006t3rNBP DMr2lVPR9EW1Yma4HrmG4UKOKECaGZX/6FZFAGnO5zHijEqtet84KCNaGYoemhZvc8pXYT kTHbSFZk1ACWvIVyNR8iqXjXYT9v4T7k9a8JkNG1Z/33/clmUnK5cZWUI+g1GnGAW6PWa6 PwQbLQz8z1NjdoT/4xTr1ZC14U3LsjZveyauDZIUmlgPihN3gRxmYLFFmwqwLM5A73xVfw LckPnO6xQKvSTyzoWzDdxNaxOKEvxcKIlz/2wOMLYHWh7aNb69X0edMFGHfVmI5nvH21jq s6GHqLV7SGXmvWA6EuM8yWZqMDQ== Received: from coldharbour.home.arpa (bzq-109-64-87-185.red.bezeqint.net [109.64.87.185]) by dsemy.com (OpenSMTPD) with ESMTPSA id 49150d03 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Wed, 21 Jun 2023 16:28:20 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id 090771e2; Wed, 21 Jun 2023 14:26:25 +0000 (UTC) From: Daniel Semyonov To: Eli Zaretskii Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <83y1kd3r4s.fsf@gnu.org> (Eli Zaretskii's message of "Wed, 21 Jun 2023 16:02:27 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <83y1kd3r4s.fsf@gnu.org> Date: Wed, 21 Jun 2023 17:26:25 +0300 Message-ID: <87legcuc1a.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: Andrew Cohen , Eric Abrahamsen , Lars Ingebrigtsen , Stefan Monnier , 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain >>>>> Eli Zaretskii writes: >> Date: Wed, 21 Jun 2023 10:08:51 +0300 From: Daniel Semyonov via >> "Bug reports for GNU Emacs, the Swiss army knife of text editors" >> >> >> This patch adds a back end for Atom feeds to Gnus, and documents >> it. > Thanks for working on this. > I don't use Gnus, so my comments will be mostly to the > documentation parts. I expect others (CC'ed) to review the code > more thoroughly. Thanks, amended patch attached. >> +Some web sites provide an Atom Syndication Format feed. Atom is >> a web +feed format similar in function to RDF Site Summary >> (@xref{RSS}). > ^^^^^^^^^^ > This should be @pxref, not @xref. Fixed. >> +Note, however, that the server address shouldn't be prefixed >> with +@code{http://} or @code{https://}. > These should use @file, not @code, unless most or all of the rest > of the Gnus manual uses @code for URLs. I changed it to @url, as the close by RSS node uses it for URL. Hopefully that's okay. >> +The @code{nnatom} back end saves a server data file in the >> +@code{atom} sub-directory of @code{gnus-directory} for each >> feed. > These should definitely be @file, not @code, as these are file > names. Fixed ('atom' and 'gnus-directory', not 'nnatom' as I think that should stay @code). >> +@defmac nnatom-define-basic-backend-interface backend +Define >> server variables expected by @code{nnatom} and import its back >> +end functions for BACKEND. This macro (or code similar to it) >> should > ^^^^^^^ This should be @var{backend}. Fixed. I also attached another back end I made between my last email and now, for JSON feeds. This is just a POC I made to see how easy it is to create a "full" derivative back end. --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch Content-Description: patch >From 6085ee8139cc3d815a5028babb4daf438df9d06b Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnatom.el: New file. * doc/misc/gnus.texi: * etc/NEWS: Document nnatom --- doc/misc/gnus.texi | 42 +++ etc/NEWS | 7 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 705 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 755 insertions(+) create mode 100644 lisp/gnus/nnatom.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 8d25e868c8a..95eba21c4dd 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -17250,6 +17251,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17494,6 +17496,46 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@xref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds +as Gnus servers (with a single group), as you would with any other +method, by supplying the location of the feed as the server address. +Note, however, that the server address shouldn't be prefixed with +@code{http://} or @code{https://}. + +The @code{nnatom} back end saves a server data file in the +@code{atom} sub-directory of @code{gnus-directory} for each feed. + +The @code{nnatom} back end generates an article part for both the +summary and content of each article in the feed. + +@code{nnatom} has been designed to be very modular, and theoretically +supports many features which aren't available in the Atom Syndication +Format, in an effort to reduce the work required to build back ends +for other web feed formats. + +Every parsing step is handled by a function stored in a server +variable; these are all called @code{nnatom-read-*-function}, and +their requirements are detailed in their docstrings. + +The function responsible for printing the content of each article part +is stored in a server variable (@code{nnatom-print-content-function}). +The default function only handles (X)HTML and plain text content. + +@defmac nnatom-define-basic-backend-interface backend +Define server variables expected by @code{nnatom} and import its back +end functions for BACKEND. This macro (or code similar to it) should +be used by any inheriting back end. +@end defmac + + @node Other Sources @section Other Sources diff --git a/etc/NEWS b/etc/NEWS index 77ca749ccc3..46e6aeb34dc 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -324,6 +324,13 @@ distracting and easily confused with actual code, or a significant early aid that relieves you from moving the buffer or reaching for the mouse to consult an error message. +** Gnus + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + ** Python Mode --- diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index fc8518512ee..d35d709c448 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1343,6 +1343,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..3da4644de20 --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,705 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format , or any +;; other type of feed with customized parsing functions. + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'gv) + (require 'subr-x)) + +(require 'gnus) +(require 'nnheader) +(require 'nnoo) +(require 'gnus-group) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'gnus) + +(nnoo-declare nnatom) + +(defvoo nnatom-backend 'nnatom + "Symbol which identifies this backend.") + +(defvoo nnatom-status-string nil + "Last status message reported by this backend.") + +(defsubst nnatom--backend-prefix (backend) + (concat (symbol-name backend) ":")) + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "^https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-readable-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (if (libxml-available-p) + (libxml-parse-xml-region (point-min) (point-max)) + (require 'xml) + (car (xml-parse-region (point-min) (point-max))))))))) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) + (setq data (dom-by-tag data 'entry))) + (and data `(,(car data) . , (setq data (cdr data))))) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) + +(defun nnatom--read-article-or-group-author (article-or-group) + "Return the author of ARTICLE-OR-GROUP, or nil." + (let* ((author (dom-child-by-tag article-or-group 'author)) + (name (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (or (and name email (format "%s <%s>" name email)) + name email))) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((rel (make-vector 5 0))) ; [ALTERNATE RELATED SELF ENCLOSURE VIA] + (mapcan + (lambda (link) + (when-let + (((and (consp link) (eq (car link) 'link))) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf (aref rel 1))) + (format " %s" (aref rel 1))))) + ("self" + (concat "More" + (and (< 1 (cl-incf (aref rel 2))) + (format " %s" (aref rel 2))))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf (aref rel 3))) + (format " %s" (aref rel 3))))) + ("via" + (concat "Source" + (and (< 1 (cl-incf (aref rel 4))) + (format " %s" (aref rel 4))))) + (_ (if-let ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat "Link" + (and (< 1 (cl-incf (aref rel 0))) + (format " %s" (aref rel 0)))))))))) + `(,(vector (cdr (assq 'href attrs)) + (concat label ":") (concat "[" label "]"))))) + article))) + +(defsubst nnatom--read-part (part type links) + `(,part + ("Content-Type" . ,(concat "text/" (or type "plain"))) + ,(and (or (string= type "html") (string= type "xhtml")) + 'html) + ,(and links 'links))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (dom-attr summary 'type)) + (summary (dom-text summary)) + (summary (unless (string-blank-p summary) summary)) + (content (dom-child-by-tag article 'content)) + (ctype (dom-attr content 'type)) + (content (dom-text content)) + (content (unless (string-blank-p content) content)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push (nnatom--read-part summary stype nil) parts)) + (push (nnatom--read-part content ctype t) parts) + (or st (push (nnatom--read-part summary stype nil) parts)) + parts) + ((setq parts (or summary content)) + `(,(nnatom--read-part parts (if content ctype stype) t))) + (t '((nil ("Content-Type" . "text/html") html links)))))) + +;;;; Feed I/O: + +(defvoo nnatom-read-feed-function #'nnatom--read-feed + "Function returning a Lisp object representing a feed (or part of it). +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnatom-read-group-function').") + +(defvoo nnatom-read-group-function #'nnatom--read-group + "Function returning a cons cell of a group and remaining data from a feed.") + +(defvoo nnatom-read-article-function #'nnatom--read-article + "Function returning a cons cell of an article and remaining data from a group. +It should accept a two arguments, a Lisp object representing a feed, +and a flag indicating whether the last article was stale (not new or updated).") + +(defvoo nnatom-read-title-function #'nnatom--read-title + "Function returning the title of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-description-function #'nnatom--read-description + "Function returning the description of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-group-author-function #'nnatom--read-article-or-group-author + "Function returning the author of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-id-function #'nnatom--read-id + "Function returning the ID of an article. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-subject-function #'nnatom--read-subject + "Function returning the subject of an article (a string). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + "Function returning the publish date of an article (a time value). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-update-date-function #'nnatom--read-update + "Function returning the update date of an article (a time value). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-author + "Function returning the author of an article (a string). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-links-function #'nnatom--read-links + "Function returning all links contained in an article. +For the default `nnatom-print-content-function', it should return a +list of links, where each link is a vector of the form +[LINK LABEL HTML-LABEL], where: +- LINK is the link. +- LABEL is a label used for LINK (printed \"LABEL: LINK\"). +- HTML-LABEL is a label used for LINK, but only if the type of the + part in which LINK is printed is \"html\" or \"xhtml\" (printed \"[LABEL]\"). +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-parts-function #'nnatom--read-parts + "Function returning an alist associating parts of an article to their headers. +For the default `nnatom-print-content-function', each part should be +a string. Otherwise, it can be any Lisp object. +The \"headers\" of each part should be a list where each element is +either a cons of a MIME header (a string) and its value (a string) or +any other Lisp object. MIME headers will be printed, the rest will be +passed on to `nnatom-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +- `html', if present, will format the part as HTML. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnatom-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnatom-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnatom--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnatom-backend) + "nn") + (gnus-newsgroup-savable-name server)) + gnus-directory)) + +(defun nnatom--read-server (server) + "Read SERVER's information from `nnatom-directory'." + (if-let ((f (nnatom--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnatom-servers)) + (nnheader-report nnatom-backend "Can't read %s" server))) + +(defun nnatom--write-server (server) + "Write SERVER's information to `nnatom-directory'." + (if-let ((f (nnatom--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnatom-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n" + (prin1-to-string s)) + t) + t) + (nnheader-report nnatom-backend "Can't write %s" f))) + +(defsubst nnatom--server-address (server) + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnatom-backend 'address) + (cddr (gnus-server-to-method + (concat (nnatom--backend-prefix + nnatom-backend) + server))))) + server)) + server)) + +(defun nnatom--parse-feed (feed &optional group) + "Parse FEED into a new or existing server. +Optionally, only parse GROUP." + (let* ((feed (nnatom--server-address feed)) + (prefix (nnatom--backend-prefix nnatom-backend)) + (s (or (gethash feed nnatom-servers) + (nnatom--read-server feed) t)) + (g (or (and (hash-table-p s) (gethash group s)) `[,group])) + (name group) ; (Maybe) fake name + (group (or (aref g 0) name)) ; Real name + data) + (when (setq data (funcall nnatom-read-feed-function feed group) + s (or (gethash feed nnatom-servers) + (make-hash-table :test 'equal))) + (while-let ((cg (or (and name `(,data)) + (funcall nnatom-read-group-function data))) + ((progn (setq data (cdr cg)) t)) + (cg (car cg))) + (let* ((name (funcall nnatom-read-title-function cg)) ; Real name + (group (gethash name nnatom-group-names name)) ; (Maybe) fake name + (info (gnus-get-info (concat prefix group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnatom-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnatom-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnatom-read-article-function cg stale)) + ((progn (setq cg (cdr article)) t)) + (article (car article))) + (when-let ((id (funcall nnatom-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnatom-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnatom-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (cl-incf max)))) + (publish (funcall nnatom-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnatom-read-author-function article) + group-author group) + ,(or (funcall nnatom-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnatom-read-links-function article) + ,(funcall nnatom-read-parts-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnatom-servers)))) + +;;;; Gnus backend functions: + +(defvoo nnatom-group nil + "Name of the current group.") + +(defvoo nnatom-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnatom-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers.") + +(defvoo nnatom-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnatom-group-article-min-num 1 + "Minimum article number for the current group.") + +(defvar nnatom-date-format "%F %X" + "Format of displayed dates.") + +(nnoo-define-basics nnatom) + +(defun nnatom--group-data (group &optional server) + (let ((s (gethash server nnatom-servers)) c) + (or (and (hash-table-p s) (gethash group s)) + (and (setq c (nnoo-current-server nnatom-backend)) + (setq s (gethash c nnatom-servers)) + (hash-table-p s) (gethash group s)) + (catch :stop (maphash (lambda (n s) + (or (string= n server) + (string= n c) + (when-let (((hash-table-p s)) + (g (gethash group s))) + (throw :stop g)))) + nnatom-servers))))) + +(defun nnatom-retrieve-article (article group) + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string nnatom-date-format (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnatom-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnoo-current-server nnatom-backend))) + (g (or (nnatom--group-data group server) + `[ nil ,nnatom-group-article-ids ,nnatom-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnatom-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnatom-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnatom-backend "Group %s not found" (or group "")))) + +(deffoo nnatom-open-server (server &optional defs backend) + (let ((backend (or backend 'nnatom)) + (a (nnatom--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (nnatom--read-server a)) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash group g nnatom-group-names))) + s)) + (or s (file-writable-p (nnatom--server-file a)) + (and (nnoo-close-server (or backend 'nnatom) server) + (nnheader-report + nnatom-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnatom-request-close () + (maphash (lambda (server _) + (nnatom--write-server + (nnatom--server-address server))) + nnatom-servers) + (setq nnatom-servers (make-hash-table :test 'equal) + nnatom-status-string nil) + t) + +(defun nnatom--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, possibly with LINKS added." + (let ((html (memq 'html attributes)) + (links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and html "") + (and content (format "%s\n\n" content)) + (and links html "

") + (and links + (if html + (mapconcat + (lambda (link) + (format "%s" (aref link 0) (aref link 2))) + links " ") + (mapconcat + (lambda (link) + (format "%s %s\n" (aref link 1) (aref link 0))) + links "\n"))) + (and links html "

") + (and html ""))))) + +(defvoo nnatom-print-content-function #'nnatom--print-content + "Function returning a single piece of content (a string). +It should accept three arguments, a part and its attributes (as returned +by `nnatom-read-parts-function'), and links.") + +(defsubst nnatom--print-part (content headers mime links) + (concat + (mapconcat (lambda (header) + (when-let (((consp header)) + (m (car header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnatom-print-content-function content headers links))) + +(deffoo nnatom-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnoo-current-server nnatom-backend))) + (g (or (nnatom--group-data group server) + (and (setq group nnatom-group) + `[ nil ,nnatom-group-article-ids + ,nnatom-group-articles + ,nnatom-group-article-max-num + ,nnatom-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((boundary (format "-_%s_-" nnatom-backend)) + (links (aref a 5)) + (parts (aref a 6)) + (multi (length> parts 1)) + (mime '( "Content-Type" "Content-Disposition" + "Content-Transfer-Encoding"))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnatom-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + "MIME-Version: 1.0\n" + (if multi + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary) + (prog1 (nnatom--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (let ((headers (cdr part))) + (format "--%s\n%s\n" boundary + (nnatom--print-part + (car part) headers mime links)))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnatom-backend "No such article"))) + +(deffoo nnatom-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnoo-current-server nnatom-backend))) + (g (or (if fast (nnatom--group-data group server) + (setq server (nnatom--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnatom-group group + nnatom-group-article-ids (aref g 1) + nnatom-group-articles (aref g 2) + nnatom-group-article-max-num (aref g 3) + nnatom-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnatom-group-article-ids) + nnatom-group-article-min-num + nnatom-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnatom-backend "Group %s not found" group)))) + +(deffoo nnatom-close-group (group &optional server) + (and (string= group nnatom-group) + (setq nnatom-group nil + nnatom-group-article-ids (make-hash-table :test 'equal) + nnatom-group-articles (make-hash-table :test 'eql) + nnatom-group-article-max-num 0 + nnatom-group-article-min-num 1)) + (setq server (or server (nnoo-current-server nnatom-backend))) + (nnatom--write-server server)) + +(deffoo nnatom-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnatom--parse-feed + (or server (nnoo-current-server nnatom-backend)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnatom-request-post (&optional _server) + (nnheader-report nnatom-backend "%s is a read only backend" nnatom-backend)) + +;;;; Optional back end functions: + +(deffoo nnatom-retrieve-groups (_groups &optional server) + (nnatom-request-list (or server (nnoo-current-server nnatom-backend))) + 'active) + +(deffoo nnatom-request-type (_group &optional _article) + 'unknown) + +(deffoo nnatom-request-group-description (group &optional server) + (when-let ((server (or server (nnoo-current-server nnatom-backend))) + (g (nnatom--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnatom-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnoo-current-server nnatom-backend))) + (s (gethash server nnatom-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnatom-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnoo-current-server nnatom-backend))) + (s (gethash server nnatom-servers)) + (g (or (nnatom--group-data group server) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnatom-group-names) + (remhash group s) + (and (string= group nnatom-group) + (setq nnatom-group new-name)) + t)) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +;;;; Utilities: + +(defmacro nnatom-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnatom-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnatom-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnatom-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnatom-servers) + (defvoo ,(nnoo-symbol backend 'group-names) + (make-hash-table :test 'equal) nil nnatom-group-names) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnatom-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnatom-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnatom-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnatom-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnatom-open-server server defs ',backend)) + (nnoo-import ,backend (nnatom)))) + +(provide 'nnatom) + +;;; nnatom.el ends here -- 2.41.0 --=-=-= Content-Type: application/emacs-lisp Content-Disposition: attachment; filename=nnjsonfeed.el Content-Transfer-Encoding: quoted-printable Content-Description: nnjsonfeed ;;; nnjsonfeed.el --- JSON Feed backend for Gnus -*- lexical-binding: t -*- ;; Copyright (C) 2023 Free Software Foundation, Inc. ;; Author: Daniel Semyonov ;; This file is part of GNU Emacs. ;; nnjsonfeed is free software: you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. ;; nnjsonfeed is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; You should have received a copy of the GNU General Public License ;; along with nnjsonfeed. If not, see . ;;; Commentary: ;; Gnus backend for JSON feeds. ;;; Code: (require 'nnatom) (nnoo-declare nnjsonfeed nnatom) (nnatom-define-basic-backend-interface nnjsonfeed) ;;;; JSON feed parser: (defun nnjsonfeed--read-feed (feed _) (with-temp-buffer (condition-case e (if (file-readable-p feed) (insert-file-contents feed) (mm-url-insert-file-contents (concat "https://" feed))) (file-error (nnheader-report nnjsonfeed-backend (cdr e))) (:success (or (when-let ((j (json-parse-buffer))) `(,j . 0)) (nnheader-report nnjsonfeed-backend "Invalid feed: %s" feed)))))) (defun nnjsonfeed--read-item (feed _) (when-let ((f (car feed)) (i (cdr feed)) (items (gethash "items" f)) (item t)) (unless (length=3D items i) (setcdr feed (1+ i)) (setq item (aref items i)) (aset items i nil) `(,item . ,feed)))) (defun nnjsonfeed--read-title (feed-or-item) (setq feed-or-item (if (listp feed-or-item) (car feed-or-item) feed-or-item)) (gethash "title" feed-or-item)) (defun nnjsonfeed--read-description (feed) (gethash "description" (car feed))) (defsubst nnjsonfeed--read-author-name (author) (when (stringp (setq author (gethash "name" author))) (concat author ", "))) (defun nnjsonfeed--read-authors (feed-or-item) (setq feed-or-item (if (listp feed-or-item) (car feed-or-item) feed-or-item)) (string-trim-right (if-let ((authors (gethash "authors" feed-or-item))) (mapconcat #'nnjsonfeed--read-author-name authors) (if-let ((author (gethash "author" feed-or-item))) (nnjsonfeed--read-author-name author) "")) ", ")) (defun nnjsonfeed--read-id (item) (gethash "id" item)) (defun nnjsonfeed--read-date (item mod) (when-let ((d (gethash (if mod "date_modified" "date_published") item))) (date-to-time d))) (defun nnjsonfeed--read-links (item) (let ((atts 0) links) (when-let ((l (gethash "url" item))) (push (vector l "Link:" "[Link]") links)) (when-let ((l (gethash "external_url" item))) (push (vector l "Related:" "[Related]") links)) (append links (mapcar (lambda (a) (let* ((l (gethash "url" a)) (type (gethash "mime_type" a)) (size (gethash "size_in_bytes" a)) (dur (gethash "duration_in_seconds" a)) (title (concat (or (gethash "title" a) (format "Attachment%s" (if (< 1 (cl-incf atts)) (format " %d" atts) ""))) (if dur " [" " ") (and dur (format-seconds "%y year, %d days, %.2h:%z%.2m:%.2s= ]" dur)) " (" type (and size ", ") (and size (file-size-human-readable size)) ")= "))) (vector l (concat title ":") (concat "[" title "]")))) (gethash "attachments" item))))) (defun nnjsonfeed--read-parts (item) (let ((summary (gethash "summary" item)) (contentt (gethash "content_text" item)) (contenth (gethash "content_html" item)) parts) (if (not (or summary contentt contenth)) '((nil ("Content-Type" . "text/html") html links)) (setq parts nil) (and contenth (push (nnatom--read-part contenth "html" t) parts)) (and summary (push (nnatom--read-part summary "plain" (not (or contenth contentt))) parts)) (and contentt (push (nnatom--read-part summary "plain" (not contenth)) parts)) parts))) (defvoo nnjsonfeed-read-feed-function #'nnjsonfeed--read-feed nil nnatom-read-feed-function) (defvoo nnjsonfeed-read-item-function #'nnjsonfeed--read-item nil nnatom-read-article-function) (defvoo nnjsonfeed-read-title-function #'nnjsonfeed--read-title nil nnatom-read-title-function nnatom-read-subject-function) (defvoo nnjsonfeed-read-description-function #'nnjsonfeed--read-description nil nnatom-read-description-function) (defvoo nnjsonfeed-read-authors-function #'nnjsonfeed--read-authors nil nnatom-read-group-author-function nnatom-read-author-function) (defvoo nnjsonfeed-read-id-function #'nnjsonfeed--read-id nil nnatom-read-id-function) (defvoo nnjsonfeed-read-publish-date-function (lambda (item) (nnjsonfeed--read-date item nil)) nil nnatom-read-publish-date-function) (defvoo nnjsonfeed-read-modified-date-function (lambda (item) (nnjsonfeed--read-date item t)) nil nnatom-read-update-date-function) (defvoo nnjsonfeed-read-links-function #'nnjsonfeed--read-links nil nnatom-read-links-function) (defvoo nnjsonfeed-read-parts-function #'nnjsonfeed--read-parts nil nnatom-read-parts-function) (gnus-declare-backend (symbol-name nnjsonfeed-backend) 'address) (provide 'nnjsonfeed) ;;; nnjsonfeed.el ends here --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 25 06:09:45 2023 Received: (at 64202) by debbugs.gnu.org; 25 Jun 2023 10:09:45 +0000 Received: from localhost ([127.0.0.1]:42070 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qDMgh-0000Se-Lc for submit@debbugs.gnu.org; Sun, 25 Jun 2023 06:09:45 -0400 Received: from dsemy.com ([46.23.89.208]:25075) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qDMgd-0000SP-St for 64202@debbugs.gnu.org; Sun, 25 Jun 2023 06:09:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=ls8VO6ppOG9X6 ConUDS0OUdgVZpSdnteIa1x0B6hNpU=; h=date:references:in-reply-to: subject:to:from; d=dsemy.com; b=uClWZJLf0lr8sl4G5qGO0alo3w5OPRCoppoxsP PIPA/bHy++bno4ym4GLcuQvkXI++ZOq9UmsLhSAAU13rkWVggZMfOXFf6cNg2913dmtJ32 ZC1NHKvHVw7g+Ai1bOs8iXFDJ2Qr6aDMYbAAROl5F6tt96goLQtqHqwrpm6PH9xzzidxnc gr+xnn0TVnTHr7EINBuRgl6+ZTjM2bl0uH9RdAkcSfDC0vdw6YLB/UpKjFUZoxc+e6X/A0 XBt0ksZ/gfgQEsobYDFuohTSoaWbqy4iNi8Wb/HQWAQrZJ5ym1aEj4Pf+Oh5dbypceRLn/ YIFXj7rBojB4jyjcD94R3LZQ== Received: from coldharbour.home.arpa (bzq-109-64-87-185.red.bezeqint.net [109.64.87.185]) by dsemy.com (OpenSMTPD) with ESMTPSA id e4933e88 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for <64202@debbugs.gnu.org>; Sun, 25 Jun 2023 12:09:32 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id 5426fc2f for <64202@debbugs.gnu.org>; Sun, 25 Jun 2023 10:07:58 +0000 (UTC) From: Daniel Semyonov To: 64202@debbugs.gnu.org Subject: Re: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87v8fhmgvw.fsf@dsemy.com> (Daniel Semyonov's message of "Wed, 21 Jun 2023 10:08:51 +0300") References: <87v8fhmgvw.fsf@dsemy.com> Date: Sun, 25 Jun 2023 13:07:58 +0300 Message-ID: <87y1k76eip.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 64202 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain >>>>> Daniel Semyonov writes: > This patch adds a back end for Atom feeds to Gnus, and > documents it. Attached is an updated version of the patch with the following changes: - Remove "backend+" prefix from server addresses returned by 'nnoo-current-server'. - Don't handle HTML in any special way when printing content. - Fix group renaming when the last server of a backend inheriting from nnatom was of a different backend than the groups'. - Fix newly parsed feed information being overwritten by old on-disk information (not sure how I missed this one before). - Support all content types supported by Atom (I think). Previously I stated Atom only supports plain text and (X)HTML. This is wrong - Atom supports arbitrary MIME types too, and thankfully the encoding of binary data is defined to be base64, which is natively supported by Gnus. - Change format of stored links - now an alist mapping MIME types (or lists of them) to a string to print is used. - Support printing custom article-wide MIME headers. - Probably some more minor stuff I don't remember. Also attached is an updated version of the nnyt backend I use for testing inheritance. Daniel --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch Content-Description: nnatom >From 615d00a9294595574bab4f59df081c39f75724c2 Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH 1/2] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnatom.el: New file. * doc/misc/gnus.texi: * etc/NEWS: Document nnatom --- doc/misc/gnus.texi | 58 ++++ etc/NEWS | 7 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 762 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 828 insertions(+) create mode 100644 lisp/gnus/nnatom.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 8d25e868c8a..9184a4035df 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -17250,6 +17251,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17494,6 +17496,62 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@pxref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds +as Gnus servers (with a single group), as you would with any other +method, by supplying the location of the feed as the server address. +Note, however, that the server address shouldn't be prefixed with +@url{http://} or @url{https://}. + +Features of @code{nnatom} include: + +@itemize @bullet + +@item +Server data is saved per-server in the @file{atom} sub-directory of +@file{gnus-directory}. + +@item +An article part is generated for both the summary and the content for +each entry in the feed. Content of all MIME types should be displayed +correctly through Gnus (as long as the feed specifies a MIME type), +even base64 encoded files (like images for example), which are +supported by Atom. + +@item +Article modification and publish dates are tracked, and articles are +updated if changed. + +@item +Designed with modularity in mind, the back end theoretically supports +features which aren't available in Atom (and thus aren't actually +implemented in practice). For example, the back end operates under +the assumption that ``servers'' (Atom feeds) can have multiple groups, +when in reality the parsing functions will only ever return servers +with a single group. In addition, all parsing steps and some of the +printing is done in functions stored in server variables, to allow +very easily defining new back ends for different kinds of feeds by +inheriting from @code{nnatom}; these are called +@code{nnatom-read-*-function} and +@code{nnatom-print-content-function}, and their requirements are +detailed in their docstrings. A macro is provided to ease creation of +new inheriting backends. + +@defmac nnatom-define-basic-backend-interface backend +Define server variables expected by @code{nnatom} and import its back +end functions for @var{backend}. This macro (or code similar to it) should +be used by any inheriting back end. +@end defmac + +@end itemize + @node Other Sources @section Other Sources diff --git a/etc/NEWS b/etc/NEWS index 77ca749ccc3..46e6aeb34dc 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -324,6 +324,13 @@ distracting and easily confused with actual code, or a significant early aid that relieves you from moving the buffer or reaching for the mouse to consult an error message. +** Gnus + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + ** Python Mode --- diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index fc8518512ee..d35d709c448 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1343,6 +1343,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..74ca2a8a5da --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,762 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format , or any +;; other type of feed with customized parsing functions. + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'gv) + (require 'subr-x)) + +(require 'gnus) +(require 'nnheader) +(require 'nnoo) +(require 'gnus-group) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'gnus) + +(nnoo-declare nnatom) + +(defvoo nnatom-backend 'nnatom + "Symbol which identifies this backend.") + +(defvoo nnatom-status-string nil + "Last status message reported by this backend.") + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "^https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-readable-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (if (libxml-available-p) + (libxml-parse-xml-region (point-min) (point-max)) + (require 'xml) + (car (xml-parse-region (point-min) (point-max))))))))) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) + (setq data (dom-by-tag data 'entry))) + (and data `(,(car data) . , (setq data (cdr data))))) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) + +(defun nnatom--read-article-or-group-author (article-or-group) + "Return the author of ARTICLE-OR-GROUP, or nil." + (let* ((author (dom-child-by-tag article-or-group 'author)) + (name (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (or (and name email (format "%s <%s>" name email)) + name email))) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((rel (make-vector 6 0))) ; [ALTERNATE RELATED SELF ENCLOSURE VIA AUTHOR] + (mapcan + (lambda (link) + (when (consp link) + (or + (when-let (((eq (car link) 'content)) + (src (dom-attr link 'src)) + (label (concat "Link" + (and (< 1 (cl-incf (aref rel 0))) + (format " %s" (aref rel 0)))))) + `(((("text/plain") . ,(format "%s: %s\n" label src)) + (("text/html") . ,(format "[%s] " + src label))))) + (when-let (((eq (car link) 'author)) + (name (dom-text (dom-child-by-tag link 'name))) + (name (if (string-blank-p name) + (concat "Author" + (and (< 1 (cl-incf (aref rel 5))) + (format " %s" (aref rel 5)))) + name)) + (uri (dom-text (dom-child-by-tag link 'uri))) + ((not (string-blank-p uri)))) + `(((("text/plain") . ,(format "%s: %s\n" name uri)) + (("text/html") . ,(format "[%s] " + uri name))))) + (when-let (((eq (car link) 'link)) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf (aref rel 1))) + (format " %s" (aref rel 1))))) + ("self" + (concat "More" + (and (< 1 (cl-incf (aref rel 2))) + (format " %s" (aref rel 2))))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf (aref rel 3))) + (format " %s" (aref rel 3))))) + ("via" + (concat "Source" + (and (< 1 (cl-incf (aref rel 4))) + (format " %s" (aref rel 4))))) + (_ (if-let + ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat + "Link" + (and (< 1 (cl-incf (aref rel 0))) + (format " %s" (aref rel 0))))))))) + (link (cdr (assq 'href attrs)))) + `(((("text/plain") . ,(format "%s: %s\n" label link)) + (("text/html") . ,(format "[%s] " + link label)))))))) + article))) + +(defsubst nnatom--read-part (part type) + (let* ((atypes '("html" "plain")) + (mtypes '(("xhtml" . "text/html") ("text" . "text/plain"))) + (xsuff (concat "[+/]xml\\(-\\(dtd\\|external-parsed-entity\\)\\)?\\'" + "\\|^text")) + (part (if (string= type "xhtml") + (with-temp-buffer + (dom-print (dom-child-by-tag part 'div) nil t) + (buffer-substring-no-properties + (point-min) (point-max))) + (dom-text part))) + (type (if (member type atypes) (concat "text/" type) type)) + (type (or (cdr (assoc type mtypes)) type))) + (unless (string-blank-p part) + `(,part (Content-Type . ,(or type (setq type "text/plain"))) + ,(and (not (string-match-p xsuff type)) + '(Content-Transfer-Encoding . "base64")))))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (cdr (assq 'type (dom-attributes summary)))) + (summary (nnatom--read-part summary stype)) + (content (dom-child-by-tag article 'content)) + (ctype (cdr (assq 'type (dom-attributes content)))) + (content (nnatom--read-part content ctype)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push summary parts)) + (push (append content '(links)) parts) + (or st (push summary parts)) + parts) + ((setq parts (or summary content)) + `(,(append (or summary content) '(links)))) + (t '((nil (Content-Type . "text/html") links)))))) + +;;;; Feed I/O: + +(defvoo nnatom-read-feed-function #'nnatom--read-feed + "Function returning a Lisp object representing a feed (or part of it). +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnatom-read-group-function').") + +(defvoo nnatom-read-group-function #'nnatom--read-group + "Function returning a cons cell of a group and remaining data from a feed.") + +(defvoo nnatom-read-article-function #'nnatom--read-article + "Function returning a cons cell of an article and remaining data from a group. +It should accept a two arguments, a Lisp object representing a feed, +and a flag indicating whether the last article was stale (not new or updated).") + +(defvoo nnatom-read-title-function #'nnatom--read-title + "Function returning the title of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-description-function #'nnatom--read-description + "Function returning the description of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-group-author-function #'nnatom--read-article-or-group-author + "Function returning the author of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-id-function #'nnatom--read-id + "Function returning the ID of an article. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-subject-function #'nnatom--read-subject + "Function returning the subject of an article (a string). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + "Function returning the publish date of an article (a time value). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-update-date-function #'nnatom--read-update + "Function returning the update date of an article (a time value). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-author + "Function returning the author of an article (a string). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-headers-function #'ignore + "Function returning an alist of article-wide MIME headers. +Each element of this alist maps a MIME header (a symbol, +i.e. `Content-Type') to its value. As a special case, `:boundary' +maps to a string which will serve as the boundary between article +parts. This must be supplied if a custom boundary is used in a +multipart content type header. The default boundary is \"-_nnatom_-\", +and is automatically modified to match the name of the back end. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-links-function #'nnatom--read-links + "Function returning all links contained in an article. +For the default `nnatom-print-content-function', it should return a +list of links, where each link is an alist mapping MIME content types +to links formatted for display in a part of that type. Each content +type may also be a list of content types. +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-parts-function #'nnatom--read-parts + "Function returning an alist associating parts of an article to their headers. +For the default `nnatom-print-content-function', each part should be a +string. Otherwise, it can be any Lisp object. The \"headers\" of +each part should be a list where each element is either a cons of a +MIME header (a symbol, i.e. `Content-Type') and its value (a string), +or any other Lisp object. MIME headers will be printed, the rest will +be passed on to `nnatom-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnatom-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnatom-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnatom--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnatom-backend) + "nn") + (gnus-newsgroup-savable-name server)) + gnus-directory)) + +(defun nnatom--read-server (server) + "Read SERVER's information from `nnatom-directory'." + (if-let ((f (nnatom--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnatom-servers)) + (nnheader-report nnatom-backend "Can't read %s" server))) + +(defun nnatom--write-server (server) + "Write SERVER's information to `nnatom-directory'." + (if-let ((f (nnatom--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnatom-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n" + (prin1-to-string s)) + t) + t) + (nnheader-report nnatom-backend "Can't write %s" f))) + +(defsubst nnatom--server-address (server) + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnatom-backend 'address) + (cddr (gnus-server-to-method + (concat + (symbol-name nnatom-backend) ":" + server))))) + server)) + server)) + +(defun nnatom--parse-feed (feed &optional group) + "Parse FEED into a new or existing server. +Optionally, only parse GROUP." + (let* ((feed (nnatom--server-address feed)) + (s (or (gethash feed nnatom-servers) (nnatom--read-server feed))) + (g (or (and (hash-table-p s) (gethash group s)) `[,group])) + (name group) ; (Maybe) fake name + (group (or (aref g 0) name)) ; Real name + data) + (when (setq data (funcall nnatom-read-feed-function feed group) + s (or (gethash feed nnatom-servers) + (make-hash-table :test 'equal))) + (while-let ((cg (or (and name `(,data)) + (funcall nnatom-read-group-function data))) + ((progn (setq data (cdr cg)) t)) + (cg (car cg))) + (let* ((name (funcall nnatom-read-title-function cg)) ; Real name + (group (gethash name nnatom-group-names name)) ; (Maybe) fake name + (info (gnus-get-info + (concat (symbol-name nnatom-backend) "+" feed ":" group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnatom-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnatom-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnatom-read-article-function cg stale)) + ((progn (setq cg (cdr article)) t)) + (article (car article))) + (when-let ((id (funcall nnatom-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnatom-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnatom-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (cl-incf max)))) + (publish (funcall nnatom-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnatom-read-author-function article) + group-author group) + ,(or (funcall nnatom-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnatom-read-links-function article) + ,(funcall nnatom-read-parts-function article) + ,(funcall nnatom-read-headers-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnatom-servers)))) + +;;;; Gnus backend functions: + +(defvoo nnatom-group nil + "Name of the current group.") + +(defvoo nnatom-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnatom-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS HEADERS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers. +- HEADERS is an alist associating article-wide MIME headers to their value.") + +(defvoo nnatom-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnatom-group-article-min-num 1 + "Minimum article number for the current group.") + +(defvar nnatom-date-format "%F %X" + "Format of displayed dates.") + +(nnoo-define-basics nnatom) + +(defsubst nnatom--current-server-no-prefix () + (string-remove-prefix (concat (symbol-name nnatom-backend) "+") + (nnoo-current-server nnatom-backend))) + +(defun nnatom--group-data (group &optional server) + (let ((s (gethash server nnatom-servers)) c) + (or (and (hash-table-p s) (gethash group s)) + (and (setq c (nnatom--current-server-no-prefix)) + (setq s (gethash c nnatom-servers)) + (hash-table-p s) (gethash group s)) + (catch :stop (maphash (lambda (n s) + (or (string= n server) + (string= n c) + (when-let (((hash-table-p s)) + (g (gethash group s))) + (throw :stop g)))) + nnatom-servers))))) + +(defun nnatom-retrieve-article (article group) + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string nnatom-date-format (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnatom-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnatom--current-server-no-prefix))) + (g (or (nnatom--group-data group server) + `[ nil ,nnatom-group-article-ids ,nnatom-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnatom-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnatom-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnatom-backend "Group %s not found" (or group "")))) + +(deffoo nnatom-open-server (server &optional defs backend) + (let ((backend (or backend 'nnatom)) + (a (nnatom--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (or (gethash server nnatom-servers) + (nnatom--read-server a))) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash group g nnatom-group-names))) + s)) + (or s (file-writable-p (nnatom--server-file a)) + (and (nnoo-close-server nnatom-backend server) + (nnheader-report + nnatom-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnatom-request-close () + (maphash (lambda (server _) + (nnatom--write-server + (nnatom--server-address server))) + nnatom-servers) + (setq nnatom-servers (make-hash-table :test 'equal) + nnatom-status-string nil) + t) + +(defun nnatom--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, possibly with LINKS added." + (let ((links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and content (format "%s\n\n" content)) + (mapconcat (lambda (link) + (cdr (assoc (cdr (assq 'Content-Type attributes)) link + (lambda (types type) + (if (stringp types) (string= types type) + (member type types)))))) + links))))) + +(defvoo nnatom-print-content-function #'nnatom--print-content + "Function returning a single piece of content (a string). +It should accept three arguments, a part and its attributes (as returned +by `nnatom-read-parts-function'), and links.") + +(defsubst nnatom--print-part (content headers mime links) + (concat + (mapconcat (lambda (header) + (when-let (((consp header)) + (m (car header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnatom-print-content-function content headers links))) + +(deffoo nnatom-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnatom--current-server-no-prefix))) + (g (or (nnatom--group-data group server) + (and (setq group nnatom-group) + `[ nil ,nnatom-group-article-ids + ,nnatom-group-articles + ,nnatom-group-article-max-num + ,nnatom-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((links (aref a 5)) + (parts (aref a 6)) + (headers (aref a 7)) + (boundary (or (cdr (assq :boundary headers)) + (format "-_%s_-" nnatom-backend))) + (multi (length> parts 1)) + (mime '( Content-Type Content-Disposition + Content-Transfer-Encoding))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnatom-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + (if (assq 'MIME-Version headers) "" "MIME-Version: 1.0\n") + (mapconcat (lambda (header) + (unless (keywordp (car header)) + (format "%s: %s\n" (car header) (cdr header)))) + headers) + (if multi + (if (assq 'Content-Type headers) "" + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary)) + (prog1 (nnatom--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (format "--%s\n%s\n" boundary + (nnatom--print-part + (car part) (cdr part) mime links))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnatom-backend "No such article"))) + +(deffoo nnatom-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnatom--current-server-no-prefix))) + (g (or (if fast (nnatom--group-data group server) + (setq server (nnatom--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnatom-group group + nnatom-group-article-ids (aref g 1) + nnatom-group-articles (aref g 2) + nnatom-group-article-max-num (aref g 3) + nnatom-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnatom-group-article-ids) + nnatom-group-article-min-num + nnatom-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnatom-backend "Group %s not found" group)))) + +(deffoo nnatom-close-group (group &optional server) + (and (string= group nnatom-group) + (setq nnatom-group nil + nnatom-group-article-ids (make-hash-table :test 'equal) + nnatom-group-articles (make-hash-table :test 'eql) + nnatom-group-article-max-num 0 + nnatom-group-article-min-num 1)) + (setq server (or server (nnatom--current-server-no-prefix))) + (nnatom--write-server server)) + +(deffoo nnatom-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnatom--parse-feed + (or server (nnatom--current-server-no-prefix)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnatom-request-post (&optional _server) + (nnheader-report nnatom-backend "%s is a read only backend" nnatom-backend)) + +;;;; Optional back end functions: + +(deffoo nnatom-retrieve-groups (_groups &optional server) + (nnatom-request-list (or server (nnatom--current-server-no-prefix))) + 'active) + +(deffoo nnatom-request-type (_group &optional _article) + 'unknown) + +(deffoo nnatom-request-group-description (group &optional server) + (when-let ((server (or server (nnatom--current-server-no-prefix))) + (g (nnatom--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnatom-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnatom--current-server-no-prefix))) + (s (gethash server nnatom-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnatom-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnatom--current-server-no-prefix))) + (s (or (gethash server nnatom-servers) + (and ; Open the server to add it to `nnatom-servers' + (save-match-data + (nnatom-open-server + server + (cdr ; Get defs and backend. + (assoc server (cdr (assq 'nnatom nnoo-state-alist)) + (lambda (car key) + (and (stringp car) + (string-match + (concat + "\\(\\(nn[[:alpha:]]+\\)\\+\\)?" + "\\(" (regexp-quote key) "\\)") + car) + (setq server car))))) + (prog1 (if (stringp (match-string 1 server)) + (intern (match-string 2 server)) + 'nnatom) + (setq server (match-string 3 server)))) + (gethash server nnatom-servers))))) + (g (or (nnatom--group-data group server) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnatom-group-names) + (remhash group s) + (and (string= group nnatom-group) + (setq nnatom-group new-name)) + t)) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +;;;; Utilities: + +(defmacro nnatom-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnatom-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnatom-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnatom-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnatom-servers) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnatom-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnatom-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnatom-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnatom-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnatom-open-server server defs ',backend)) + (nnoo-import ,backend (nnatom)))) + +(provide 'nnatom) + +;;; nnatom.el ends here -- 2.41.0 --=-=-= Content-Type: application/emacs-lisp Content-Disposition: attachment; filename=nnyt.el Content-Transfer-Encoding: quoted-printable Content-Description: nnyt ;;; nnyt.el --- YouTube backend for Gnus -*- lexical-binding: t -*- ;; Copyright (C) 2023 Daniel Semyonov. ;; Package-Version: 0.1 ;; Author: Daniel Semyonov ;; This file is not part of GNU Emacs. ;; nnyt is free software: you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. ;; nnyt is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; You should have received a copy of the GNU General Public License ;; along with nnyt. If not, see . ;;; Commentary: ;;; Code: (require 'nnatom) (nnoo-declare nnyt nnatom) (nnatom-define-basic-backend-interface nnyt) ;;;; YouTube feed parser: ;; All channels have the same prefix, so an address can be just a channel i= d. (defvoo nnyt-read-channel-function (lambda (feed group) (nnatom--read-feed (concat "youtube.com/feeds/videos.xml?channel_id=3D" feed) group)) nil nnatom-read-feed-function) ;; YouTube feeds all contain a link to the video and the channel in each en= try. (defvoo nnyt-read-links-function (lambda (article) `((("text/plain" . ,(format "Video: %s\n" (dom-attr (dom-child-by-tag article 'link) 'href)))) (("text/plain" . ,(format "Channel: %s\n" (dom-text (dom-child-by-tag (dom-child-by-tag article 'author) 'uri))))))) nil nnatom-read-links-function) ;; YouTube feeds don't contain a summary or content, only a "description". (defvoo nnyt-read-description-function (lambda (article) `((,(dom-text (if-let ((d (dom-child-by-tag article 'group))) (dom-child-by-tag d 'description) (dom-child-by-tag (dom-child-by-tag article 'media:gr= oup) 'media:description))) (Content-Type . "text/plain") links))) nil nnatom-read-parts-function) (gnus-declare-backend (symbol-name nnyt-backend) 'address) (provide 'nnyt) ;;; nnyt.el ends here --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 26 09:29:01 2023 Received: (at 64202) by debbugs.gnu.org; 26 Jun 2023 13:29:02 +0000 Received: from localhost ([127.0.0.1]:44865 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qDmH7-0005dd-LA for submit@debbugs.gnu.org; Mon, 26 Jun 2023 09:29:01 -0400 Received: from dsemy.com ([46.23.89.208]:16931) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qDmH5-0005dN-Jx for 64202@debbugs.gnu.org; Mon, 26 Jun 2023 09:29:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=9+Ornxd806aqP w1Lltiz16EeYC5Ov/7TciawfuxHQkQ=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=G3S980VEvx1gKs/glilAox61bt+DStaFJwU VUmk7fy+EbqCQCdeNvOurlxcQVcWGH9LIV6Pk1CIYmg6QKI4xtx8lvKeexVxUb44x+FWIV 1ssPuP27CH/DTawnrApWCQng4Rd+TvA6gF51/nz0PqIKpYlv5caJ3CGnwlYP+a1rjd0ZTk 7YkuPt/lZm3pYPhOl+EWN8ADRmr17/BOwiXfihz/Adx/Y08ObnekoZS2HOIZHhlWdPhQ7X 8S1ZLSLQswHoqem5SM0nykd0V/oRWlJc40U7Yj7Pc4RDzNKfu42MhdJgyhsmx+a8FW6a0G uy1J2EFS8y94Kprrp57bBiP84TA== Received: from coldharbour.home.arpa (bzq-109-64-87-185.red.bezeqint.net [109.64.87.185]) by dsemy.com (OpenSMTPD) with ESMTPSA id 796ff71f (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 26 Jun 2023 15:28:52 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id f1f6d0fa; Mon, 26 Jun 2023 13:26:34 +0000 (UTC) From: Daniel Semyonov To: 64202@debbugs.gnu.org Subject: Re: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87y1k76eip.fsf@dsemy.com> (Daniel Semyonov's message of "Sun, 25 Jun 2023 13:07:58 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> Date: Mon, 26 Jun 2023 16:26:34 +0300 Message-ID: <87fs6epd6d.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: Andrew Cohen , Eric Abrahamsen , Lars Ingebrigtsen , Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Attached is an updated version of the patch. Changes: - Add support for multiple authors/contributors. - Change how links are parsed to avoid looping over all items in an article for both links and authors. - Optimize the creation of a list of entries from a feed. I also forgot to mention I expanded the info node in the last update. At this point, other than optimizations and bug fixes, the backend is basically complete; it has very good support for the Atom Syndication Format, and (I think) it should be possible to implement any Gnus backend as an inheriting backend of it, without changing any backend interface functions other than possibly 'nnatom-request-type' and 'nnatom-request-rename-group' (a backend like nnimap will need to add some backend functions though). The nnyt backend I posted in my last message still works with this version (although it does some unnecessary work now). Apologies if you're not interested in this and you're CC'd; I just copied the CC header from Eli's last message here since I don't really know who works on Gnus. Regards, Daniel From debbugs-submit-bounces@debbugs.gnu.org Mon Jun 26 10:55:16 2023 Received: (at 64202) by debbugs.gnu.org; 26 Jun 2023 14:55:16 +0000 Received: from localhost ([127.0.0.1]:46825 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qDncY-0002XK-H5 for submit@debbugs.gnu.org; Mon, 26 Jun 2023 10:55:16 -0400 Received: from dsemy.com ([46.23.89.208]:37962) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qDncU-0002X2-SA for 64202@debbugs.gnu.org; Mon, 26 Jun 2023 10:55:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=ip6OvDR9wMdvi Bih9UauNWW1kSwmfnt+pV+M+P10lYo=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=LNyTTYspArkLo6d6efILvSbj1vv2m1YpU5t afvH3sL1K5Q5m8FB0IGMNQZBV2FW6Dpyuv+TiQ0qBnCF5CF3lt53aYCtP4GQz7N72I/3Hs fWmhfZq7Skk1tie7b27ioO25J8Ncr2xNqoSk8wr86u9ooAjU9ecai8Zk6Lx4fxTkMBSyRJ P4js1M5y7u6LCcqE3pDLwZ7wFio80O/TvR/zTAjtuv5iIyHMpVnPt3tL8GV0ZM+tEbkeEk HfcESYpaeo8WYAsTg1MfUVXRQ+3JF0RcsWPfx4T+zDwZeHSluDQnPLwnmUbTNHFGuxlGyj /OBJZsC9eSdvMCVCntBRpzhz04g== Received: from coldharbour.home.arpa (bzq-109-64-87-185.red.bezeqint.net [109.64.87.185]) by dsemy.com (OpenSMTPD) with ESMTPSA id d0bc77df (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 26 Jun 2023 16:55:03 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id c9869502; Mon, 26 Jun 2023 14:52:47 +0000 (UTC) From: Daniel Semyonov To: 64202@debbugs.gnu.org Subject: Re: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87fs6epd6d.fsf@dsemy.com> (Daniel Semyonov's message of "Mon, 26 Jun 2023 16:26:34 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> Date: Mon, 26 Jun 2023 17:52:47 +0300 Message-ID: <87bkh2p96o.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: Andrew Cohen , Eric Abrahamsen , Lars Ingebrigtsen , Stefan Monnier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain >>>>> Daniel Semyonov writes: > Attached is an updated version of the patch. Actually attached this time. --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch Content-Description: nnatom >From d6c176915f52b3a73b1210740d2f06f91a0b76cb Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnatom.el: New file. * doc/misc/gnus.texi: * etc/NEWS: Document nnatom --- doc/misc/gnus.texi | 58 ++++ etc/NEWS | 7 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 795 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 861 insertions(+) create mode 100644 lisp/gnus/nnatom.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 8d25e868c8a..9184a4035df 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -17250,6 +17251,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17494,6 +17496,62 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@pxref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds +as Gnus servers (with a single group), as you would with any other +method, by supplying the location of the feed as the server address. +Note, however, that the server address shouldn't be prefixed with +@url{http://} or @url{https://}. + +Features of @code{nnatom} include: + +@itemize @bullet + +@item +Server data is saved per-server in the @file{atom} sub-directory of +@file{gnus-directory}. + +@item +An article part is generated for both the summary and the content for +each entry in the feed. Content of all MIME types should be displayed +correctly through Gnus (as long as the feed specifies a MIME type), +even base64 encoded files (like images for example), which are +supported by Atom. + +@item +Article modification and publish dates are tracked, and articles are +updated if changed. + +@item +Designed with modularity in mind, the back end theoretically supports +features which aren't available in Atom (and thus aren't actually +implemented in practice). For example, the back end operates under +the assumption that ``servers'' (Atom feeds) can have multiple groups, +when in reality the parsing functions will only ever return servers +with a single group. In addition, all parsing steps and some of the +printing is done in functions stored in server variables, to allow +very easily defining new back ends for different kinds of feeds by +inheriting from @code{nnatom}; these are called +@code{nnatom-read-*-function} and +@code{nnatom-print-content-function}, and their requirements are +detailed in their docstrings. A macro is provided to ease creation of +new inheriting backends. + +@defmac nnatom-define-basic-backend-interface backend +Define server variables expected by @code{nnatom} and import its back +end functions for @var{backend}. This macro (or code similar to it) should +be used by any inheriting back end. +@end defmac + +@end itemize + @node Other Sources @section Other Sources diff --git a/etc/NEWS b/etc/NEWS index 3061a147b26..a01177be3d4 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -339,6 +339,13 @@ distracting and easily confused with actual code, or a significant early aid that relieves you from moving the buffer or reaching for the mouse to consult an error message. +** Gnus + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + ** Python Mode --- diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index fc8518512ee..d35d709c448 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1343,6 +1343,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..c3756562c23 --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,795 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format , or any +;; other type of feed with customized parsing functions. + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'gv) + (require 'subr-x)) + +(require 'gnus) +(require 'nnheader) +(require 'nnoo) +(require 'gnus-group) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'gnus) + +(nnoo-declare nnatom) + +(defvoo nnatom-backend 'nnatom + "Symbol which identifies this backend.") + +(defvoo nnatom-status-string nil + "Last status message reported by this backend.") + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "^https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-readable-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (when-let ((data (if (libxml-available-p) + (libxml-parse-xml-region + (point-min) (point-max)) + (car (xml-parse-region + (point-min) (point-max))))) + (auth (list 'authors))) + (dom-add-child-before data auth) + (catch :stop ; Collect feed authors, stop at first entry. + (dolist (child (cdddr data) data) + (let ((tag (car child))) + (if (eq tag 'entry) + (throw :stop data) + (and (or (eq tag 'author) + (eq tag 'contributor)) + (dom-add-child-before auth child)))))))))))) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) (setq data (cddr data))) + (while (and data (not (eq (caar data) 'entry))) (pop data)) + (when-let ((article (car data)) + (auths (list 'authors)) (links (list 'links))) + (dom-add-child-before article links) + (dom-add-child-before article auths) + (dolist (child (cddddr article) `(,article . ,(cdr data))) + (pcase (car child) ; Authors and links can appear + ((or 'author 'contributor) ; anywhere in the entry so we + (dom-add-child-before auths child) ; collect them all here to avoid + (dom-add-child-before links child)) ; looping over the entry multiple + ((or 'link ; times later. + (and 'content (guard (assq 'src (dom-attributes child))))) + (dom-add-child-before links child)))))) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) + +(defun nnatom--read-article-or-group-authors (article-or-group) + "Return the authors of ARTICLE-OR-GROUP, or nil." + (when-let + ((a (string-trim-right + (mapconcat (lambda (author) + (let* ((name + (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email + (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (concat + (or (and name email (format "%s <%s>" name email)) + name email) + ", "))) + (dom-children + (dom-child-by-tag article-or-group 'authors))) + ", ")) + ((not (string-blank-p a)))) + a)) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((rel (make-vector 6 0))) ; [ALTERNATE RELATED SELF ENCLOSURE VIA AUTHOR] + (mapcan + (lambda (link) + (when-let (((consp link)) + (l (car link))) + (or + (when-let (((eq l 'content)) + (src (dom-attr link 'src)) + (label (concat "Link" + (and (< 1 (cl-incf (aref rel 0))) + (format " %s" (aref rel 0)))))) + `(((("text/plain") . ,(format "%s: %s\n" label src)) + (("text/html") . ,(format "[%s] " + src label))))) + (when-let (((or (eq l 'author) (eq l 'contributor))) + (name (dom-text (dom-child-by-tag link 'name))) + (name (if (string-blank-p name) + (concat "Author" + (and (< 1 (cl-incf (aref rel 5))) + (format " %s" (aref rel 5)))) + name)) + (uri (dom-text (dom-child-by-tag link 'uri))) + ((not (string-blank-p uri)))) + `(((("text/plain") . ,(format "%s: %s\n" name uri)) + (("text/html") . ,(format "[%s] " + uri name))))) + (when-let (((eq l 'link)) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf (aref rel 1))) + (format " %s" (aref rel 1))))) + ("self" + (concat "More" + (and (< 1 (cl-incf (aref rel 2))) + (format " %s" (aref rel 2))))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf (aref rel 3))) + (format " %s" (aref rel 3))))) + ("via" + (concat "Source" + (and (< 1 (cl-incf (aref rel 4))) + (format " %s" (aref rel 4))))) + (_ (if-let + ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat + "Link" + (and (< 1 (cl-incf (aref rel 0))) + (format " %s" (aref rel 0))))))))) + (link (cdr (assq 'href attrs)))) + `(((("text/plain") . ,(format "%s: %s\n" label link)) + (("text/html") . ,(format "[%s] " + link label)))))))) + (dom-children (dom-child-by-tag article 'links))))) + +(defsubst nnatom--read-part (part type) + (let* ((atypes '("html" "plain")) + (mtypes '(("xhtml" . "text/html") ("text" . "text/plain"))) + (xsuff (concat "[+/]xml\\(-\\(dtd\\|external-parsed-entity\\)\\)?\\'" + "\\|^text")) + (part (if (string= type "xhtml") + (with-temp-buffer + (dom-print (dom-child-by-tag part 'div) nil t) + (buffer-substring-no-properties + (point-min) (point-max))) + (dom-text part))) + (type (if (member type atypes) (concat "text/" type) type)) + (type (or (cdr (assoc type mtypes)) type))) + (unless (string-blank-p part) + `(,part (Content-Type . ,(or type (setq type "text/plain"))) + ,(and (not (string-match-p xsuff type)) + '(Content-Transfer-Encoding . "base64")))))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (cdr (assq 'type (dom-attributes summary)))) + (summary (nnatom--read-part summary stype)) + (content (dom-child-by-tag article 'content)) + (ctype (cdr (assq 'type (dom-attributes content)))) + (content (nnatom--read-part content ctype)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push summary parts)) + (push (append content '(links)) parts) + (or st (push summary parts)) + parts) + ((setq parts (or summary content)) + `(,(append (or summary content) '(links)))) + (t '((nil (Content-Type . "text/html") links)))))) + +;;;; Feed I/O: + +(defvoo nnatom-read-feed-function #'nnatom--read-feed + "Function returning a Lisp object representing a feed (or part of it). +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnatom-read-group-function').") + +(defvoo nnatom-read-group-function #'nnatom--read-group + "Function returning a cons cell of a group and remaining data from a feed.") + +(defvoo nnatom-read-article-function #'nnatom--read-article + "Function returning a cons cell of an article and remaining data from a group. +It should accept a two arguments, a Lisp object representing a feed, +and a flag indicating whether the last article was stale (not new or updated).") + +(defvoo nnatom-read-title-function #'nnatom--read-title + "Function returning the title of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-description-function #'nnatom--read-description + "Function returning the description of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-group-author-function + #'nnatom--read-article-or-group-authors + "Function returning the author of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-id-function #'nnatom--read-id + "Function returning the ID of an article. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-subject-function #'nnatom--read-subject + "Function returning the subject of an article (a string). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + "Function returning the publish date of an article (a time value). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-update-date-function #'nnatom--read-update + "Function returning the update date of an article (a time value). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-authors + "Function returning the author of an article (a string). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-headers-function #'ignore + "Function returning an alist of article-wide MIME headers. +Each element of this alist maps a MIME header (a symbol, +i.e. `Content-Type') to its value. As a special case, `:boundary' +maps to a string which will serve as the boundary between article +parts. This must be supplied if a custom boundary is used in a +multipart content type header. The default boundary is \"-_nnatom_-\", +and is automatically modified to match the name of the back end. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-links-function #'nnatom--read-links + "Function returning all links contained in an article. +For the default `nnatom-print-content-function', it should return a +list of links, where each link is an alist mapping MIME content types +to links formatted for display in a part of that type. Each content +type may also be a list of content types. +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-parts-function #'nnatom--read-parts + "Function returning an alist associating parts of an article to their headers. +For the default `nnatom-print-content-function', each part should be a +string. Otherwise, it can be any Lisp object. The \"headers\" of +each part should be a list where each element is either a cons of a +MIME header (a symbol, i.e. `Content-Type') and its value (a string), +or any other Lisp object. MIME headers will be printed, the rest will +be passed on to `nnatom-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnatom-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnatom-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnatom--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnatom-backend) + "nn") + (gnus-newsgroup-savable-name server)) + gnus-directory)) + +(defun nnatom--read-server (server) + "Read SERVER's information from `nnatom-directory'." + (if-let ((f (nnatom--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnatom-servers)) + (nnheader-report nnatom-backend "Can't read %s" server))) + +(defun nnatom--write-server (server) + "Write SERVER's information to `nnatom-directory'." + (if-let ((f (nnatom--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnatom-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n" + (prin1-to-string s)) + t) + t) + (nnheader-report nnatom-backend "Can't write %s" f))) + +(defsubst nnatom--server-address (server) + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnatom-backend 'address) + (cddr (gnus-server-to-method + (concat + (symbol-name nnatom-backend) ":" + server))))) + server)) + server)) + +(defun nnatom--parse-feed (feed &optional group) + "Parse FEED into a new or existing server. +Optionally, only parse GROUP." + (let* ((feed (nnatom--server-address feed)) + (s (or (gethash feed nnatom-servers) (nnatom--read-server feed))) + (g (or (and (hash-table-p s) (gethash group s)) `[,group])) + (name group) ; (Maybe) fake name + (group (or (aref g 0) name)) ; Real name + data) + (when (setq data (funcall nnatom-read-feed-function feed group) + s (or (gethash feed nnatom-servers) + (make-hash-table :test 'equal))) + (while-let ((cg (or (and name `(,data)) + (funcall nnatom-read-group-function data))) + (cg (prog1 (car cg) (setq data (cdr cg))))) + (let* ((name (funcall nnatom-read-title-function cg)) ; Real name + (group (gethash name nnatom-group-names name)) ; (Maybe) fake name + (info (gnus-get-info + (concat (symbol-name nnatom-backend) "+" feed ":" group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnatom-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnatom-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnatom-read-article-function cg stale)) + (article (prog1 (car article) (setq cg (cdr article))))) + (when-let ((id (funcall nnatom-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnatom-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnatom-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (cl-incf max)))) + (publish (funcall nnatom-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnatom-read-author-function article) + group-author group) + ,(or (funcall nnatom-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnatom-read-links-function article) + ,(funcall nnatom-read-parts-function article) + ,(funcall nnatom-read-headers-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnatom-servers)))) + +;;;; Gnus backend functions: + +(defvoo nnatom-group nil + "Name of the current group.") + +(defvoo nnatom-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnatom-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS HEADERS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers. +- HEADERS is an alist associating article-wide MIME headers to their value.") + +(defvoo nnatom-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnatom-group-article-min-num 1 + "Minimum article number for the current group.") + +(defvar nnatom-date-format "%F %X" + "Format of displayed dates.") + +(nnoo-define-basics nnatom) + +(defsubst nnatom--current-server-no-prefix () + (string-remove-prefix (concat (symbol-name nnatom-backend) "+") + (nnoo-current-server nnatom-backend))) + +(defun nnatom--group-data (group &optional server) + (let ((s (gethash server nnatom-servers)) c) + (or (and (hash-table-p s) (gethash group s)) + (and (setq c (nnatom--current-server-no-prefix)) + (setq s (gethash c nnatom-servers)) + (hash-table-p s) (gethash group s)) + (catch :stop (maphash (lambda (n s) + (or (string= n server) + (string= n c) + (when-let (((hash-table-p s)) + (g (gethash group s))) + (throw :stop g)))) + nnatom-servers))))) + +(defun nnatom-retrieve-article (article group) + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string nnatom-date-format (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnatom-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnatom--current-server-no-prefix))) + (g (or (nnatom--group-data group server) + `[ nil ,nnatom-group-article-ids ,nnatom-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnatom-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnatom-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnatom-backend "Group %s not found" (or group "")))) + +(deffoo nnatom-open-server (server &optional defs backend) + (let ((backend (or backend 'nnatom)) + (a (nnatom--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (or (gethash server nnatom-servers) + (nnatom--read-server a))) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash group g nnatom-group-names))) + s)) + (or s (file-writable-p (nnatom--server-file a)) + (and (nnoo-close-server nnatom-backend server) + (nnheader-report + nnatom-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnatom-request-close () + (maphash (lambda (server _) + (nnatom--write-server + (nnatom--server-address server))) + nnatom-servers) + (setq nnatom-servers (make-hash-table :test 'equal) + nnatom-status-string nil) + t) + +(defun nnatom--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, possibly with LINKS added." + (let ((links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and content (format "%s\n\n" content)) + (mapconcat (lambda (link) + (cdr (assoc (cdr (assq 'Content-Type attributes)) link + (lambda (types type) + (if (stringp types) (string= types type) + (member type types)))))) + links))))) + +(defvoo nnatom-print-content-function #'nnatom--print-content + "Function returning a single piece of content (a string). +It should accept three arguments, a part and its attributes (as returned +by `nnatom-read-parts-function'), and links.") + +(defsubst nnatom--print-part (content headers mime links) + (concat + (mapconcat (lambda (header) + (when-let (((consp header)) + (m (car header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnatom-print-content-function content headers links))) + +(deffoo nnatom-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnatom--current-server-no-prefix))) + (g (or (nnatom--group-data group server) + (and (setq group nnatom-group) + `[ nil ,nnatom-group-article-ids + ,nnatom-group-articles + ,nnatom-group-article-max-num + ,nnatom-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((links (aref a 5)) + (parts (aref a 6)) + (headers (aref a 7)) + (boundary (or (cdr (assq :boundary headers)) + (format "-_%s_-" nnatom-backend))) + (multi (length> parts 1)) + (mime '( Content-Type Content-Disposition + Content-Transfer-Encoding))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnatom-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + (if (assq 'MIME-Version headers) "" "MIME-Version: 1.0\n") + (mapconcat (lambda (header) + (unless (keywordp (car header)) + (format "%s: %s\n" (car header) (cdr header)))) + headers) + (if multi + (if (assq 'Content-Type headers) "" + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary)) + (prog1 (nnatom--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (format "--%s\n%s\n" boundary + (nnatom--print-part + (car part) (cdr part) mime links))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnatom-backend "No such article"))) + +(deffoo nnatom-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnatom--current-server-no-prefix))) + (g (or (if fast (nnatom--group-data group server) + (setq server (nnatom--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnatom-group group + nnatom-group-article-ids (aref g 1) + nnatom-group-articles (aref g 2) + nnatom-group-article-max-num (aref g 3) + nnatom-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnatom-group-article-ids) + nnatom-group-article-min-num + nnatom-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnatom-backend "Group %s not found" group)))) + +(deffoo nnatom-close-group (group &optional server) + (and (string= group nnatom-group) + (setq nnatom-group nil + nnatom-group-article-ids (make-hash-table :test 'equal) + nnatom-group-articles (make-hash-table :test 'eql) + nnatom-group-article-max-num 0 + nnatom-group-article-min-num 1)) + (setq server (or server (nnatom--current-server-no-prefix))) + (nnatom--write-server server)) + +(deffoo nnatom-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnatom--parse-feed + (or server (nnatom--current-server-no-prefix)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnatom-request-post (&optional _server) + (nnheader-report nnatom-backend "%s is a read only backend" nnatom-backend)) + +;;;; Optional back end functions: + +(deffoo nnatom-retrieve-groups (_groups &optional server) + (nnatom-request-list (or server (nnatom--current-server-no-prefix))) + 'active) + +(deffoo nnatom-request-type (_group &optional _article) + 'unknown) + +(deffoo nnatom-request-group-description (group &optional server) + (when-let ((server (or server (nnatom--current-server-no-prefix))) + (g (nnatom--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnatom-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnatom--current-server-no-prefix))) + (s (gethash server nnatom-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnatom-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnatom--current-server-no-prefix))) + (s (or (gethash server nnatom-servers) + (and ; Open the server to add it to `nnatom-servers' + (save-match-data + (nnatom-open-server + server + (cdr ; Get defs and backend. + (assoc server (cdr (assq 'nnatom nnoo-state-alist)) + (lambda (car key) + (and (stringp car) + (string-match + (concat + "\\(\\(nn[[:alpha:]]+\\)\\+\\)?" + "\\(" (regexp-quote key) "\\)") + car) + (setq server car))))) + (prog1 (if (stringp (match-string 1 server)) + (intern (match-string 2 server)) + 'nnatom) + (setq server (match-string 3 server)))) + (gethash server nnatom-servers))))) + (g (or (nnatom--group-data group server) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnatom-group-names) + (remhash group s) + (and (string= group nnatom-group) + (setq nnatom-group new-name)) + t)) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +;;;; Utilities: + +(defmacro nnatom-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnatom-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnatom-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnatom-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnatom-servers) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnatom-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnatom-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnatom-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnatom-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnatom-open-server server defs ',backend)) + (nnoo-import ,backend (nnatom)))) + +(provide 'nnatom) + +;;; nnatom.el ends here -- 2.41.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Jul 01 04:39:01 2023 Received: (at 64202) by debbugs.gnu.org; 1 Jul 2023 08:39:01 +0000 Received: from localhost ([127.0.0.1]:56533 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qFW8D-0005e2-EK for submit@debbugs.gnu.org; Sat, 01 Jul 2023 04:39:01 -0400 Received: from eggs.gnu.org ([209.51.188.92]:40116) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qFW8B-0005dj-1Q for 64202@debbugs.gnu.org; Sat, 01 Jul 2023 04:39:00 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qFW84-0001uq-KJ; Sat, 01 Jul 2023 04:38:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=xk5mKA6Qem7YmwlrRsDxfmALNqeT853pJVg4TQjHzqU=; b=FjM1ar0/UgC3 guNplnO1w0p7feUWH7WueJOV6/iII8BXJW/mZN9nDh/O1M0dZ4SYLcCUTBwmWzPvrIWmPY5vjo75Y 2E8oIjTAe/IqiM2+4bqcWBcRa5+O9mFMuD8CJlDfgFaATw2z6b1XL7EChXGpPS9ZSd59BKB6CykEi /rbTBjwHf2xexWQdgbBnbRgE/+uyxwd51gPyA6h25wMEeL3LHWF6MT6dnh0HbPbQalIw0fSK2zw1Y sdgUScjjn6lrqzey9VVS0igC0BpRkcwue5fhR9ClCcJAJRlznmXW/h3vG5g2BSeCOl1CVLpixPAfO Iffa/qIf8fK6mt7AlR/9kg==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qFW83-0000vw-RC; Sat, 01 Jul 2023 04:38:52 -0400 Date: Sat, 01 Jul 2023 11:39:23 +0300 Message-Id: <83ilb4qb44.fsf@gnu.org> From: Eli Zaretskii To: Daniel Semyonov In-Reply-To: <87bkh2p96o.fsf@dsemy.com> (bug-gnu-emacs@gnu.org) Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64202 Cc: cohen@bu.edu, eric@ericabrahamsen.net, larsi@gnus.org, 64202@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: Andrew Cohen , Eric Abrahamsen , > Lars Ingebrigtsen , Stefan Monnier > Date: Mon, 26 Jun 2023 17:52:47 +0300 > From: Daniel Semyonov via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" > > >>>>> Daniel Semyonov writes: > > > Attached is an updated version of the patch. > > Actually attached this time. Thanks. Lars, Andrew, Eric: please install when you are satisfied with this new feature. From debbugs-submit-bounces@debbugs.gnu.org Sat Jul 01 07:35:40 2023 Received: (at 64202) by debbugs.gnu.org; 1 Jul 2023 11:35:40 +0000 Received: from localhost ([127.0.0.1]:56832 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qFYtA-0007L9-6T for submit@debbugs.gnu.org; Sat, 01 Jul 2023 07:35:40 -0400 Received: from dsemy.com ([46.23.89.208]:1803) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qFYt4-0007Kq-VO for 64202@debbugs.gnu.org; Sat, 01 Jul 2023 07:35:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=nPoonzNHD68wk Xzbv8XHy2fl5ItHbcupK3C+WYU7U9I=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=Rkb3+QYJdj4K19WCjTH4ZxED5Ryg/sMSDsw ZzsUeLVpBUhiEleR/YAy6/GPS4OSBQb7OBguIrjW31189qF+nXlLPH3AK5fjGuZWN15/XI m0GnT9+RpVo2t9cQUFKCC17gGtm4eOW8dazOdE9VqwrmIF1IiclEprPtF11ZYOA3UgJRt0 QgGEAPYGHrBO6hWcHvT0pVI8yT6iCon4cw0ZupzsGKcp6AX5OaD2ihUMJVStDbY8lFC8dg /2sCLsVX5fEnTrcRvc9XBwKcYQdPdCN5Ggfc+3JBgUxdpJ0BfA9WqClAM3GW174b7hhGSA dHDv/Y/cmrpxT1m76ofur5rVs0w== Received: from coldharbour.home.arpa (bzq-79-180-56-116.red.bezeqint.net [79.180.56.116]) by dsemy.com (OpenSMTPD) with ESMTPSA id 0e75aaf8 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sat, 1 Jul 2023 13:35:27 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id 2f2fa8d4; Sat, 1 Jul 2023 11:33:00 +0000 (UTC) From: Daniel Semyonov To: Eli Zaretskii Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <83ilb4qb44.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 01 Jul 2023 11:39:23 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> Date: Sat, 01 Jul 2023 14:33:00 +0300 Message-ID: <87r0prkgsz.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: cohen@bu.edu, eric@ericabrahamsen.net, larsi@gnus.org, 64202@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >>>>> Eli Zaretskii writes: >> >>>>> Daniel Semyonov writes: >> >> > Attached is an updated version of the patch. >> >> Actually attached this time. > Thanks. Lars, Andrew, Eric: please install when you are satisfied > with this new feature. I'll upload a new patch in (hopefully) a few days based on comments from Stefan, so please don't install anything yet. BTW, is there any interest in having this backend also support RSS feeds? Most feed readers transparently support both Atom and RSS, but I was initially reluctant to do this too since most Gnus backends are very specific and there is already an RSS backend. I'm also not sure what to name a backend which supports both RSS and Atom, and can be inherited from to support other types of "feeds". However, I do think that adding this support would be pretty easy, and Stefan also correctly pointed out to me that RSS has become a generic term for web feeds, and it seems that many websites today offer Atom feeds labeled as RSS (or using the well-known RSS icon as a link to it). Thanks, Daniel From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 02 18:59:14 2023 Received: (at 64202) by debbugs.gnu.org; 2 Jul 2023 22:59:14 +0000 Received: from localhost ([127.0.0.1]:32886 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qG62E-0004On-1c for submit@debbugs.gnu.org; Sun, 02 Jul 2023 18:59:14 -0400 Received: from mail.ericabrahamsen.net ([52.70.2.18]:43234) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qG628-0004OW-Ms for 64202@debbugs.gnu.org; Sun, 02 Jul 2023 18:59:12 -0400 Received: from localhost (97-113-52-110.tukw.qwest.net [97.113.52.110]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 821C8FA09E; Sun, 2 Jul 2023 22:59:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1688338741; bh=7v9QxIkszaRDC/MXRvxD3WkldxRj8PDmzsm8aPEPWvY=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=UbPBt31b1QufboLK+uO4gJIB4o7mSClLFs/lvyVTuqTTNTmo/1N7luTZ5fR7nw1Oc 5M53hORlHGhmwlsGPmwF/4Eq04RJo6VqJ+WyTpi9iPYSDNwJ1mULX00JfhqcUQ74vl xsa3Pi8yprZpPOPa8vK5cPqbDAiksHc6f7cW2u04= From: Eric Abrahamsen To: Daniel Semyonov Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87r0prkgsz.fsf@dsemy.com> (Daniel Semyonov's message of "Sat, 01 Jul 2023 14:33:00 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> <87r0prkgsz.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) Date: Sun, 02 Jul 2023 15:59:00 -0700 Message-ID: <873525udhn.fsf@ericabrahamsen.net> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64202 Cc: cohen@bu.edu, larsi@gnus.org, Eli Zaretskii , 64202@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On 07/01/23 14:33 PM, Daniel Semyonov wrote: >>>>>> Eli Zaretskii writes: > > >> >>>>> Daniel Semyonov writes: > >> > >> > Attached is an updated version of the patch. > >> > >> Actually attached this time. > > > Thanks. Lars, Andrew, Eric: please install when you are satisfied > > with this new feature. > > I'll upload a new patch in (hopefully) a few days based on comments from > Stefan, so please don't install anything yet. Thanks again for working on this -- I'm getting around to trying it out. I have a dumb first question: how do I create a Atom group? I wanted to try it out with the US weather.gov feed for my state: https://alerts.weather.gov/cap/wa.php?x=0 I tried with "a" in the *Server* buffer, "G m" in the *Group* buffer, but I guess neither allow for prompting the user for a URL. I also added (nnatom "alerts.weather.gov/cap/wa.php?x=0) to my `gnus-secondary-select-methods' variable, but it said "Server file alerts.weather.gov/cap/wa.php?x=0 not readable or writable" when I restarted Gnus. I guess because we're going straight to `nnatom-open-server', which reads a local file, without ever fetching the file. We'd have to hit `nnatom--parse-feed' to do that, but I don't see how we arrive there without already having the server created. What am I missing? From debbugs-submit-bounces@debbugs.gnu.org Sun Jul 02 20:02:43 2023 Received: (at 64202) by debbugs.gnu.org; 3 Jul 2023 00:02:43 +0000 Received: from localhost ([127.0.0.1]:32942 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qG71e-00061k-L5 for submit@debbugs.gnu.org; Sun, 02 Jul 2023 20:02:43 -0400 Received: from dsemy.com ([46.23.89.208]:21683) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qG71b-00061T-AN for 64202@debbugs.gnu.org; Sun, 02 Jul 2023 20:02:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=Rlv9WmXDJ1cWE BKM0Xg8xqy03AwwVfKmJG8Joz33qMs=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=Pz1gaGyLcDgY/YIOu/jkIlbmcdePUdJpWYs t9lNaypLkUx0Wldjn1TqZsHT7q/lk3uGrud8VJ55zGfJZSdubjWvZhNDf+qP6WUs8IAJ74 fRDGCGxvYaUaoH91pUWRX4M8mc/eo+/qypLCRko+fjh0ZDE1V2u3++cu7D/7nB8O2+ZqUs hxo9NNxGzT8WErnlEYfx9tnbjhixGuOxVbqZ+/zqGGSf526aKXPBlVg+oCeiFLxBHNh8Wr nOcqFUMXZj6zqCVsboLFZnuUi5aMstLAD2wOmuucmACAUGajzMkARt5LgicbgwJj9WZsLk tkfL6M24r5QUIcr98bPCK5Vb1bQ== Received: from coldharbour.home.arpa (bzq-79-180-56-116.red.bezeqint.net [79.180.56.116]) by dsemy.com (OpenSMTPD) with ESMTPSA id 22881505 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 3 Jul 2023 02:02:32 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id 8c27eb6f; Mon, 3 Jul 2023 00:00:14 +0000 (UTC) From: Daniel Semyonov To: Eric Abrahamsen Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <873525udhn.fsf@ericabrahamsen.net> (Eric Abrahamsen's message of "Sun, 02 Jul 2023 15:59:00 -0700") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> <87r0prkgsz.fsf@dsemy.com> <873525udhn.fsf@ericabrahamsen.net> Date: Mon, 03 Jul 2023 03:00:14 +0300 Message-ID: <87pm59984x.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: cohen@bu.edu, larsi@gnus.org, Eli Zaretskii , 64202@debbugs.gnu.org, monnier@iro.umontreal.ca X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >>>>> Eric Abrahamsen writes: > Thanks again for working on this -- I'm getting around to trying > it out. I have a dumb first question: how do I create a Atom > group? I wanted to try it out with the US weather.gov feed for my > state: https://alerts.weather.gov/cap/wa.php?x=0 > I tried with "a" in the *Server* buffer, "G m" in the *Group* > buffer, but I guess neither allow for prompting the user for a > URL. You can use "B nnatom RET
RET" in the group buffer for this purpose. > I also added > (nnatom "alerts.weather.gov/cap/wa.php?x=0) > to my `gnus-secondary-select-methods' variable, but it said > "Server file alerts.weather.gov/cap/wa.php?x=0 not readable or > writable" when I restarted Gnus. I guess because we're going > straight to `nnatom-open-server', which reads a local file, > without ever fetching the file. We'd have to hit > `nnatom--parse-feed' to do that, but I don't see how we arrive > there without already having the server created. The server should be allowed to open without an existing local file, as long as that local file is writable, however... > What am I missing? You're not missing anything, you actually just reminded me of a bug I forgot to fix - the directory holding the local feed data isn't automatically created if it's missing, which causes 'file-writable-p' to return nil, preventing the server from opening. As a workaround, manually create an "atom" subdir under 'gnus-directory'. I'll fix this in the next version. However, this particular feed still doesn't work since the comment at the start of the feed changes the structure of the parsed representation of the feed when libxml is supported (this also made me realize I accidentally broke parsing without libxml in the last version, oops). This is very useful info, as this is the first feed I encountered like this, and I will fix both those issues too in the next version. As a quick fix, redefine 'nnatom--read-feed' as: (defun nnatom--read-feed (feed _) "Return a list structure representing FEED, or nil." (if (string-match-p "^https?://" feed) (nnheader-report nnatom-backend "Address shouldn't start with \"http://\" or \"https://\"") (with-temp-buffer (condition-case e (if (file-readable-p feed) (insert-file-contents feed) (mm-url-insert-file-contents (concat "https://" feed))) (file-error (nnheader-report nnatom-backend (cdr e))) (:success (when-let ((data (if (libxml-available-p) (libxml-parse-xml-region (point-min) (point-max)) (car (xml-parse-region (point-min) (point-max))))) (auth (list 'authors))) (when (eq (car data) 'top) (setq data (assq 'feed data))) (dom-add-child-before data auth) (catch :stop ; Collect feed authors, stop at first entry. (dolist (child (cdddr data) data) (let ((tag (car-safe child))) (if (eq tag 'entry) (throw :stop data) (and (or (eq tag 'author) (eq tag 'contributor)) (dom-add-child-before auth child)))))))))))) Thanks, Daniel From debbugs-submit-bounces@debbugs.gnu.org Mon Jul 03 00:02:28 2023 Received: (at submit) by debbugs.gnu.org; 3 Jul 2023 04:02:28 +0000 Received: from localhost ([127.0.0.1]:33000 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qGAlg-0003qc-1n for submit@debbugs.gnu.org; Mon, 03 Jul 2023 00:02:28 -0400 Received: from lists.gnu.org ([209.51.188.17]:54454) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qGAlb-0003qR-8t for submit@debbugs.gnu.org; Mon, 03 Jul 2023 00:02:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qGAla-0002Pk-8N for bug-gnu-emacs@gnu.org; Mon, 03 Jul 2023 00:02:22 -0400 Received: from ciao.gmane.io ([116.202.254.214]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qGAlY-0001y5-1d for bug-gnu-emacs@gnu.org; Mon, 03 Jul 2023 00:02:21 -0400 Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1qGAlT-0002dE-SK for bug-gnu-emacs@gnu.org; Mon, 03 Jul 2023 06:02:15 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: bug-gnu-emacs@gnu.org From: Eric Abrahamsen Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) Date: Sun, 02 Jul 2023 21:01:59 -0700 Message-ID: <87a5wdskw8.fsf@ericabrahamsen.net> References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> <87r0prkgsz.fsf@dsemy.com> <873525udhn.fsf@ericabrahamsen.net> <87pm59984x.fsf@dsemy.com> Mime-Version: 1.0 Content-Type: text/plain User-Agent: Gnus/5.13 (Gnus v5.13) Cancel-Lock: sha1:3pOGjItEhKFneVT0y2vK/qx/kXg= Received-SPF: pass client-ip=116.202.254.214; envelope-from=geb-bug-gnu-emacs@m.gmane-mx.org; helo=ciao.gmane.io X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.1 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.1 (--) Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" writes: >>>>>> Eric Abrahamsen writes: > > > Thanks again for working on this -- I'm getting around to trying > > it out. I have a dumb first question: how do I create a Atom > > group? I wanted to try it out with the US weather.gov feed for my > > state: https://alerts.weather.gov/cap/wa.php?x=0 > > > I tried with "a" in the *Server* buffer, "G m" in the *Group* > > buffer, but I guess neither allow for prompting the user for a > > URL. > > You can use "B nnatom RET
RET" in the group buffer for this > purpose. Huh! In all my years of using and working on Gnus I've never used a "foreign" server, nor have I really understood what it means. At some point it would be good to make sure this works via other entrypoints as well, but so far so good. > > I also added > > > (nnatom "alerts.weather.gov/cap/wa.php?x=0) > > > to my `gnus-secondary-select-methods' variable, but it said > > "Server file alerts.weather.gov/cap/wa.php?x=0 not readable or > > writable" when I restarted Gnus. I guess because we're going > > straight to `nnatom-open-server', which reads a local file, > > without ever fetching the file. We'd have to hit > > `nnatom--parse-feed' to do that, but I don't see how we arrive > > there without already having the server created. > > The server should be allowed to open without an existing local file, as > long as that local file is writable, however... > > > What am I missing? > > You're not missing anything, you actually just reminded me of a bug I > forgot to fix - the directory holding the local feed data isn't > automatically created if it's missing, which causes 'file-writable-p' to > return nil, preventing the server from opening. As a workaround, > manually create an "atom" subdir under 'gnus-directory'. I'll fix this > in the next version. Yup, that did the trick. > However, this particular feed still doesn't work since the comment at > the start of the feed changes the structure of the parsed representation > of the feed when libxml is supported (this also made me realize I > accidentally broke parsing without libxml in the last version, oops). > This is very useful info, as this is the first feed I encountered like > this, and I will fix both those issues too in the next version. > > As a quick fix, redefine 'nnatom--read-feed' as: > > (defun nnatom--read-feed (feed _) > "Return a list structure representing FEED, or nil." > (if (string-match-p "^https?://" feed) > (nnheader-report > nnatom-backend > "Address shouldn't start with \"http://\" or \"https://\"") > (with-temp-buffer > (condition-case e > (if (file-readable-p feed) > (insert-file-contents feed) > (mm-url-insert-file-contents (concat "https://" feed))) > (file-error (nnheader-report nnatom-backend (cdr e))) > (:success (when-let ((data (if (libxml-available-p) > (libxml-parse-xml-region > (point-min) (point-max)) > (car (xml-parse-region > (point-min) (point-max))))) > (auth (list 'authors))) > (when (eq (car data) 'top) > (setq data (assq 'feed data))) > (dom-add-child-before data auth) > (catch :stop ; Collect feed authors, stop at first entry. > (dolist (child (cdddr data) data) > (let ((tag (car-safe child))) > (if (eq tag 'entry) > (throw :stop data) > (and (or (eq tag 'author) > (eq tag 'contributor)) > (dom-add-child-before auth child)))))))))))) Yes, that fixed it. I'll do some more poking around. Regarding your earlier question about having this backend handle RSS too, I'm not aware of any significant difference between the two beyond the format of the XML. Is that true? If so, it seems like it would make most sense to merge the code. Have you looked at nnrss? It would be good to know if there was anything in there worth stealing for nnatom -- if one of them has a faster parser than the other, for instance, or better logic for keeping updates efficient. I just subscribed to a feed with nnrss, and noticed that after I marked all the items in the feed as read, I couldn't re-enter the group and see the old items. It gave me "Can't select group". So that's not very encouraging. If you do want to expand this to be a general "feed" backend, we might want to do some boring things like rename it nnfeed.el, and add support for ridiculous things like JSON feed[0] (why?!?). I assume a derived backend could handle JSON feeds by setting the appropriate values for the `nnatom-read-*-function' deffoos? One of the awkward things about nnrss is that it's never really fit well into Gnus' one-server-many-groups paradigm, which you allude to in the nnatom Info section. Do you have any further ideas in that direction? Thanks, Eric [0]: https://www.jsonfeed.org/ From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 05 08:39:34 2023 Received: (at 64202) by debbugs.gnu.org; 5 Jul 2023 12:39:34 +0000 Received: from localhost ([127.0.0.1]:37882 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qH1nB-0005CX-NK for submit@debbugs.gnu.org; Wed, 05 Jul 2023 08:39:34 -0400 Received: from dsemy.com ([46.23.89.208]:3798) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qH1n9-0005CF-2J for 64202@debbugs.gnu.org; Wed, 05 Jul 2023 08:39:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=NaU8so+3k8TDi Xgr7HLJDdD2pyCBmIDHMtfOWAPJGF8=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=nv4LBLPcM5LbJS8hTUV8iM8qWVrtTDb2lGN RYck4t3Bzt6/gkN5KYMaFfFCr0YXcWyFH1lznnlh1mV+5f+fmZBVgdQCpaXILhZ1t/OnZq DpLzt6B2lPk373htcL2GLgoFmkqx9UFuYvSRVzOfgxMcxqu2R7S1qcTNCXFXfYWKDY4Ki2 oFNiOnzhHbXTVPdslF5AtZNkthjpil0MF7CEnDGxZIh0KOts5kbj0So49cAbsb+da6Sool xgS88KY5yCCAaXlGREiwyr858frGLqbCTpOTuFmXJpv7kSFxiPOTOIdtbouYugOtAuaC7h H/yR+JZMJges/TB35GNNOlargdQ== Received: from coldharbour.home.arpa (bzq-79-180-56-116.red.bezeqint.net [79.180.56.116]) by dsemy.com (OpenSMTPD) with ESMTPSA id 5640fd5c (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Wed, 5 Jul 2023 14:39:24 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id 8b358bc9; Wed, 5 Jul 2023 12:36:54 +0000 (UTC) From: Daniel Semyonov To: Eric Abrahamsen Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87a5wdskw8.fsf@ericabrahamsen.net> (Eric Abrahamsen's message of "Sun, 02 Jul 2023 21:01:59 -0700") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> <87r0prkgsz.fsf@dsemy.com> <873525udhn.fsf@ericabrahamsen.net> <87pm59984x.fsf@dsemy.com> <87a5wdskw8.fsf@ericabrahamsen.net> Date: Wed, 05 Jul 2023 15:36:54 +0300 Message-ID: <87edlm5yc9.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Sorry, I missed this email (I didn't realize debbugs doesn't forward all messages to the author of the "bug report"). >>>>> Eric Abrahamsen writes: > Huh! In all my years of using and working on Gnus I've never used > a "foreign" server, nor have I really understood what it means. At > some point it would be good to make sure this works via other > entrypoints as well, but so far so good. AFAIK it should work for any entry point which allows you to define a new server or select method (Atom feeds are represented as servers with a single group, so there is no way to create a group without a pre-existing corresponding server). I'll be honest though, I only ever use this method and 'gnus-secondary-select-methods', so I'm not really sure which other entry points there are. > Regarding your earlier question about having this backend handle > RSS too, I'm not aware of any significant difference between the > two beyond the format of the XML. Is that true? Yes, even the XML format is very similar. > If so, it seems like it would make most sense to merge the > code. Have you looked at nnrss? It would be good to know if there > was anything in there worth stealing for nnatom -- if one of them > has a faster parser than the other, for instance, or better logic > for keeping updates efficient. The issue with merging the two is that nnrss saves feed data differently (both on disk and in memory), and also represents each feed as a group, with a virtual "server" holding all groups. I'm not sure if it's possible/a good idea to migrate feed data from nnrss to a hypothetical merged backend, at least not automatically. Stefan floated the idea of adding RSS support, deprecating nnrss and creating an interactive migration command - so users who wish to migrate will have to do so manually (which should also potentially allow asking the user some questions if the migration includes some non-trivial steps). nnrss does do some cool stuff that nnatom doesn't, though (for example, it tries very hard to find an RSS feed when you provide it with a link to a website, while nnatom currently requires a direct link to a feed). > I just subscribed to a feed with nnrss, and noticed that after I > marked all the items in the feed as read, I couldn't re-enter the > group and see the old items. It gave me "Can't select group". So > that's not very encouraging. Honestly, from my experience nnrss has many small issues (although I never encountered this exact issue). It is partly why I developed nnatom (previously I used a hack documented on the Emacs wiki which converted Atom feeds to RSS feeds on the fly). > If you do want to expand this to be a general "feed" backend, we > might want to do some boring things like rename it nnfeed.el, This is the name I thought of too, and I guess if two people thought of it independently it's probably fine. > and add support for ridiculous things like JSON feed[0] (why?!?). I > assume a derived backend could handle JSON feeds by setting the > appropriate values for the `nnatom-read-*-function' deffoos? I actually attached a derivative "nnjsonfeed" backend I made as an experiment to one of my previous messages (it doesn't work with the current version of the patch, but it won't be hard to fix). It wasn't 100% conforming to the standard (JSON feeds support some weird features like pagination, which can actually be supported very well in theory by nnatom, but I didn't feel like doing that), but it worked and it was very easy to make. > One of the awkward things about nnrss is that it's never really > fit well into Gnus' one-server-many-groups paradigm, which you > allude to in the nnatom Info section. Do you have any further > ideas in that direction? Well, nnatom theoretically supports this paradigm, but it doesn't do this with Atom feeds, since it doesn't really make sense IMO. However, there is a standardized way to include links to Atom feeds in HTML documents, so it might be a good idea to support adding them as servers which show any linked Atom feeds as groups. I also have a (very experimental) derivative backend using the API of some website, which exposes various categories of content, which I expose as groups (this is what I use to test support for multiple groups in a single server). As a side note: I had hoped to publish an updated version of the patch by now, but unfortunately I was a bit under the weather lately, so I didn't feel like working on it. > Thanks, Eric Thanks for your testing and feedback, Daniel From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 05 13:18:15 2023 Received: (at submit) by debbugs.gnu.org; 5 Jul 2023 17:18:16 +0000 Received: from localhost ([127.0.0.1]:38893 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qH68t-0004IB-CJ for submit@debbugs.gnu.org; Wed, 05 Jul 2023 13:18:15 -0400 Received: from lists.gnu.org ([209.51.188.17]:60610) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qH68r-0004I4-If for submit@debbugs.gnu.org; Wed, 05 Jul 2023 13:18:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qH68o-0001T5-W5 for bug-gnu-emacs@gnu.org; Wed, 05 Jul 2023 13:18:11 -0400 Received: from mail.ericabrahamsen.net ([52.70.2.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qH68m-0002za-TF for bug-gnu-emacs@gnu.org; Wed, 05 Jul 2023 13:18:10 -0400 Received: from localhost (c-71-197-232-41.hsd1.wa.comcast.net [71.197.232.41]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 372A2FA002; Wed, 5 Jul 2023 17:17:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1688577478; bh=YxC0YX0k8gklCooI/5uWX3n7zRA1Cgfy7otalsMPXQ8=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=QqQ1XghSatuPifjRCXW+PWJ0fj7mXoJy8zjoJH9t8dpmzOtyEr59f/7wkY2x90E1s xMDZF1G7W4CpGgEMDyqz5md5Vy2oxKJbUplJPSp92BmnJ/7P75jRRFG84S9JsCILrh roaPbFVqmfvR7JsMSr3z0PfomTHgs1pODvYGqJI4= From: Eric Abrahamsen To: Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87edlm5yc9.fsf@dsemy.com> (Daniel Semyonov via's message of "Wed, 05 Jul 2023 15:36:54 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> <87r0prkgsz.fsf@dsemy.com> <873525udhn.fsf@ericabrahamsen.net> <87pm59984x.fsf@dsemy.com> <87a5wdskw8.fsf@ericabrahamsen.net> <87edlm5yc9.fsf@dsemy.com> Date: Wed, 05 Jul 2023 10:17:56 -0700 Message-ID: <874jmie0qj.fsf@ericabrahamsen.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=52.70.2.18; envelope-from=eric@ericabrahamsen.net; helo=mail.ericabrahamsen.net X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit Cc: 64202@debbugs.gnu.org, Daniel Semyonov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" writes: > Sorry, I missed this email (I didn't realize debbugs doesn't forward all > messages to the author of the "bug report"). No worries, usually I do a reply all, but sometimes forget. >>>>>> Eric Abrahamsen writes: > > > Huh! In all my years of using and working on Gnus I've never used > > a "foreign" server, nor have I really understood what it means. At > > some point it would be good to make sure this works via other > > entrypoints as well, but so far so good. > > AFAIK it should work for any entry point which allows you to define a > new server or select method (Atom feeds are represented as servers with > a single group, so there is no way to create a group without a > pre-existing corresponding server). > I'll be honest though, I only ever use this method and > 'gnus-secondary-select-methods', so I'm not really sure which other > entry points there are. What I usually start with is "G m", for `gnus-group-make-group'. That prompts for a group name, and also a backend. The viable list of backends includes all your currently-defined servers, plus the generic symbols for all defined backends, in this case a plain 'nnatom. As you point out, that doesn't work for nnatom because each group will need its own server. What *does* work is to create a group this way, then put point on the new group and hit "M-e", for `gnus-group-edit-group-method', put in the actual URL, then "C-c C-c". Obviously this is horribly baroque and it would be better if `gnus-group-make-group' could prompt for the URL. No matter what, it would be good to document the various ways to do this. > > Regarding your earlier question about having this backend handle > > RSS too, I'm not aware of any significant difference between the > > two beyond the format of the XML. Is that true? > > Yes, even the XML format is very similar. > > > If so, it seems like it would make most sense to merge the > > code. Have you looked at nnrss? It would be good to know if there > > was anything in there worth stealing for nnatom -- if one of them > > has a faster parser than the other, for instance, or better logic > > for keeping updates efficient. > > The issue with merging the two is that nnrss saves feed data differently > (both on disk and in memory), and also represents each feed as a group, > with a virtual "server" holding all groups. > I'm not sure if it's possible/a good idea to migrate feed data from > nnrss to a hypothetical merged backend, at least not automatically. > > Stefan floated the idea of adding RSS support, deprecating nnrss and > creating an interactive migration command - so users who wish to migrate > will have to do so manually (which should also potentially allow asking > the user some questions if the migration includes some non-trivial > steps). > > nnrss does do some cool stuff that nnatom doesn't, though (for example, > it tries very hard to find an RSS feed when you provide it with a link > to a website, while nnatom currently requires a direct link to a feed). Right, I should have been more explicit here -- what I was thinking is what Stefan suggests: just deprecate nnrss altogether. If it does anything cool like feed discovery, just steal that code. If you're inclined to be nice enough to provide a migration function, that's a bonus. > > I just subscribed to a feed with nnrss, and noticed that after I > > marked all the items in the feed as read, I couldn't re-enter the > > group and see the old items. It gave me "Can't select group". So > > that's not very encouraging. > > Honestly, from my experience nnrss has many small issues (although I > never encountered this exact issue). It is partly why I developed > nnatom (previously I used a hack documented on the Emacs wiki which > converted Atom feeds to RSS feeds on the fly). > > > If you do want to expand this to be a general "feed" backend, we > > might want to do some boring things like rename it nnfeed.el, > > This is the name I thought of too, and I guess if two people thought of > it independently it's probably fine. Good! > > and add support for ridiculous things like JSON feed[0] (why?!?). I > > assume a derived backend could handle JSON feeds by setting the > > appropriate values for the `nnatom-read-*-function' deffoos? > > I actually attached a derivative "nnjsonfeed" backend I made as an > experiment to one of my previous messages (it doesn't work with the > current version of the patch, but it won't be hard to fix). > > It wasn't 100% conforming to the standard (JSON feeds support some weird > features like pagination, which can actually be supported very well in > theory by nnatom, but I didn't feel like doing that), but it worked and > it was very easy to make. I missed nnjsonfeed, sorry. That's great you're already working in this direction. I don't think we need to support absolutely everything it does (what would pagination look like in Gnus?), just the basics to get started with. > > One of the awkward things about nnrss is that it's never really > > fit well into Gnus' one-server-many-groups paradigm, which you > > allude to in the nnatom Info section. Do you have any further > > ideas in that direction? > > Well, nnatom theoretically supports this paradigm, but it doesn't do > this with Atom feeds, since it doesn't really make sense IMO. > However, there is a standardized way to include links to Atom feeds in > HTML documents, so it might be a good idea to support adding them as > servers which show any linked Atom feeds as groups. It's also perfectly possible that a single website would publish multiple Atom feeds, right? Like weather.gov, for instance. That would have an added benefit of letting us simplify the server name from the full URL (https://alerts.weather.gov/cap/wa.php?x=0) which looks ugly in the Group buffer, to just eg alerts.weather.gov. But I guess I don't know why it would matter, really. The only practical use for wanting multiple feeds under a single server would be to set some configuration at the server level, that would apply to all feeds from that server. And at the moment I don't think there are many such settings (right?). > I also have a (very experimental) derivative backend using the API of > some website, which exposes various categories of content, which I > expose as groups (this is what I use to test support for multiple groups > in a single server). > > As a side note: I had hoped to publish an updated version of the patch > by now, but unfortunately I was a bit under the weather lately, so I > didn't feel like working on it. No rush! Eric From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 05 14:52:57 2023 Received: (at submit) by debbugs.gnu.org; 5 Jul 2023 18:52:57 +0000 Received: from localhost ([127.0.0.1]:38950 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qH7cW-0000uZ-F0 for submit@debbugs.gnu.org; Wed, 05 Jul 2023 14:52:57 -0400 Received: from lists.gnu.org ([209.51.188.17]:38886) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qH7cV-0000uS-0h for submit@debbugs.gnu.org; Wed, 05 Jul 2023 14:52:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qH7cU-0002Rf-NH for bug-gnu-emacs@gnu.org; Wed, 05 Jul 2023 14:52:54 -0400 Received: from dsemy.com ([46.23.89.208]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qH7cS-00020J-Ho for bug-gnu-emacs@gnu.org; Wed, 05 Jul 2023 14:52:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=tACbZ+ETfmyK6 8jHGm3eWNK972mgcic9Tigy3wJZ5nk=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=CGkryNz/cVSaVCUmN5XQaNd7tnCujQbeHM8 kCNoGtZYIyqdZUHz+MweqYx6diipKV0qpxLVZw4cs4D0uwO+D8grGi56Axkh28r+NQpOQK DgUX/UZZu3noftoqOhDN59QFEFA8LCU63Y72gcOThzwrSoe82k6lUDqcWj+8slKDQk/Cxj TQ3/ak70SVHCoGiHvOIIFp0X/kNeOCXaiKoY/OuQ6sAc91w4i9CooLSKU7U7C1leDsJNgZ D5r6nueEJar9jqdOMJIXe7BIjTeLK6R1IGRRFSVNwVfycXrQ/Wp1W1Dnet0XpkEZAly5uw +THJkFp9OtuNKlLRuXgGXqVAg9w== Received: from coldharbour.home.arpa (bzq-79-180-56-116.red.bezeqint.net [79.180.56.116]) by dsemy.com (OpenSMTPD) with ESMTPSA id f529ef45 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Wed, 5 Jul 2023 20:52:44 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id c9f2f1b4; Wed, 5 Jul 2023 18:50:16 +0000 (UTC) From: Daniel Semyonov To: Eric Abrahamsen Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <874jmie0qj.fsf@ericabrahamsen.net> (Eric Abrahamsen's message of "Wed, 05 Jul 2023 10:17:56 -0700") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> <87r0prkgsz.fsf@dsemy.com> <873525udhn.fsf@ericabrahamsen.net> <87pm59984x.fsf@dsemy.com> <87a5wdskw8.fsf@ericabrahamsen.net> <87edlm5yc9.fsf@dsemy.com> <874jmie0qj.fsf@ericabrahamsen.net> Date: Wed, 05 Jul 2023 21:50:16 +0300 Message-ID: <87bkgq2nx3.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=46.23.89.208; envelope-from=daniel@dsemy.com; helo=dsemy.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit Cc: "Daniel Semyonov via Bug reports for GNU Emacs, the Swiss army knife of text editors" , 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) >>>>> Eric Abrahamsen writes: > Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army > knife of text editors" writes: >> AFAIK it should work for any entry point which allows you to >> define a new server or select method (Atom feeds are represented >> as servers with a single group, so there is no way to create a >> group without a pre-existing corresponding server). I'll be >> honest though, I only ever use this method and >> 'gnus-secondary-select-methods', so I'm not really sure which >> other entry points there are. > What I usually start with is "G m", for > `gnus-group-make-group'. That prompts for a group name, and also a > backend. The viable list of backends includes all your > currently-defined servers, plus the generic symbols for all > defined backends, in this case a plain 'nnatom. > As you point out, that doesn't work for nnatom because each group > will need its own server. What *does* work is to create a group > this way, then put point on the new group and hit "M-e", for > `gnus-group-edit-group-method', put in the actual URL, then "C-c > C-c". > Obviously this is horribly baroque and it would be better if > `gnus-group-make-group' could prompt for the URL. No matter what, > it would be good to document the various ways to do this. Weirdly enough, 'gnus-group-make-group' accepts an ADDRESS argument, and the docstring even claims it prompts for it, but the function fails if you pass it an address it seems. However, since Gnus knows which backends require an address, and adds it to the name of the method when present, it shouldn't be hard to fix this command to actually prompt for an address when needed. I haven't looked deeply enough into how that function works to know whether it would actually work as expected in that case though. >> > Regarding your earlier question about having this backend >> handle > RSS too, I'm not aware of any significant difference >> between the > two beyond the format of the XML. Is that true? >> >> Yes, even the XML format is very similar. >> >> > If so, it seems like it would make most sense to merge the > >> code. Have you looked at nnrss? It would be good to know if there >> > was anything in there worth stealing for nnatom -- if one of >> them > has a faster parser than the other, for instance, or >> better logic > for keeping updates efficient. >> >> The issue with merging the two is that nnrss saves feed data >> differently (both on disk and in memory), and also represents >> each feed as a group, with a virtual "server" holding all groups. >> I'm not sure if it's possible/a good idea to migrate feed data >> from nnrss to a hypothetical merged backend, at least not >> automatically. >> >> Stefan floated the idea of adding RSS support, deprecating nnrss >> and creating an interactive migration command - so users who wish >> to migrate will have to do so manually (which should also >> potentially allow asking the user some questions if the migration >> includes some non-trivial steps). >> >> nnrss does do some cool stuff that nnatom doesn't, though (for >> example, it tries very hard to find an RSS feed when you provide >> it with a link to a website, while nnatom currently requires a >> direct link to a feed). > Right, I should have been more explicit here -- what I was > thinking is what Stefan suggests: just deprecate nnrss > altogether. If it does anything cool like feed discovery, just > steal that code. If you're inclined to be nice enough to provide a > migration function, that's a bonus. I guess I'll see how hard it would be. First I have to actually implement RSS support though. >> > I just subscribed to a feed with nnrss, and noticed that after >> I > marked all the items in the feed as read, I couldn't re-enter >> the > group and see the old items. It gave me "Can't select >> group". So > that's not very encouraging. >> >> Honestly, from my experience nnrss has many small issues >> (although I never encountered this exact issue). It is partly >> why I developed nnatom (previously I used a hack documented on >> the Emacs wiki which converted Atom feeds to RSS feeds on the >> fly). >> >> > If you do want to expand this to be a general "feed" backend, >> we > might want to do some boring things like rename it >> nnfeed.el, >> >> This is the name I thought of too, and I guess if two people >> thought of it independently it's probably fine. > Good! >> > and add support for ridiculous things like JSON feed[0] >> (why?!?). I > assume a derived backend could handle JSON feeds by >> setting the > appropriate values for the `nnatom-read-*-function' >> deffoos? >> >> I actually attached a derivative "nnjsonfeed" backend I made as >> an experiment to one of my previous messages (it doesn't work >> with the current version of the patch, but it won't be hard to >> fix). >> >> It wasn't 100% conforming to the standard (JSON feeds support >> some weird features like pagination, which can actually be >> supported very well in theory by nnatom, but I didn't feel like >> doing that), but it worked and it was very easy to make. > I missed nnjsonfeed, sorry. That's great you're already working in > this direction. I don't think we need to support absolutely > everything it does (what would pagination look like in Gnus?), > just the basics to get started with. The feed itself is paginated, meaning that the entries are spread across multiple files, with each one pointing to the next with a special tag. The design of nnatom allows for this since it doesn't necessarily require parsing the entire feed ahead of time, and also notifies the backend when a stale (not new or updated) article is reached, so the backend can intelligently stop (instead of trying to parse thousands of pages every time you update a group, for example). These capabilities were actually added in anticipation of a JSON feed backend, the Atom parser doesn't use them at all. Gnus itself won't show pages of course, just a list of articles as normal, but they will come from different JSON feed pages. >> > One of the awkward things about nnrss is that it's never really >> > fit well into Gnus' one-server-many-groups paradigm, which you >> > allude to in the nnatom Info section. Do you have any further > >> ideas in that direction? >> >> Well, nnatom theoretically supports this paradigm, but it doesn't >> do this with Atom feeds, since it doesn't really make sense IMO. >> However, there is a standardized way to include links to Atom >> feeds in HTML documents, so it might be a good idea to support >> adding them as servers which show any linked Atom feeds as >> groups. > It's also perfectly possible that a single website would publish > multiple Atom feeds, right? Like weather.gov, for instance. That > would have an added benefit of letting us simplify the server name > from the full URL (https://alerts.weather.gov/cap/wa.php?x=0) > which looks ugly in the Group buffer, to just eg > alerts.weather.gov. Yes, that's actually what I meant, although looking at real world examples, it seems most websites either only link to one Atom feed in the "correct" way, or don't link to any at all in that way. doesn't link to any feed in the standard way at all, for example. > But I guess I don't know why it would matter, really. The only > practical use for wanting multiple feeds under a single server > would be to set some configuration at the server level, that would > apply to all feeds from that server. And at the moment I don't > think there are many such settings (right?). Actually, since the various parsing functions are stored in server variables, all of them can be changed on a per-server basis, which is pretty cool. As an example, after I add support for fetching feeds contained in HTML, I can imagine a scenario where a user adds a website which publishes multiple, non-standard feeds, and fixes the parsing of all of them at once by providing some customized parsing functions for that specific server. This is a contrived example, but I know at least of youtube, which publishes slightly customized Atom feeds for each channel, so this is already useful. Daniel From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 05 15:08:14 2023 Received: (at submit) by debbugs.gnu.org; 5 Jul 2023 19:08:14 +0000 Received: from localhost ([127.0.0.1]:38958 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qH7rJ-0001L8-8s for submit@debbugs.gnu.org; Wed, 05 Jul 2023 15:08:14 -0400 Received: from lists.gnu.org ([209.51.188.17]:39096) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qH7rG-0001Kz-Q3 for submit@debbugs.gnu.org; Wed, 05 Jul 2023 15:08:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qH7r9-0004ja-D9 for bug-gnu-emacs@gnu.org; Wed, 05 Jul 2023 15:08:06 -0400 Received: from mail.ericabrahamsen.net ([52.70.2.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qH7r7-0008Un-7z for bug-gnu-emacs@gnu.org; Wed, 05 Jul 2023 15:08:03 -0400 Received: from localhost (c-71-197-232-41.hsd1.wa.comcast.net [71.197.232.41]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 5AA81FA002; Wed, 5 Jul 2023 19:07:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1688584076; bh=ENqgyvVhBpPhOE9v9JDu/sqV2lOXi7KKIOV4VIsWNXY=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=cOH3OiDU+b0q/sYLLSq48d9KPWkpth6KMiM/9lFMZMaCLy7fUajYd8sJ9I4lKrfld ed9gPHPNQdK3q/f4379ZpoOWyQXMmBtpWsyIFSGHbrnYsCcB5U4PTBpjbUepp0yNlf t/vwtOTF8jaG4Rg6AIS58Cnnl/NQkJPYGcB+0CdQ= From: Eric Abrahamsen To: Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87bkgq2nx3.fsf@dsemy.com> (Daniel Semyonov via's message of "Wed, 05 Jul 2023 21:50:16 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> <87r0prkgsz.fsf@dsemy.com> <873525udhn.fsf@ericabrahamsen.net> <87pm59984x.fsf@dsemy.com> <87a5wdskw8.fsf@ericabrahamsen.net> <87edlm5yc9.fsf@dsemy.com> <874jmie0qj.fsf@ericabrahamsen.net> <87bkgq2nx3.fsf@dsemy.com> Date: Wed, 05 Jul 2023 12:07:54 -0700 Message-ID: <87o7kqch2t.fsf@ericabrahamsen.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=52.70.2.18; envelope-from=eric@ericabrahamsen.net; helo=mail.ericabrahamsen.net X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit Cc: 64202@debbugs.gnu.org, Daniel Semyonov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" writes: >>>>>> Eric Abrahamsen writes: > > > Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army > > knife of text editors" writes: > > >> AFAIK it should work for any entry point which allows you to > >> define a new server or select method (Atom feeds are represented > >> as servers with a single group, so there is no way to create a > >> group without a pre-existing corresponding server). I'll be > >> honest though, I only ever use this method and > >> 'gnus-secondary-select-methods', so I'm not really sure which > >> other entry points there are. > > > What I usually start with is "G m", for > > `gnus-group-make-group'. That prompts for a group name, and also a > > backend. The viable list of backends includes all your > > currently-defined servers, plus the generic symbols for all > > defined backends, in this case a plain 'nnatom. > > > As you point out, that doesn't work for nnatom because each group > > will need its own server. What *does* work is to create a group > > this way, then put point on the new group and hit "M-e", for > > `gnus-group-edit-group-method', put in the actual URL, then "C-c > > C-c". > > > Obviously this is horribly baroque and it would be better if > > `gnus-group-make-group' could prompt for the URL. No matter what, > > it would be good to document the various ways to do this. > > Weirdly enough, 'gnus-group-make-group' accepts an ADDRESS argument, and > the docstring even claims it prompts for it, but the function fails if > you pass it an address it seems. > However, since Gnus knows which backends require an address, and adds > it to the name of the method when present, it shouldn't be hard to fix > this command to actually prompt for an address when needed. > I haven't looked deeply enough into how that function works to know > whether it would actually work as expected in that case though. Where do you see that Gnus knows which backends require an address? That would be very handy. But my reading of this function is that we don't even known that the backend is loaded until the end of the group creation process. You could probably mildly abuse the 'request-create-group server function to further prompt the user for the URL. It is a bit annoying that the ADDRESS argument is in there, but not prompted for. It looks like that's been the case for at least two decades, though. > >> > Regarding your earlier question about having this backend > >> handle > RSS too, I'm not aware of any significant difference > >> between the > two beyond the format of the XML. Is that true? > >> > >> Yes, even the XML format is very similar. > >> > >> > If so, it seems like it would make most sense to merge the > > >> code. Have you looked at nnrss? It would be good to know if there > >> > was anything in there worth stealing for nnatom -- if one of > >> them > has a faster parser than the other, for instance, or > >> better logic > for keeping updates efficient. > >> > >> The issue with merging the two is that nnrss saves feed data > >> differently (both on disk and in memory), and also represents > >> each feed as a group, with a virtual "server" holding all groups. > >> I'm not sure if it's possible/a good idea to migrate feed data > >> from nnrss to a hypothetical merged backend, at least not > >> automatically. > >> > >> Stefan floated the idea of adding RSS support, deprecating nnrss > >> and creating an interactive migration command - so users who wish > >> to migrate will have to do so manually (which should also > >> potentially allow asking the user some questions if the migration > >> includes some non-trivial steps). > >> > >> nnrss does do some cool stuff that nnatom doesn't, though (for > >> example, it tries very hard to find an RSS feed when you provide > >> it with a link to a website, while nnatom currently requires a > >> direct link to a feed). > > > Right, I should have been more explicit here -- what I was > > thinking is what Stefan suggests: just deprecate nnrss > > altogether. If it does anything cool like feed discovery, just > > steal that code. If you're inclined to be nice enough to provide a > > migration function, that's a bonus. > > I guess I'll see how hard it would be. First I have to actually > implement RSS support though. I also don't think it would hurt to start with what we've got, and incrementally add to it after an initial deployment. The important thing would be factoring out a nnfeed.el backend, then providing nnatom.el on top of that. The rest could come as you get to it. > >> > I just subscribed to a feed with nnrss, and noticed that after > >> I > marked all the items in the feed as read, I couldn't re-enter > >> the > group and see the old items. It gave me "Can't select > >> group". So > that's not very encouraging. > >> > >> Honestly, from my experience nnrss has many small issues > >> (although I never encountered this exact issue). It is partly > >> why I developed nnatom (previously I used a hack documented on > >> the Emacs wiki which converted Atom feeds to RSS feeds on the > >> fly). > >> > >> > If you do want to expand this to be a general "feed" backend, > >> we > might want to do some boring things like rename it > >> nnfeed.el, > >> > >> This is the name I thought of too, and I guess if two people > >> thought of it independently it's probably fine. > > > Good! > > >> > and add support for ridiculous things like JSON feed[0] > >> (why?!?). I > assume a derived backend could handle JSON feeds by > >> setting the > appropriate values for the `nnatom-read-*-function' > >> deffoos? > >> > >> I actually attached a derivative "nnjsonfeed" backend I made as > >> an experiment to one of my previous messages (it doesn't work > >> with the current version of the patch, but it won't be hard to > >> fix). > >> > >> It wasn't 100% conforming to the standard (JSON feeds support > >> some weird features like pagination, which can actually be > >> supported very well in theory by nnatom, but I didn't feel like > >> doing that), but it worked and it was very easy to make. > > > I missed nnjsonfeed, sorry. That's great you're already working in > > this direction. I don't think we need to support absolutely > > everything it does (what would pagination look like in Gnus?), > > just the basics to get started with. > > The feed itself is paginated, meaning that the entries are spread across > multiple files, with each one pointing to the next with a special tag. > The design of nnatom allows for this since it doesn't necessarily > require parsing the entire feed ahead of time, and also notifies the > backend when a stale (not new or updated) article is reached, so the > backend can intelligently stop (instead of trying to parse thousands of > pages every time you update a group, for example). These capabilities > were actually added in anticipation of a JSON feed backend, the Atom > parser doesn't use them at all. > > Gnus itself won't show pages of course, just a list of articles as > normal, but they will come from different JSON feed pages. I see! Sounds like a reasonable optimization for making updates faster. > >> > One of the awkward things about nnrss is that it's never really > >> > fit well into Gnus' one-server-many-groups paradigm, which you > >> > allude to in the nnatom Info section. Do you have any further > > >> ideas in that direction? > >> > >> Well, nnatom theoretically supports this paradigm, but it doesn't > >> do this with Atom feeds, since it doesn't really make sense IMO. > >> However, there is a standardized way to include links to Atom > >> feeds in HTML documents, so it might be a good idea to support > >> adding them as servers which show any linked Atom feeds as > >> groups. > > > It's also perfectly possible that a single website would publish > > multiple Atom feeds, right? Like weather.gov, for instance. That > > would have an added benefit of letting us simplify the server name > > from the full URL (https://alerts.weather.gov/cap/wa.php?x=0) > > which looks ugly in the Group buffer, to just eg > > alerts.weather.gov. > > Yes, that's actually what I meant, although looking at real world > examples, it seems most websites either only link to one Atom feed in > the "correct" way, or don't link to any at all in that way. > > doesn't link to any feed in the > standard way at all, for example. > > > But I guess I don't know why it would matter, really. The only > > practical use for wanting multiple feeds under a single server > > would be to set some configuration at the server level, that would > > apply to all feeds from that server. And at the moment I don't > > think there are many such settings (right?). > > Actually, since the various parsing functions are stored in server > variables, all of them can be changed on a per-server basis, which is > pretty cool. > > As an example, after I add support for fetching feeds contained in HTML, > I can imagine a scenario where a user adds a website which publishes > multiple, non-standard feeds, and fixes the parsing of all of them at > once by providing some customized parsing functions for that specific > server. > This is a contrived example, but I know at least of youtube, which > publishes slightly customized Atom feeds for each channel, so this is > already useful. Thanks for the background. So it does sound like one Gnus server per remote server is the way to go. From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 14 21:02:44 2023 Received: (at 64202) by debbugs.gnu.org; 15 Jul 2023 01:02:44 +0000 Received: from localhost ([127.0.0.1]:43682 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qKTgI-00041q-Ee for submit@debbugs.gnu.org; Fri, 14 Jul 2023 21:02:44 -0400 Received: from dsemy.com ([46.23.89.208]:19663) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qKTgE-00041Q-JO for 64202@debbugs.gnu.org; Fri, 14 Jul 2023 21:02:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=vQ0apsl+izxvE 6b6WNPPIoq+SbVd/l27m2VIuFnuh9Y=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=IM7O/E9CA1/umY9vRd6QGx6/417LUJhm5RV ylYN5qCRsIRB7CeWXy1meA4HLFobH+PUhbjwjlxGG3miZaXUZU+DBBuQd++8a3t6rX/Rjr NeABxnULuSSZR1WP6v1GYlhBdiu0AuBvK4Ncm+RcUaObOoXYxjKNEc0ZbZg5uVQxDsAFDx dNwVaeuguWlkGuH4CLVOC9l9qX4AHqlU/yYnzf5IzvuzCiZTafNnQI7CN41JOvplWw9nBf Nigrl3i8HYTvEWdqqskZ8+oC7M7DjPHHXlMOVmykU6u0r/O788T8Ex/qXdpFKSSfauIKvo jEEGnspVN6hHGEoulEnxi7VcdWw== Received: from coldharbour.home.arpa ( [147.235.213.12]) by dsemy.com (OpenSMTPD) with ESMTPSA id 2d9abc6e (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sat, 15 Jul 2023 03:02:31 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id f4fa08bd; Sat, 15 Jul 2023 00:59:49 +0000 (UTC) From: Daniel Semyonov To: Eric Abrahamsen Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87o7kqch2t.fsf@ericabrahamsen.net> (Eric Abrahamsen's message of "Wed, 05 Jul 2023 12:07:54 -0700") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> <87r0prkgsz.fsf@dsemy.com> <873525udhn.fsf@ericabrahamsen.net> <87pm59984x.fsf@dsemy.com> <87a5wdskw8.fsf@ericabrahamsen.net> <87edlm5yc9.fsf@dsemy.com> <874jmie0qj.fsf@ericabrahamsen.net> <87bkgq2nx3.fsf@dsemy.com> <87o7kqch2t.fsf@ericabrahamsen.net> Date: Sat, 15 Jul 2023 03:59:49 +0300 Message-ID: <878rbikn0a.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain >>>>> Eric Abrahamsen writes: > Where do you see that Gnus knows which backends require an > address? That would be very handy. Forgot to answer this -- 'gnus-valid-select-methods' holds this information (and a bunch of other info too). > But my reading of this function is that we don't even known that > the backend is loaded until the end of the group creation > process. You could probably mildly abuse the 'request-create-group > server function to further prompt the user for the URL. > It is a bit annoying that the ADDRESS argument is in there, but > not prompted for. It looks like that's been the case for at least > two decades, though. I'll add this to the pile of stuff I wanna change in Gnus once I'm satisfied with this backend (quite a large pile at this point :). In any case, I still didn't have much time to work on the backend unfortunately, but I attached a patch which incorporates the fixes from my last message + changes based on comments from Stefan. Things left to do: 1. Ensure support for none-libxml builds (I should probably figure out how ERT works to make this a non-issue going forward). 2. Factor out nnfeed.el and base nnatom on top of it (shouldn't be hard). 3. Add support for RSS (and steal cool stuff from nnrss). 4. Support Atom feeds linked in HTML documents. Regards, Daniel --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch Content-Description: patch >From 0ddc9702d2205dc8323a9f172e7378c02946b157 Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnatom.el: New file. * doc/misc/gnus.texi: * etc/NEWS: Document nnatom --- doc/misc/gnus.texi | 58 ++++ etc/NEWS | 7 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 796 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 862 insertions(+) create mode 100644 lisp/gnus/nnatom.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 8d25e868c8a..9184a4035df 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -17250,6 +17251,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17494,6 +17496,62 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@pxref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds +as Gnus servers (with a single group), as you would with any other +method, by supplying the location of the feed as the server address. +Note, however, that the server address shouldn't be prefixed with +@url{http://} or @url{https://}. + +Features of @code{nnatom} include: + +@itemize @bullet + +@item +Server data is saved per-server in the @file{atom} sub-directory of +@file{gnus-directory}. + +@item +An article part is generated for both the summary and the content for +each entry in the feed. Content of all MIME types should be displayed +correctly through Gnus (as long as the feed specifies a MIME type), +even base64 encoded files (like images for example), which are +supported by Atom. + +@item +Article modification and publish dates are tracked, and articles are +updated if changed. + +@item +Designed with modularity in mind, the back end theoretically supports +features which aren't available in Atom (and thus aren't actually +implemented in practice). For example, the back end operates under +the assumption that ``servers'' (Atom feeds) can have multiple groups, +when in reality the parsing functions will only ever return servers +with a single group. In addition, all parsing steps and some of the +printing is done in functions stored in server variables, to allow +very easily defining new back ends for different kinds of feeds by +inheriting from @code{nnatom}; these are called +@code{nnatom-read-*-function} and +@code{nnatom-print-content-function}, and their requirements are +detailed in their docstrings. A macro is provided to ease creation of +new inheriting backends. + +@defmac nnatom-define-basic-backend-interface backend +Define server variables expected by @code{nnatom} and import its back +end functions for @var{backend}. This macro (or code similar to it) should +be used by any inheriting back end. +@end defmac + +@end itemize + @node Other Sources @section Other Sources diff --git a/etc/NEWS b/etc/NEWS index d7f5fdc4cbb..16fc419834c 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -386,6 +386,13 @@ distracting and easily confused with actual code, or a significant early aid that relieves you from moving the buffer or reaching for the mouse to consult an error message. +** Gnus + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + ** Python Mode --- diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index fc8518512ee..d35d709c448 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1343,6 +1343,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..6ea47c317c0 --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,796 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format . +;; Other types of feeds may be supported by providing custom parsing +;; functions. + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'subr-x)) + +(require 'gnus) +(require 'nnheader) +(require 'nnoo) +(require 'gnus-group) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'gnus) + +(nnoo-declare nnatom) + +(defvoo nnatom-backend 'nnatom + "Symbol which identifies this backend.") + +(defvoo nnatom-status-string nil + "Last status message reported by this backend.") + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "\\`https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-name-absolute-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (when-let ((data (if (libxml-available-p) + (libxml-parse-xml-region + (point-min) (point-max)) + (car (xml-parse-region + (point-min) (point-max))))) + (auth (list 'authors))) + (when (eq (car data) 'top) + (setq data (assq 'feed data))) + (dom-add-child-before data auth) + (catch :stop ; Collect feed authors, stop at first entry. + (dolist (child (cdddr data)) + (let ((tag (car-safe child))) + (if (eq tag 'entry) + (throw :stop data) + (and (or (eq tag 'author) + (eq tag 'contributor)) + (dom-add-child-before auth child))))) + data))))))) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) (setq data (cddr data))) + (while (and data (not (eq (car-safe (car data)) 'entry))) (pop data)) + (when-let ((article (car data)) + (auths (list 'authors)) (links (list 'links))) + (dom-add-child-before article links) + (dom-add-child-before article auths) + (dolist (child (cddddr article) `(,article . ,(cdr data))) + (pcase (car-safe child) ; Authors and links can appear + ((or 'author 'contributor) ; anywhere in the entry so we + (dom-add-child-before auths child) ; collect them all here to + (dom-add-child-before links child)) ; avoid looping over the + ((or 'link ; entry multiple times later. + (and 'content (guard (assq 'src (dom-attributes child))))) + (dom-add-child-before links child)))))) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) + +(defun nnatom--read-article-or-group-authors (article-or-group) + "Return the authors of ARTICLE-OR-GROUP, or nil." + (when-let + ((a (string-trim-right + (mapconcat (lambda (author) + (let* ((name + (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email + (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (concat + (or (and name email (format "%s <%s>" name email)) + name email) + ", "))) + (dom-children + (dom-child-by-tag article-or-group 'authors))) + ", ")) + ((not (string-blank-p a)))) + a)) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((alt 0) (rel 0) (sel 0) (enc 0) (via 0) (aut 0)) + (mapcan + (lambda (link) + (when-let ((l (car-safe link))) + (or + (when-let (((eq l 'content)) + (src (dom-attr link 'src)) + (label (concat "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt))))) + `(((("text/plain") . ,(format "%s: %s\n" label src)) + (("text/html") . ,(format "[%s] " + src label))))) + (when-let (((or (eq l 'author) (eq l 'contributor))) + (name (dom-text (dom-child-by-tag link 'name))) + (name (if (string-blank-p name) + (concat "Author" + (and (< 1 (cl-incf aut)) + (format " %s" aut))) + name)) + (uri (dom-text (dom-child-by-tag link 'uri))) + ((not (string-blank-p uri)))) + `(((("text/plain") . ,(format "%s: %s\n" name uri)) + (("text/html") . ,(format "[%s] " + uri name))))) + (when-let (((eq l 'link)) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf rel)) + (format " %s" rel)))) + ("self" + (concat "More" + (and (< 1 (cl-incf sel)) + (format " %s" sel)))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf enc)) + (format " %s" enc)))) + ("via" + (concat "Source" + (and (< 1 (cl-incf via)) + (format " %s" via)))) + (_ (if-let + ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat + "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt)))))))) + (link (cdr (assq 'href attrs)))) + `(((("text/plain") . ,(format "%s: %s\n" label link)) + (("text/html") . ,(format "[%s] " + link label)))))))) + (dom-children (dom-child-by-tag article 'links))))) + +(defun nnatom--read-part (part type) + (let* ((atypes '("html" "plain")) + (mtypes '(("xhtml" . "text/html") ("text" . "text/plain"))) + (xsuff (concat "[+/]xml\\(-\\(dtd\\|external-parsed-entity\\)\\)?\\'" + "\\|^text")) + (part (if (string= type "xhtml") + (with-temp-buffer + (dom-print (dom-child-by-tag part 'div) nil t) + (buffer-substring-no-properties + (point-min) (point-max))) + (dom-text part))) + (type (if (member type atypes) (concat "text/" type) type)) + (type (or (cdr (assoc type mtypes)) type))) + (unless (string-blank-p part) + `(,part (Content-Type . ,(or type (setq type "text/plain"))) + ,(and (not (string-match-p xsuff type)) + '(Content-Transfer-Encoding . "base64")))))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (cdr (assq 'type (dom-attributes summary)))) + (summary (nnatom--read-part summary stype)) + (content (dom-child-by-tag article 'content)) + (ctype (cdr (assq 'type (dom-attributes content)))) + (content (nnatom--read-part content ctype)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push summary parts)) + (push (append content '(links)) parts) + (or st (push summary parts)) + parts) + ((setq parts (or summary content)) + `(,(append (or summary content) '(links)))) + (t '((nil (Content-Type . "text/html") links)))))) + +;;;; Feed I/O: + +(defvoo nnatom-read-feed-function #'nnatom--read-feed + "Function returning a Lisp object representing a feed (or part of it). +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnatom-read-group-function').") + +(defvoo nnatom-read-group-function #'nnatom--read-group + "Function returning a cons cell of a group and remaining data from a feed.") + +(defvoo nnatom-read-article-function #'nnatom--read-article + "Function returning a cons cell of an article and remaining data from a group. +It should accept a two arguments, a Lisp object representing a feed, +and a flag indicating whether the last article was stale (not new or updated).") + +(defvoo nnatom-read-title-function #'nnatom--read-title + "Function returning the title of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-description-function #'nnatom--read-description + "Function returning the description of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-group-author-function + #'nnatom--read-article-or-group-authors + "Function returning the author of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-id-function #'nnatom--read-id + "Function returning the ID of an article. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-subject-function #'nnatom--read-subject + "Function returning the subject of an article (a string). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + "Function returning the publish date of an article (a time value). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-update-date-function #'nnatom--read-update + "Function returning the update date of an article (a time value). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-authors + "Function returning the author of an article (a string). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-headers-function #'ignore + "Function returning an alist of article-wide MIME headers. +Each element of this alist maps a MIME header (a symbol, +i.e. `Content-Type') to its value. As a special case, `:boundary' +maps to a string which will serve as the boundary between article +parts. This must be supplied if a custom boundary is used in a +multipart content type header. The default boundary is \"-_nnatom_-\", +and is automatically modified to match the name of the back end. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-links-function #'nnatom--read-links + "Function returning all links contained in an article. +For the default `nnatom-print-content-function', it should return a +list of links, where each link is an alist mapping MIME content types +to links formatted for display in a part of that type. Each content +type may also be a list of content types. +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-parts-function #'nnatom--read-parts + "Function returning an alist associating parts of an article to their headers. +For the default `nnatom-print-content-function', each part should be a +string. Otherwise, it can be any Lisp object. The \"headers\" of +each part should be a list where each element is either a cons of a +MIME header (a symbol, i.e. `Content-Type') and its value (a string), +or any other Lisp object. MIME headers will be printed, the rest will +be passed on to `nnatom-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnatom-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnatom-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnatom--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnatom-backend) + "nn") + (gnus-newsgroup-savable-name server)) + gnus-directory)) + +(defun nnatom--read-server (server) + "Read SERVER's information from `nnatom-directory'." + (if-let ((f (nnatom--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnatom-servers)) + (nnheader-report nnatom-backend "Can't read %s" server))) + +(defun nnatom--write-server (server) + "Write SERVER's information to `nnatom-directory'." + (if-let ((f (nnatom--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnatom-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n" + (prin1 s (current-buffer)) "\n") + t) + t) + (nnheader-report nnatom-backend "Can't write %s" f))) + +(defun nnatom--server-address (server) + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnatom-backend 'address) + (cddr (gnus-server-to-method + (concat + (symbol-name nnatom-backend) ":" + server))))) + server)) + server)) + +(defun nnatom--parse-feed (feed &optional group) + "Parse FEED into a new or existing server. +Optionally, only parse GROUP." + (let* ((feed (nnatom--server-address feed)) + (s (or (gethash feed nnatom-servers) (nnatom--read-server feed))) + (g (or (and (hash-table-p s) (gethash group s)) `[,group])) + (name group) ; (Maybe) fake name + (group (or (aref g 0) name)) ; Real name + data) + (when (setq data (funcall nnatom-read-feed-function feed group) + s (or (gethash feed nnatom-servers) + (make-hash-table :test 'equal))) + (while-let ((cg (or (and name `(,data)) + (funcall nnatom-read-group-function data))) + (cg (prog1 (car cg) (setq data (cdr cg))))) + (let* ((name (funcall nnatom-read-title-function cg)) ; Real name + (group (gethash name nnatom-group-names name)) ; (Maybe) fake name + (info (gnus-get-info + (concat (symbol-name nnatom-backend) "+" feed ":" group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnatom-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnatom-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnatom-read-article-function cg stale)) + (article (prog1 (car article) (setq cg (cdr article))))) + (when-let ((id (funcall nnatom-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnatom-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnatom-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (cl-incf max)))) + (publish (funcall nnatom-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnatom-read-author-function article) + group-author group) + ,(or (funcall nnatom-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnatom-read-links-function article) + ,(funcall nnatom-read-parts-function article) + ,(funcall nnatom-read-headers-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnatom-servers)))) + +;;;; Gnus backend functions: + +(defvoo nnatom-group nil + "Name of the current group.") + +(defvoo nnatom-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnatom-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS HEADERS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers. +- HEADERS is an alist associating article-wide MIME headers to their value.") + +(defvoo nnatom-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnatom-group-article-min-num 1 + "Minimum article number for the current group.") + +(defvar nnatom-date-format "%F %X" + "Format of displayed dates.") + +(nnoo-define-basics nnatom) + +(defun nnatom--current-server-no-prefix () + (string-remove-prefix (concat (symbol-name nnatom-backend) "+") + (nnoo-current-server nnatom-backend))) + +(defun nnatom--group-data (group &optional server) + (let ((s (gethash server nnatom-servers)) c) + (or (and (hash-table-p s) (gethash group s)) + (and (setq c (nnatom--current-server-no-prefix)) + (setq s (gethash c nnatom-servers)) + (hash-table-p s) (gethash group s)) + (catch :stop (maphash (lambda (n s) + (or (string= n server) + (string= n c) + (when-let (((hash-table-p s)) + (g (gethash group s))) + (throw :stop g)))) + nnatom-servers))))) + +(defun nnatom-retrieve-article (article group) + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string nnatom-date-format (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnatom-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnatom--current-server-no-prefix))) + (g (or (nnatom--group-data group server) + `[ nil ,nnatom-group-article-ids ,nnatom-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnatom-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnatom-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnatom-backend "Group %s not found" (or group "")))) + +(deffoo nnatom-open-server (server &optional defs backend) + (let ((backend (or backend 'nnatom)) + (a (nnatom--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (or (gethash server nnatom-servers) + (nnatom--read-server a))) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash group g nnatom-group-names))) + s)) + (or s (file-writable-p (nnatom--server-file a)) + (and (nnoo-close-server nnatom-backend server) + (nnheader-report + nnatom-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnatom-request-close () + (maphash (lambda (server _) + (nnatom--write-server + (nnatom--server-address server))) + nnatom-servers) + (setq nnatom-servers (make-hash-table :test 'equal) + nnatom-status-string nil) + t) + +(defun nnatom--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, possibly with LINKS added." + (let ((links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and content (format "%s\n\n" content)) + (mapconcat (lambda (link) + (cdr (assoc (cdr (assq 'Content-Type attributes)) link + (lambda (types type) + (if (stringp types) (string= types type) + (member type types)))))) + links))))) + +(defvoo nnatom-print-content-function #'nnatom--print-content + "Function returning a single piece of content (a string). +It should accept three arguments, a part and its attributes (as returned +by `nnatom-read-parts-function'), and links.") + +(defun nnatom--print-part (content headers mime links) + (concat + (mapconcat (lambda (header) + (when-let ((m (car-safe header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnatom-print-content-function content headers links))) + +(deffoo nnatom-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnatom--current-server-no-prefix))) + (g (or (nnatom--group-data group server) + (and (setq group nnatom-group) + `[ nil ,nnatom-group-article-ids + ,nnatom-group-articles + ,nnatom-group-article-max-num + ,nnatom-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((links (aref a 5)) + (parts (aref a 6)) + (headers (aref a 7)) + (boundary (or (cdr (assq :boundary headers)) + (format "-_%s_-" nnatom-backend))) + (multi (length> parts 1)) + (mime '( Content-Type Content-Disposition + Content-Transfer-Encoding))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnatom-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + (if (assq 'MIME-Version headers) "" "MIME-Version: 1.0\n") + (mapconcat (lambda (header) + (unless (keywordp (car header)) + (format "%s: %s\n" (car header) (cdr header)))) + headers) + (if multi + (if (assq 'Content-Type headers) "" + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary)) + (prog1 (nnatom--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (format "--%s\n%s\n" boundary + (nnatom--print-part + (car part) (cdr part) mime links))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnatom-backend "No such article"))) + +(deffoo nnatom-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnatom--current-server-no-prefix))) + (g (or (if fast (nnatom--group-data group server) + (setq server (nnatom--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnatom-group group + nnatom-group-article-ids (aref g 1) + nnatom-group-articles (aref g 2) + nnatom-group-article-max-num (aref g 3) + nnatom-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnatom-group-article-ids) + nnatom-group-article-min-num + nnatom-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnatom-backend "Group %s not found" group)))) + +(deffoo nnatom-close-group (group &optional server) + (and (string= group nnatom-group) + (setq nnatom-group nil + nnatom-group-article-ids (make-hash-table :test 'equal) + nnatom-group-articles (make-hash-table :test 'eql) + nnatom-group-article-max-num 0 + nnatom-group-article-min-num 1)) + (setq server (or server (nnatom--current-server-no-prefix))) + (nnatom--write-server server)) + +(deffoo nnatom-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnatom--parse-feed + (or server (nnatom--current-server-no-prefix)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnatom-request-post (&optional _server) + (nnheader-report nnatom-backend "%s is a read only backend" nnatom-backend)) + +;;;; Optional back end functions: + +(deffoo nnatom-retrieve-groups (_groups &optional server) + (nnatom-request-list (or server (nnatom--current-server-no-prefix))) + 'active) + +(deffoo nnatom-request-type (_group &optional _article) + 'unknown) + +(deffoo nnatom-request-group-description (group &optional server) + (when-let ((server (or server (nnatom--current-server-no-prefix))) + (g (nnatom--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnatom-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnatom--current-server-no-prefix))) + (s (gethash server nnatom-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnatom-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnatom--current-server-no-prefix))) + (s (or (gethash server nnatom-servers) + (and ; Open the server to add it to `nnatom-servers' + (save-match-data + (nnatom-open-server + server + (cdr ; Get defs and backend. + (assoc server (cdr (assq 'nnatom nnoo-state-alist)) + (lambda (car key) + (and (stringp car) + (string-match + (concat + "\\(\\(nn[[:alpha:]]+\\)\\+\\)?" + "\\(" (regexp-quote key) "\\)") + car) + (setq server car))))) + (prog1 (if (stringp (match-string 1 server)) + (intern (match-string 2 server)) + 'nnatom) + (setq server (match-string 3 server)))) + (gethash server nnatom-servers))))) + (g (or (nnatom--group-data group server) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnatom-group-names) + (remhash group s) + (and (string= group nnatom-group) + (setq nnatom-group new-name)) + t)) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +;;;; Utilities: + +(defmacro nnatom-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnatom-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnatom-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnatom-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnatom-servers) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnatom-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnatom-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnatom-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnatom-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnatom-open-server server defs ',backend)) + (nnoo-import ,backend (nnatom)))) + +(provide 'nnatom) + +;;; nnatom.el ends here -- 2.41.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 14 21:29:23 2023 Received: (at 64202) by debbugs.gnu.org; 15 Jul 2023 01:29:24 +0000 Received: from localhost ([127.0.0.1]:43699 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qKU66-0004zJ-6O for submit@debbugs.gnu.org; Fri, 14 Jul 2023 21:29:23 -0400 Received: from dsemy.com ([46.23.89.208]:10063) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qKU62-0004yv-Ni for 64202@debbugs.gnu.org; Fri, 14 Jul 2023 21:29:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=tcxx6dbDsB21H 0vv4ANIZyhd1C31Cp0wb1wJ96iY8Vs=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=OjVBOTKpAIrVAznyto4Tf3wQV5ZyefaJobO 0DYsYZ3TM3xMHKQMokBdNUIvARTjcpCp9yTNFCJ5ES3Fpnu8sZABT2MGUPUIDf2Uw6rryG MNU94Mqqh76K0SdWvOQcouhBNn/kdpGmLnqNFVJRxXyNX9UgVebOLJH3aMArFThOYJx0e9 QhOTLb/Q4yz3vfDS1eWNvxAh2GsmIF1nJZhQN019Ca9GiiKxTy5YdN8NGVRPc0KiENtrUt SjCd730V1J8dI13Xxyj8cI8FnU40ePdqKiZdgFhDQJopom+vCH+SJbOLnlIfcUZkZVmbQ9 x/dCjmuaAW14z8NONZ2dTlSFekQ== Received: from coldharbour.home.arpa ( [147.235.213.12]) by dsemy.com (OpenSMTPD) with ESMTPSA id 9168913a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sat, 15 Jul 2023 03:29:12 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id 1a67b94b; Sat, 15 Jul 2023 01:26:31 +0000 (UTC) From: Daniel Semyonov To: Eric Abrahamsen Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <878rbikn0a.fsf@dsemy.com> (Daniel Semyonov's message of "Sat, 15 Jul 2023 03:59:49 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> <87r0prkgsz.fsf@dsemy.com> <873525udhn.fsf@ericabrahamsen.net> <87pm59984x.fsf@dsemy.com> <87a5wdskw8.fsf@ericabrahamsen.net> <87edlm5yc9.fsf@dsemy.com> <874jmie0qj.fsf@ericabrahamsen.net> <87bkgq2nx3.fsf@dsemy.com> <87o7kqch2t.fsf@ericabrahamsen.net> <878rbikn0a.fsf@dsemy.com> Date: Sat, 15 Jul 2023 04:26:31 +0300 Message-ID: <87bkgehsmw.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain >>>>> Daniel Semyonov writes: > I attached a patch which incorporates the fixes > from my last message + changes based on comments from Stefan. I sent a slightly out of date version of the patch with my last message. Attached is the current version. --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch Content-Description: patch >From 63df5f6f046b3aa1dacfeccd4649cf4af381c0d0 Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnatom.el: New file. * doc/misc/gnus.texi: * etc/NEWS: Document nnatom --- doc/misc/gnus.texi | 58 ++++ etc/NEWS | 7 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 797 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 863 insertions(+) create mode 100644 lisp/gnus/nnatom.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 8d25e868c8a..9184a4035df 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -17250,6 +17251,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17494,6 +17496,62 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@pxref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds +as Gnus servers (with a single group), as you would with any other +method, by supplying the location of the feed as the server address. +Note, however, that the server address shouldn't be prefixed with +@url{http://} or @url{https://}. + +Features of @code{nnatom} include: + +@itemize @bullet + +@item +Server data is saved per-server in the @file{atom} sub-directory of +@file{gnus-directory}. + +@item +An article part is generated for both the summary and the content for +each entry in the feed. Content of all MIME types should be displayed +correctly through Gnus (as long as the feed specifies a MIME type), +even base64 encoded files (like images for example), which are +supported by Atom. + +@item +Article modification and publish dates are tracked, and articles are +updated if changed. + +@item +Designed with modularity in mind, the back end theoretically supports +features which aren't available in Atom (and thus aren't actually +implemented in practice). For example, the back end operates under +the assumption that ``servers'' (Atom feeds) can have multiple groups, +when in reality the parsing functions will only ever return servers +with a single group. In addition, all parsing steps and some of the +printing is done in functions stored in server variables, to allow +very easily defining new back ends for different kinds of feeds by +inheriting from @code{nnatom}; these are called +@code{nnatom-read-*-function} and +@code{nnatom-print-content-function}, and their requirements are +detailed in their docstrings. A macro is provided to ease creation of +new inheriting backends. + +@defmac nnatom-define-basic-backend-interface backend +Define server variables expected by @code{nnatom} and import its back +end functions for @var{backend}. This macro (or code similar to it) should +be used by any inheriting back end. +@end defmac + +@end itemize + @node Other Sources @section Other Sources diff --git a/etc/NEWS b/etc/NEWS index d7f5fdc4cbb..16fc419834c 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -386,6 +386,13 @@ distracting and easily confused with actual code, or a significant early aid that relieves you from moving the buffer or reaching for the mouse to consult an error message. +** Gnus + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + ** Python Mode --- diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index fc8518512ee..d35d709c448 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1343,6 +1343,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..e58f48bcf0e --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,797 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format . +;; Other types of feeds may be supported by providing custom parsing +;; functions. + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'subr-x)) + +(require 'gnus) +(require 'nnheader) +(require 'nnoo) +(require 'gnus-group) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'gnus) + +(nnoo-declare nnatom) + +(defvoo nnatom-backend 'nnatom + "Symbol which identifies this backend.") + +(defvoo nnatom-status-string nil + "Last status message reported by this backend.") + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "\\`https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-name-absolute-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (when-let ((data (if (libxml-available-p) + (libxml-parse-xml-region + (point-min) (point-max)) + (car (xml-parse-region + (point-min) (point-max))))) + (auth (list 'authors))) + (when (eq (car data) 'top) + (setq data (assq 'feed data))) + (dom-add-child-before data auth) + (catch :stop ; Collect feed authors, stop at first entry. + (dolist (child (cdddr data)) + (let ((tag (car-safe child))) + (if (eq tag 'entry) + (throw :stop data) + (and (or (eq tag 'author) + (eq tag 'contributor)) + (dom-add-child-before auth child))))) + data))))))) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) (setq data (cddr data))) + (while (and data (not (eq (car-safe (car data)) 'entry))) (pop data)) + (when-let ((article (car data)) + (auths (list 'authors)) (links (list 'links))) + (dom-add-child-before article links) + (dom-add-child-before article auths) + (dolist (child (cddddr article) `(,article . ,(cdr data))) + (pcase (car-safe child) ; Authors and links can appear + ((or 'author 'contributor) ; anywhere in the entry so we + (dom-add-child-before auths child) ; collect them all here to + (dom-add-child-before links child)) ; avoid looping over the + ((or 'link ; entry multiple times later. + (and 'content (guard (assq 'src (dom-attributes child))))) + (dom-add-child-before links child)))))) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) + +(defun nnatom--read-article-or-group-authors (article-or-group) + "Return the authors of ARTICLE-OR-GROUP, or nil." + (when-let + ((a (string-trim-right + (mapconcat (lambda (author) + (let* ((name + (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email + (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (concat + (or (and name email (format "%s <%s>" name email)) + name email) + ", "))) + (dom-children + (dom-child-by-tag article-or-group 'authors))) + ", ")) + ((not (string-blank-p a)))) + a)) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((alt 0) (rel 0) (sel 0) (enc 0) (via 0) (aut 0)) + (mapcan + (lambda (link) + (when-let ((l (car-safe link))) + (or + (when-let (((eq l 'content)) + (src (dom-attr link 'src)) + (label (concat "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt))))) + `(((("text/plain") . ,(format "%s: %s\n" label src)) + (("text/html") . ,(format "[%s] " + src label))))) + (when-let (((or (eq l 'author) (eq l 'contributor))) + (name (dom-text (dom-child-by-tag link 'name))) + (name (if (string-blank-p name) + (concat "Author" + (and (< 1 (cl-incf aut)) + (format " %s" aut))) + name)) + (uri (dom-text (dom-child-by-tag link 'uri))) + ((not (string-blank-p uri)))) + `(((("text/plain") . ,(format "%s: %s\n" name uri)) + (("text/html") . ,(format "[%s] " + uri name))))) + (when-let (((eq l 'link)) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf rel)) + (format " %s" rel)))) + ("self" + (concat "More" + (and (< 1 (cl-incf sel)) + (format " %s" sel)))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf enc)) + (format " %s" enc)))) + ("via" + (concat "Source" + (and (< 1 (cl-incf via)) + (format " %s" via)))) + (_ (if-let + ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat + "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt)))))))) + (link (cdr (assq 'href attrs)))) + `(((("text/plain") . ,(format "%s: %s\n" label link)) + (("text/html") . ,(format "[%s] " + link label)))))))) + (dom-children (dom-child-by-tag article 'links))))) + +(defun nnatom--read-part (part type) + (let* ((atypes '("html" "plain")) + (mtypes '(("xhtml" . "text/html") ("text" . "text/plain"))) + (xsuff (concat "[+/]xml\\(-\\(dtd\\|external-parsed-entity\\)\\)?\\'" + "\\|^text")) + (part (if (string= type "xhtml") + (with-temp-buffer + (dom-print (dom-child-by-tag part 'div) nil t) + (buffer-substring-no-properties + (point-min) (point-max))) + (dom-text part))) + (type (if (member type atypes) (concat "text/" type) type)) + (type (or (cdr (assoc type mtypes)) type))) + (unless (string-blank-p part) + `(,part (Content-Type . ,(or type (setq type "text/plain"))) + ,(and (not (string-match-p xsuff type)) + '(Content-Transfer-Encoding . "base64")))))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (cdr (assq 'type (dom-attributes summary)))) + (summary (nnatom--read-part summary stype)) + (content (dom-child-by-tag article 'content)) + (ctype (cdr (assq 'type (dom-attributes content)))) + (content (nnatom--read-part content ctype)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push summary parts)) + (push (append content '(links)) parts) + (or st (push summary parts)) + parts) + ((setq parts (or summary content)) + `(,(append (or summary content) '(links)))) + (t '((nil (Content-Type . "text/html") links)))))) + +;;;; Feed I/O: + +(defvoo nnatom-read-feed-function #'nnatom--read-feed + "Function returning a Lisp object representing a feed (or part of it). +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnatom-read-group-function').") + +(defvoo nnatom-read-group-function #'nnatom--read-group + "Function returning a cons cell of a group and remaining data from a feed.") + +(defvoo nnatom-read-article-function #'nnatom--read-article + "Function returning a cons cell of an article and remaining data from a group. +It should accept a two arguments, a Lisp object representing a feed, +and a flag indicating whether the last article was stale (not new or updated).") + +(defvoo nnatom-read-title-function #'nnatom--read-title + "Function returning the title of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-description-function #'nnatom--read-description + "Function returning the description of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-group-author-function + #'nnatom--read-article-or-group-authors + "Function returning the author of a group (a string). +It should accept a single argument, a Lisp object representing a group.") + +(defvoo nnatom-read-id-function #'nnatom--read-id + "Function returning the ID of an article. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-subject-function #'nnatom--read-subject + "Function returning the subject of an article (a string). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + "Function returning the publish date of an article (a time value). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-update-date-function #'nnatom--read-update + "Function returning the update date of an article (a time value). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-authors + "Function returning the author of an article (a string). +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-headers-function #'ignore + "Function returning an alist of article-wide MIME headers. +Each element of this alist maps a MIME header (a symbol, +i.e. `Content-Type') to its value. As a special case, `:boundary' +maps to a string which will serve as the boundary between article +parts. This must be supplied if a custom boundary is used in a +multipart content type header. The default boundary is \"-_nnatom_-\", +and is automatically modified to match the name of the back end. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-links-function #'nnatom--read-links + "Function returning all links contained in an article. +For the default `nnatom-print-content-function', it should return a +list of links, where each link is an alist mapping MIME content types +to links formatted for display in a part of that type. Each content +type may also be a list of content types. +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-read-parts-function #'nnatom--read-parts + "Function returning an alist associating parts of an article to their headers. +For the default `nnatom-print-content-function', each part should be a +string. Otherwise, it can be any Lisp object. The \"headers\" of +each part should be a list where each element is either a cons of a +MIME header (a symbol, i.e. `Content-Type') and its value (a string), +or any other Lisp object. MIME headers will be printed, the rest will +be passed on to `nnatom-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +It should accept a single argument, a Lisp object representing an article.") + +(defvoo nnatom-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnatom-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnatom-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnatom--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnatom-backend) + "nn") + (gnus-newsgroup-savable-name server)) + gnus-directory)) + +(defun nnatom--read-server (server) + "Read SERVER's information from `nnatom-directory'." + (if-let ((f (nnatom--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnatom-servers)) + (nnheader-report nnatom-backend "Can't read %s" server))) + +(defun nnatom--write-server (server) + "Write SERVER's information to `nnatom-directory'." + (if-let ((f (nnatom--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnatom-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n") + (prin1 s (current-buffer)) + (insert "\n") + t) + t) + (nnheader-report nnatom-backend "Can't write %s" f))) + +(defun nnatom--server-address (server) + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnatom-backend 'address) + (cddr (gnus-server-to-method + (concat + (symbol-name nnatom-backend) ":" + server))))) + server)) + server)) + +(defun nnatom--parse-feed (feed &optional group) + "Parse FEED into a new or existing server. +Optionally, only parse GROUP." + (let* ((feed (nnatom--server-address feed)) + (s (or (gethash feed nnatom-servers) (nnatom--read-server feed))) + (g (or (and (hash-table-p s) (gethash group s)) `[,group])) + (name group) ; (Maybe) fake name + (group (or (aref g 0) name)) ; Real name + data) + (when (setq data (funcall nnatom-read-feed-function feed group) + s (or (gethash feed nnatom-servers) + (make-hash-table :test 'equal))) + (while-let ((cg (or (and name `(,data)) + (funcall nnatom-read-group-function data))) + (cg (prog1 (car cg) (setq data (cdr cg))))) + (let* ((name (funcall nnatom-read-title-function cg)) ; Real name + (group (gethash name nnatom-group-names name)) ; (Maybe) fake name + (info (gnus-get-info + (concat (symbol-name nnatom-backend) "+" feed ":" group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnatom-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnatom-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnatom-read-article-function cg stale)) + (article (prog1 (car article) (setq cg (cdr article))))) + (when-let ((id (funcall nnatom-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnatom-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnatom-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (cl-incf max)))) + (publish (funcall nnatom-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnatom-read-author-function article) + group-author group) + ,(or (funcall nnatom-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnatom-read-links-function article) + ,(funcall nnatom-read-parts-function article) + ,(funcall nnatom-read-headers-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnatom-servers)))) + +;;;; Gnus backend functions: + +(defvoo nnatom-group nil + "Name of the current group.") + +(defvoo nnatom-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnatom-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS HEADERS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers. +- HEADERS is an alist associating article-wide MIME headers to their value.") + +(defvoo nnatom-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnatom-group-article-min-num 1 + "Minimum article number for the current group.") + +(defvar nnatom-date-format "%F %X" + "Format of displayed dates.") + +(nnoo-define-basics nnatom) + +(defun nnatom--current-server-no-prefix () + (string-remove-prefix (concat (symbol-name nnatom-backend) "+") + (nnoo-current-server nnatom-backend))) + +(defun nnatom--group-data (group &optional server) + (let ((s (gethash server nnatom-servers)) c) + (or (and (hash-table-p s) (gethash group s)) + (and (setq c (nnatom--current-server-no-prefix)) + (setq s (gethash c nnatom-servers)) + (hash-table-p s) (gethash group s)) + (catch :stop (maphash (lambda (n s) + (or (string= n server) + (string= n c) + (when-let (((hash-table-p s)) + (g (gethash group s))) + (throw :stop g)))) + nnatom-servers))))) + +(defun nnatom-retrieve-article (article group) + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string nnatom-date-format (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnatom-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnatom--current-server-no-prefix))) + (g (or (nnatom--group-data group server) + `[ nil ,nnatom-group-article-ids ,nnatom-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnatom-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnatom-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnatom-backend "Group %s not found" (or group "")))) + +(deffoo nnatom-open-server (server &optional defs backend) + (let ((backend (or backend 'nnatom)) + (a (nnatom--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (or (gethash server nnatom-servers) + (nnatom--read-server a))) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash group g nnatom-group-names))) + s)) + (or s (file-writable-p (nnatom--server-file a)) + (and (nnoo-close-server nnatom-backend server) + (nnheader-report + nnatom-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnatom-request-close () + (maphash (lambda (server _) + (nnatom--write-server + (nnatom--server-address server))) + nnatom-servers) + (setq nnatom-servers (make-hash-table :test 'equal) + nnatom-status-string nil) + t) + +(defun nnatom--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, possibly with LINKS added." + (let ((links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and content (format "%s\n\n" content)) + (mapconcat (lambda (link) + (cdr (assoc (cdr (assq 'Content-Type attributes)) link + (lambda (types type) + (if (stringp types) (string= types type) + (member type types)))))) + links))))) + +(defvoo nnatom-print-content-function #'nnatom--print-content + "Function returning a single piece of content (a string). +It should accept three arguments, a part and its attributes (as returned +by `nnatom-read-parts-function'), and links.") + +(defun nnatom--print-part (content headers mime links) + (concat + (mapconcat (lambda (header) + (when-let ((m (car-safe header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnatom-print-content-function content headers links))) + +(deffoo nnatom-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnatom--current-server-no-prefix))) + (g (or (nnatom--group-data group server) + (and (setq group nnatom-group) + `[ nil ,nnatom-group-article-ids + ,nnatom-group-articles + ,nnatom-group-article-max-num + ,nnatom-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((links (aref a 5)) + (parts (aref a 6)) + (headers (aref a 7)) + (boundary (or (cdr (assq :boundary headers)) + (format "-_%s_-" nnatom-backend))) + (multi (length> parts 1)) + (mime '( Content-Type Content-Disposition + Content-Transfer-Encoding))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnatom-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + (if (assq 'MIME-Version headers) "" "MIME-Version: 1.0\n") + (mapconcat (lambda (header) + (unless (keywordp (car header)) + (format "%s: %s\n" (car header) (cdr header)))) + headers) + (if multi + (if (assq 'Content-Type headers) "" + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary)) + (prog1 (nnatom--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (format "--%s\n%s\n" boundary + (nnatom--print-part + (car part) (cdr part) mime links))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnatom-backend "No such article"))) + +(deffoo nnatom-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnatom--current-server-no-prefix))) + (g (or (if fast (nnatom--group-data group server) + (setq server (nnatom--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnatom-group group + nnatom-group-article-ids (aref g 1) + nnatom-group-articles (aref g 2) + nnatom-group-article-max-num (aref g 3) + nnatom-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnatom-group-article-ids) + nnatom-group-article-min-num + nnatom-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnatom-backend "Group %s not found" group)))) + +(deffoo nnatom-close-group (group &optional server) + (and (string= group nnatom-group) + (setq nnatom-group nil + nnatom-group-article-ids (make-hash-table :test 'equal) + nnatom-group-articles (make-hash-table :test 'eql) + nnatom-group-article-max-num 0 + nnatom-group-article-min-num 1)) + (setq server (or server (nnatom--current-server-no-prefix))) + (nnatom--write-server server)) + +(deffoo nnatom-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnatom--parse-feed + (or server (nnatom--current-server-no-prefix)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnatom-request-post (&optional _server) + (nnheader-report nnatom-backend "%s is a read only backend" nnatom-backend)) + +;;;; Optional back end functions: + +(deffoo nnatom-retrieve-groups (_groups &optional server) + (nnatom-request-list (or server (nnatom--current-server-no-prefix))) + 'active) + +(deffoo nnatom-request-type (_group &optional _article) + 'unknown) + +(deffoo nnatom-request-group-description (group &optional server) + (when-let ((server (or server (nnatom--current-server-no-prefix))) + (g (nnatom--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnatom-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnatom--current-server-no-prefix))) + (s (gethash server nnatom-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnatom-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnatom--current-server-no-prefix))) + (s (or (gethash server nnatom-servers) + (and ; Open the server to add it to `nnatom-servers' + (save-match-data + (nnatom-open-server + server + (cdr ; Get defs and backend. + (assoc server (cdr (assq 'nnatom nnoo-state-alist)) + (lambda (car key) + (and (stringp car) + (string-match + (concat + "\\(\\(nn[[:alpha:]]+\\)\\+\\)?" + "\\(" (regexp-quote key) "\\)") + car) + (setq server car))))) + (prog1 (if (stringp (match-string 1 server)) + (intern (match-string 2 server)) + 'nnatom) + (setq server (match-string 3 server)))) + (gethash server nnatom-servers))))) + (g (or (nnatom--group-data group server) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnatom-group-names) + (remhash group s) + (and (string= group nnatom-group) + (setq nnatom-group new-name)) + t)) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +;;;; Utilities: + +(defmacro nnatom-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnatom-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnatom-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnatom-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnatom-servers) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnatom-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnatom-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnatom-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnatom-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnatom-open-server server defs ',backend)) + (nnoo-import ,backend (nnatom)))) + +(provide 'nnatom) + +;;; nnatom.el ends here -- 2.41.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Jul 18 13:33:01 2023 Received: (at submit) by debbugs.gnu.org; 18 Jul 2023 17:33:01 +0000 Received: from localhost ([127.0.0.1]:54181 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qLoZJ-0005TK-1j for submit@debbugs.gnu.org; Tue, 18 Jul 2023 13:33:01 -0400 Received: from lists.gnu.org ([2001:470:142::17]:56294) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qLoZF-0005Ss-JU for submit@debbugs.gnu.org; Tue, 18 Jul 2023 13:32:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qLoZA-0002lX-98 for bug-gnu-emacs@gnu.org; Tue, 18 Jul 2023 13:32:52 -0400 Received: from mail.ericabrahamsen.net ([52.70.2.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qLoZ8-0000tQ-Qd for bug-gnu-emacs@gnu.org; Tue, 18 Jul 2023 13:32:51 -0400 Received: from localhost (c-71-197-232-41.hsd1.wa.comcast.net [71.197.232.41]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id C5815FA002; Tue, 18 Jul 2023 17:32:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1689701562; bh=kjdI1ABqVWo3F3BqIGjdKJ9WsXGUdd9Y9meWLjBiDj8=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=IArQRtTIvmDf9GKBJJnpAAe38PgvWIDJTG95A8KGvX6gpv3dLpK2oxxPsGc8xJB5l RpVj3H4DZB8Tm03wJydiutZP3TLY0j2X9t2nFZZtrZLx4cTye9m93Cavqa/yufacvq ScZ8MiCv1CAijZfsvUUe8GHO3l4uBJC+8fW8pUp8= From: Eric Abrahamsen To: Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <878rbikn0a.fsf@dsemy.com> (Daniel Semyonov via's message of "Sat, 15 Jul 2023 03:59:49 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> <87r0prkgsz.fsf@dsemy.com> <873525udhn.fsf@ericabrahamsen.net> <87pm59984x.fsf@dsemy.com> <87a5wdskw8.fsf@ericabrahamsen.net> <87edlm5yc9.fsf@dsemy.com> <874jmie0qj.fsf@ericabrahamsen.net> <87bkgq2nx3.fsf@dsemy.com> <87o7kqch2t.fsf@ericabrahamsen.net> <878rbikn0a.fsf@dsemy.com> Date: Tue, 18 Jul 2023 10:32:40 -0700 Message-ID: <87h6q19lc7.fsf@ericabrahamsen.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=52.70.2.18; envelope-from=eric@ericabrahamsen.net; helo=mail.ericabrahamsen.net X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit Cc: 64202@debbugs.gnu.org, Daniel Semyonov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" writes: >>>>>> Eric Abrahamsen writes: > > > Where do you see that Gnus knows which backends require an > > address? That would be very handy. > > Forgot to answer this -- 'gnus-valid-select-methods' holds this > information (and a bunch of other info too). Huh, I'd forgotten about that. This certainly looks like the right way to decide whether `gnus-group-make-group' should prompt for an address or not. > > But my reading of this function is that we don't even known that > > the backend is loaded until the end of the group creation > > process. You could probably mildly abuse the 'request-create-group > > server function to further prompt the user for the URL. > > > It is a bit annoying that the ADDRESS argument is in there, but > > not prompted for. It looks like that's been the case for at least > > two decades, though. > > I'll add this to the pile of stuff I wanna change in Gnus once I'm > satisfied with this backend (quite a large pile at this point :). Careful! Like me, you may end up a casualty of mission creep. > In any case, I still didn't have much time to work on the backend > unfortunately, but I attached a patch which incorporates the fixes from > my last message + changes based on comments from Stefan. > > Things left to do: > 1. Ensure support for none-libxml builds (I should probably figure out > how ERT works to make this a non-issue going forward). > 2. Factor out nnfeed.el and base nnatom on top of it (shouldn't be hard). > 3. Add support for RSS (and steal cool stuff from nnrss). > 4. Support Atom feeds linked in HTML documents. Is there a repository somewhere where you're working on this? Thanks, Eric From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 19 00:41:01 2023 Received: (at submit) by debbugs.gnu.org; 19 Jul 2023 04:41:01 +0000 Received: from localhost ([127.0.0.1]:54670 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qLyzk-0008Sk-NC for submit@debbugs.gnu.org; Wed, 19 Jul 2023 00:41:01 -0400 Received: from lists.gnu.org ([2001:470:142::17]:38640) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qLyza-0008SC-PM for submit@debbugs.gnu.org; Wed, 19 Jul 2023 00:40:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qLyzU-0000lb-MM for bug-gnu-emacs@gnu.org; Wed, 19 Jul 2023 00:40:45 -0400 Received: from dsemy.com ([46.23.89.208]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qLyzS-0000xM-TC for bug-gnu-emacs@gnu.org; Wed, 19 Jul 2023 00:40:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=wfDh7v+j1HECw fvtnpqU1odOx6vy9EXrqVzJdlwnsa4=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=MAaNFikwtJrzerSIF0uiRS4MPyuStmd0toy 9lYiUsc8Z+7Cq83lwbyjreYB18lT/FYoEm7KW1GGigOSWs4buieuCxZnAL3qFWh5wXavNe lhvyVDQ6QUfB6xyhNzwoeaPLdkdwGpdS88RZ5l2AmT+NewtMSzGbmbA/Rw+axJalRiQDlM 7f2XI/3KD8V4GOE1il1OddYwYJZ12IaHi16O9B88cdXe7XTvJoukJQ+Aa8jcD9+uhCmSY2 afCB/zHOGlUxBr0EIurYqOEpkAnb7f+lIMhduNxrU/0KV0h0P9SacGn15gLkeS0xUbIP4j MK4MmftQrBAkdPWLgN5UOuZIBDg== Received: from coldharbour.home.arpa ( [147.235.213.12]) by dsemy.com (OpenSMTPD) with ESMTPSA id 5d69e3a1 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Wed, 19 Jul 2023 06:40:37 +0200 (CEST) Received: from localhost (coldharbour.home.arpa [local]) by coldharbour.home.arpa (OpenSMTPD) with ESMTPA id 01dfabaa; Wed, 19 Jul 2023 04:37:50 +0000 (UTC) From: Daniel Semyonov To: Eric Abrahamsen Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87h6q19lc7.fsf@ericabrahamsen.net> (Eric Abrahamsen's message of "Tue, 18 Jul 2023 10:32:40 -0700") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> <87r0prkgsz.fsf@dsemy.com> <873525udhn.fsf@ericabrahamsen.net> <87pm59984x.fsf@dsemy.com> <87a5wdskw8.fsf@ericabrahamsen.net> <87edlm5yc9.fsf@dsemy.com> <874jmie0qj.fsf@ericabrahamsen.net> <87bkgq2nx3.fsf@dsemy.com> <87o7kqch2t.fsf@ericabrahamsen.net> <878rbikn0a.fsf@dsemy.com> <87h6q19lc7.fsf@ericabrahamsen.net> Date: Wed, 19 Jul 2023 07:37:50 +0300 Message-ID: <87zg3s8qjl.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=46.23.89.208; envelope-from=daniel@dsemy.com; helo=dsemy.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit Cc: "Daniel Semyonov via Bug reports for GNU Emacs, the Swiss army knife of text editors" , 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) >>>>> Eric Abrahamsen writes: >> In any case, I still didn't have much time to work on the backend >> unfortunately, but I attached a patch which incorporates the >> fixes from my last message + changes based on comments from >> Stefan. >> >> Things left to do: 1. Ensure support for none-libxml builds (I >> should probably figure out how ERT works to make this a non-issue >> going forward). 2. Factor out nnfeed.el and base nnatom on top >> of it (shouldn't be hard). 3. Add support for RSS (and steal >> cool stuff from nnrss). 4. Support Atom feeds linked in HTML >> documents. > Is there a repository somewhere where you're working on this? I have , but I haven't updated it since posting the first version of the patch, when I started developing it "inside" the Emacs repo (not sure of the correct terminology). Honestly, I'm not the best with Git, so I'm not really sure what would be the best way to do this. Should I just create a mirror (or whatever its called) of the Emacs repo with my patch applied and work on it from there? > Thanks, Eric Thanks, Daniel From debbugs-submit-bounces@debbugs.gnu.org Wed Jul 19 11:39:28 2023 Received: (at submit) by debbugs.gnu.org; 19 Jul 2023 15:39:28 +0000 Received: from localhost ([127.0.0.1]:56996 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qM9Gy-0000hk-4i for submit@debbugs.gnu.org; Wed, 19 Jul 2023 11:39:28 -0400 Received: from lists.gnu.org ([2001:470:142::17]:60440) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qM9Gv-0000hW-U4 for submit@debbugs.gnu.org; Wed, 19 Jul 2023 11:39:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qM9Gl-0006ro-E7 for bug-gnu-emacs@gnu.org; Wed, 19 Jul 2023 11:39:16 -0400 Received: from mail.ericabrahamsen.net ([52.70.2.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qM9Gj-0000FH-7B for bug-gnu-emacs@gnu.org; Wed, 19 Jul 2023 11:39:14 -0400 Received: from localhost (c-71-197-232-41.hsd1.wa.comcast.net [71.197.232.41]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 81FBDFA0C3; Wed, 19 Jul 2023 15:39:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1689781149; bh=8ECClMf8/eR4x6PZFrQJnMD0h9uaK6Wy2Mq1JunpxCo=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=Nq+P521JoZ9Ldj92fobIz2RpMyt8WaERZLdzUJstApgJYi0HazEA89sIzMm356M2g ciri8Lzpfa+B8CyGLP2/+ereJs1v94kYko2Z5IrG3rNhXh73YLi7hvwJkfRCRQPor0 vh1GHDSOcypbiPzD85vMqgY147DvTpiq35Ka9P9M= From: Eric Abrahamsen To: Daniel Semyonov Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87zg3s8qjl.fsf@dsemy.com> (Daniel Semyonov's message of "Wed, 19 Jul 2023 07:37:50 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87y1k76eip.fsf@dsemy.com> <87fs6epd6d.fsf@dsemy.com> <87bkh2p96o.fsf@dsemy.com> <83ilb4qb44.fsf@gnu.org> <87r0prkgsz.fsf@dsemy.com> <873525udhn.fsf@ericabrahamsen.net> <87pm59984x.fsf@dsemy.com> <87a5wdskw8.fsf@ericabrahamsen.net> <87edlm5yc9.fsf@dsemy.com> <874jmie0qj.fsf@ericabrahamsen.net> <87bkgq2nx3.fsf@dsemy.com> <87o7kqch2t.fsf@ericabrahamsen.net> <878rbikn0a.fsf@dsemy.com> <87h6q19lc7.fsf@ericabrahamsen.net> <87zg3s8qjl.fsf@dsemy.com> Date: Wed, 19 Jul 2023 08:39:08 -0700 Message-ID: <87wmyvhpwj.fsf@ericabrahamsen.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=52.70.2.18; envelope-from=eric@ericabrahamsen.net; helo=mail.ericabrahamsen.net X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit Cc: "Daniel Semyonov via Bug reports for GNU Emacs, the Swiss army knife of text editors" , 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) On 07/19/23 07:37 AM, Daniel Semyonov wrote: >>>>>> Eric Abrahamsen writes: > > >> In any case, I still didn't have much time to work on the backend > >> unfortunately, but I attached a patch which incorporates the > >> fixes from my last message + changes based on comments from > >> Stefan. > >> > >> Things left to do: 1. Ensure support for none-libxml builds (I > >> should probably figure out how ERT works to make this a non-issue > >> going forward). 2. Factor out nnfeed.el and base nnatom on top > >> of it (shouldn't be hard). 3. Add support for RSS (and steal > >> cool stuff from nnrss). 4. Support Atom feeds linked in HTML > >> documents. > > > Is there a repository somewhere where you're working on this? > > I have , but I haven't updated it since posting > the first version of the patch, when I started developing it "inside" > the Emacs repo (not sure of the correct terminology). > Honestly, I'm not the best with Git, so I'm not really sure what would > be the best way to do this. Should I just create a mirror (or whatever > its called) of the Emacs repo with my patch applied and work on it from > there? It's not a big deal, I just thought if you had an ongoing branch someplace publicly available I would pull it and follow along. It's a little harder to see what's changed each time you send a patch as an attachment, but it's not that important. From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 03 07:50:54 2023 Received: (at control) by debbugs.gnu.org; 3 Sep 2023 11:50:54 +0000 Received: from localhost ([127.0.0.1]:39870 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qclcz-0004nu-SW for submit@debbugs.gnu.org; Sun, 03 Sep 2023 07:50:54 -0400 Received: from mail-lf1-x12b.google.com ([2a00:1450:4864:20::12b]:59627) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qclcv-0004nZ-3f for control@debbugs.gnu.org; Sun, 03 Sep 2023 07:50:52 -0400 Received: by mail-lf1-x12b.google.com with SMTP id 2adb3069b0e04-500913779f5so903407e87.2 for ; Sun, 03 Sep 2023 04:50:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693741833; x=1694346633; darn=debbugs.gnu.org; h=to:subject:message-id:date:mime-version:from:from:to:cc:subject :date:message-id:reply-to; bh=DTVcalsIiEVA+secOBW3wBwmHOY8wq9lpmGoiaZ/oRY=; b=i4bSHm94Y13xszclGg/+mtCjDO4mwpnmAczYVQFfyeIlBPCuvBE27cVxA9HoR2ng2B IAyH7vFVdjNHUlrmJb32Z/+t4FECN9i8YfftLTXQwU/x/vAU40sEx6mF6z4Tg7y1aHDM s7q21T0OcYisBuk53Gyb1QNZuv4mmB2H8SQoiO3+ZTpbzaJxiCdRKurQfYb3ftu5vOXH /TbSUZTDoeFIRcjZvLUqNA0OAuuLCNwsFmtkOtgQ52jQ3Gwp+kpzHdKXt+yixQ2u7p6P TsPK5iFKz7q+/Oiuz/7TtrysCZEqSFhQrqeBnZn7OEeymcemprmfQFN0NIo1yQru/LRA Scow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693741833; x=1694346633; h=to:subject:message-id:date:mime-version:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=DTVcalsIiEVA+secOBW3wBwmHOY8wq9lpmGoiaZ/oRY=; b=b7861N+j6AeF29ha/mA2DuBvfWCcv8zTYUB1jgQl09r3O6cvSbMYcqtY6PTL2Ouolk nFju0e7HJ2DfzjA/xR6xLpSXDTs/f948rHSBc3w15p94fIvommoVECIbtNAZW7uCS1se X76JXLpSD3zEvgIRDoPeB+BtKDIxWiVaAE4P8JNcWOQ93uCFsBxSI4kpoo3Z5GXMlX5m cqvXk5/j+muCvidWSnX/rIte65rK1srUms5zQ7CYowjcaZbEbMLZTOJbbNnVw4x0871F Ihj0nvN3JpFQIlN2j0KfENullhNobCcM9ARmJF1tu5WRFo1LJ7i/pt3ssD1KBtn2h5Mv 1Riw== X-Gm-Message-State: AOJu0Yy8WtISPhM4U3L+FBzN68Dxb++xA8N73/I4jqj0Lv8bkOpuOxW+ cQ0KOGZUhuJjvCBgwICDHogxnXKHpj//cXZcFCmMDMyaF5A= X-Google-Smtp-Source: AGHT+IHn8nbmojuxcg6pIsrXuGzWvgTIW824b9+e/GwZ8P95TcXaF7HXiq9ddPDAILDFRru3dmKBRSM095qPrnEvfEc= X-Received: by 2002:a05:6512:a95:b0:500:977d:a4e1 with SMTP id m21-20020a0565120a9500b00500977da4e1mr1754357lfu.40.1693741832634; Sun, 03 Sep 2023 04:50:32 -0700 (PDT) Received: from 753933720722 named unknown by gmailapi.google.com with HTTPREST; Sun, 3 Sep 2023 04:50:32 -0700 From: Stefan Kangas MIME-Version: 1.0 Date: Sun, 3 Sep 2023 04:50:32 -0700 Message-ID: Subject: control message for bug #64202 To: control@debbugs.gnu.org Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) severity 64202 wishlist quit From debbugs-submit-bounces@debbugs.gnu.org Mon Sep 25 05:31:08 2023 Received: (at control) by debbugs.gnu.org; 25 Sep 2023 09:31:08 +0000 Received: from localhost ([127.0.0.1]:44226 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qkhvo-0002oC-Ev for submit@debbugs.gnu.org; Mon, 25 Sep 2023 05:31:08 -0400 Received: from mail-lj1-x230.google.com ([2a00:1450:4864:20::230]:55383) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qkhvm-0002hk-7p for control@debbugs.gnu.org; Mon, 25 Sep 2023 05:31:07 -0400 Received: by mail-lj1-x230.google.com with SMTP id 38308e7fff4ca-2b9338e4695so97717101fa.2 for ; Mon, 25 Sep 2023 02:30:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695634248; x=1696239048; darn=debbugs.gnu.org; h=to:subject:message-id:date:mime-version:from:from:to:cc:subject :date:message-id:reply-to; bh=KB4d7U9JInj89zc9rMnRRUcNcmltSX1rBw3w6snx9D0=; b=d3a4ppDs7EsR3DfzSY9obHS24LBqPgFJjJUTlmHVUB+bHvrQF/R51CxlBQlYuJpzMz 3QtPHmUkmRoe7uHjRK2s36/jumdQWrmv/XYXboghX25N+EWi9l9pW7eGN69S+cly/p7R NRqWIPRATPUnt0yTel3Xl+/zxl5UU6t0KjkMfcH68KReFicQDJ5L2zQMPj/pqX7ZTA7m XHWr6IX0euqr5tUAoGHoJ2c95XVtWq86F/p8XWNqX6ivm+KFjJmHS9+GhrzMibJ9vSKI ilVOLpKsrA8DBEmi1uqESuN07Jt85YXqDi/piKeN3b2eUwqNRrLr2uvD0pVIaMpGSd3O b13w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695634248; x=1696239048; h=to:subject:message-id:date:mime-version:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=KB4d7U9JInj89zc9rMnRRUcNcmltSX1rBw3w6snx9D0=; b=chzDYXd/VnIW0SY1LJN0+MPOTVyeQ0Lg8ZExAQ1Vpa7Y915+mhTJtxMFEoVJmMw89Q Ehm5uHxt1n18MN/jIcILKSlPmOGe01j92dFb32VyvkH4y0tc6caLtsmsgv4m4+wuYW1D wl2sX/3ZmEt3wgVRwhT0TNEvtnviGNZkv7wjuznckmdVX2NM0vkC1UXjhdGlgAU6jLtB g+aKhjNJdFFChOFnYNApF+OBHLTKTl2gyLSoQXIi1HkgfKyzdCeXpp8yz+Zxpej7FuP6 MDCjzs7Cjd77g/vF52JspvJ4A1LpVfIk9EqKg1L15tzz3ynuS84EkIfKgz8oR+AtSi7O 6UAg== X-Gm-Message-State: AOJu0Yxn1dRfkUJRf+jE3JMCAh83dKT5ZXlyOLHTMhVummYM3+WM8jsL i+U3b7UVyd1/BEbHhR/UMQ0mMPdhqt+AeihlaoCTqtwD X-Google-Smtp-Source: AGHT+IF+AvKWIwh6nD7BVEUSuS1D7akjhzpcyc1pquj4E5gdaQkxRlH0+MBGqquuKFzzsoWYN6KwwnmGuDN1ms4yV2Q= X-Received: by 2002:a2e:9652:0:b0:2bc:b0ae:a9e5 with SMTP id z18-20020a2e9652000000b002bcb0aea9e5mr5440466ljh.42.1695634247736; Mon, 25 Sep 2023 02:30:47 -0700 (PDT) Received: from 753933720722 named unknown by gmailapi.google.com with HTTPREST; Mon, 25 Sep 2023 02:30:47 -0700 From: Stefan Kangas MIME-Version: 1.0 Date: Mon, 25 Sep 2023 02:30:47 -0700 Message-ID: Subject: control message for bug #64202, #66188 To: control@debbugs.gnu.org Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) reassign 64202 emacs,gnus forcemerge 64202 66188 thanks From debbugs-submit-bounces@debbugs.gnu.org Thu Dec 07 23:24:08 2023 Received: (at submit) by debbugs.gnu.org; 8 Dec 2023 04:24:08 +0000 Received: from localhost ([127.0.0.1]:43575 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rBSPG-0002hA-2g for submit@debbugs.gnu.org; Thu, 07 Dec 2023 23:24:08 -0500 Received: from lists.gnu.org ([2001:470:142::17]:43960) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rBRbI-0000gl-Kh for submit@debbugs.gnu.org; Thu, 07 Dec 2023 22:32:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rBRb0-0001Ja-Cr for bug-gnu-emacs@gnu.org; Thu, 07 Dec 2023 22:32:10 -0500 Received: from wout2-smtp.messagingengine.com ([64.147.123.25]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rBRax-0004nT-9W for bug-gnu-emacs@gnu.org; Thu, 07 Dec 2023 22:32:10 -0500 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 3085F3200B6D; Thu, 7 Dec 2023 22:32:02 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 07 Dec 2023 22:32:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alshehhi.io; h= cc:cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm2; t=1702006321; x=1702092721; bh=Zn sQUTUz0J+WeSo5IodhMNNrk7ho7bTUzs3BKyZOb+s=; b=HwuVXuckz1/U4fvBOR Y3VYe2sNDnTLKshvmWGjRU4k6INL9LtleELMXG35A82BKPrF4s1y/TNEF/0Pzhhp nu9sGa5OSr2f+hDg7wxKE90sH/XFY30cniTD3RhUS/nwQ8pFyz+mTN0mSZha+8Ws 2nYnFTpqNqvaWhKcinDUrFob1qXiVrjgzYP1G+RrosdEZd3wEXv8wkXTqZgLIZdK ph8TdbpClnz6Ov6Wlz39ByS1gwhKmFon/LZNlGheZWS03zOVP1O8+GhWO8Qd1iUU DDeWvmlVnM65/8u1XD1lyrkj9UdC4boZDH1J9WLJ70FTJTr7KuWSto85bkYUYMxs W8Qw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1702006321; x=1702092721; bh=ZnsQUTUz0J+We So5IodhMNNrk7ho7bTUzs3BKyZOb+s=; b=O7NM8bvmGKqRS69O06I0QEi8BWbbT 9Ql4ceJN6Jb3UJ9sHUMyk5+aAQD06stX2AoIKz0TQMkw3tYwJfm7ORKYsgGagRIu 71CRFeOAx4OEL5Wy6ia8LwE2Ewgw21qBjbzXwTlhGUtkW2487BSq4tzT84Ra5EOU 7PrkAu8rCxK7SfUu/JxYcsrYSUYUTvEQ4y2BfMZF8c1WxDnG5QvrobsavbeGRQST zBO6hiS78+anKsYZK95GJ2TRp+rDyh1hVhyK625Cp17bFH/qChYexdz7TSmyR4sy EbuWiXbPUl0R9bplpsydIl7eJCJjSZNJC/lME0gTaPbrXPNW2YhUVDQxw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudekhedgtdegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefujghffffkfgggtgesthdtredttdertdenucfhrhhomhepfdfjuhhs rghinhcutehlshhhvghhhhhifdcuoehhuhhsrghinhesrghlshhhvghhhhhirdhioheqne cuggftrfgrthhtvghrnhepkeeujedufeehgeegffefteelgeetgeehvdejteelgfekvddv vdetfefhudffuedunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilh hfrhhomhepfhgrmhhilhihsegrlhhshhgvhhhhihdrihho X-ME-Proxy: Feedback-ID: ic2a949cb:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 7 Dec 2023 22:32:00 -0500 (EST) From: "Husain Alshehhi" To: Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87v8fhmgvw.fsf@dsemy.com> (Daniel Semyonov via's message of "Wed, 21 Jun 2023 10:08:51 +0300") References: <87v8fhmgvw.fsf@dsemy.com> Date: Thu, 07 Dec 2023 21:31:59 -0600 Message-ID: <875y1974k0.fsf@> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=64.147.123.25; envelope-from=family@alshehhi.io; helo=wout2-smtp.messagingengine.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, INVALID_MSGID=0.568, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi Daniel. Just wondering about the state of this change. Is the atom method merged in into emacs? Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.7 SPF_NEUTRAL SPF: sender does not match SPF record (neutral) 1.2 INVALID_MSGID Message-Id is not valid, according to RFC 2822 -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Thu, 07 Dec 2023 23:24:00 -0500 Cc: 64202@debbugs.gnu.org, Daniel Semyonov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.8 (/) Hi Daniel. Just wondering about the state of this change. Is the atom method merged in into emacs? From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 08 04:32:48 2023 Received: (at 64202) by debbugs.gnu.org; 8 Dec 2023 09:32:48 +0000 Received: from localhost ([127.0.0.1]:43865 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rBXDz-0006bF-Mg for submit@debbugs.gnu.org; Fri, 08 Dec 2023 04:32:48 -0500 Received: from dsemy.com ([46.23.89.208]:26362) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rBXDv-0006ap-Fw for 64202@debbugs.gnu.org; Fri, 08 Dec 2023 04:32:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=G4DpRvnOm9o03 ty9OKC8BGB2y80X/cdXJ3VGFwDN/BY=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=rS4qOe1PcYqonYIXCUNL1bEYEOHF9eab61F avrLce9VdCswUpKDeytSiB1lET4Bj+MKSzz5iORBMixWgHxcCzmvM+1Fl4QH8zM3qv7fNS jFX5YsNFfzcBwzzlcLUNdRwDqJd1c91FP2fYZheSf63HNGxdJTzy2frC9djR0p4GYNrWfY 9JTAxNAnpefIF5VGfZEqNDvXHhQ2VpRQnukMKJ7f79sohluHX2CHlSgLsERoZHq93dYKg5 tio6PlXLrm1v37Tb7sF/+JSUxES/IP609k6oQpNqg5DbCGCJl6Qu1WmOwHBbyJ5iz3nQOY MgcacuJLZ1DoEVdWMECf2cmxSxA== Received: from localhost ( [2a06:c701:4842:fe00:1aef:9a7f:fcb4:4fce]) by dsemy.com (OpenSMTPD) with ESMTPSA id a2443710 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 8 Dec 2023 10:32:23 +0100 (CET) From: Daniel Semyonov To: "Husain Alshehhi" Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <875y1974k0.fsf@> (Husain Alshehhi's message of "Thu, 07 Dec 2023 21:31:59 -0600") References: <87v8fhmgvw.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) Date: Fri, 08 Dec 2023 11:32:20 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: bug-gnu-emacs@gnu.org, 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >>>>> Husain Alshehhi writes: > Hi Daniel. Just wondering about the state of this change. Is the > atom method merged in into emacs? I've procrastinated on this for a while and then due to unforeseen circumstances I've been able to use Emacs on more than my phone only a few times in the past two months or so. I don't think these circumstances will change soon, so I'm not sure when I'll be able to work on this again unfortunately... FWIW it works in its current state, I'll see if I can find the time to separate the code between an "nnfeed" backend and nnatom on my phone (but still I don't want to promise anything), and after that there is no reason IMO it shouldn't be merged; the backend already supports in theory the features I still want to add so I should be able to just add them later (and they aren't required to use the backend or anything). If anyone feels inclined to do this before me I would be very grateful, using an Android keyboard for an extended period can get frustrating. Daniel From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 08 11:02:11 2023 Received: (at submit) by debbugs.gnu.org; 8 Dec 2023 16:02:11 +0000 Received: from localhost ([127.0.0.1]:45820 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rBdIo-00042k-Lv for submit@debbugs.gnu.org; Fri, 08 Dec 2023 11:02:11 -0500 Received: from lists.gnu.org ([2001:470:142::17]:34828) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rBdIn-00042H-DH for submit@debbugs.gnu.org; Fri, 08 Dec 2023 11:02:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rBdIT-0000zL-Pn for bug-gnu-emacs@gnu.org; Fri, 08 Dec 2023 11:01:49 -0500 Received: from mail.ericabrahamsen.net ([52.70.2.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rBdIS-0000jW-5w for bug-gnu-emacs@gnu.org; Fri, 08 Dec 2023 11:01:49 -0500 Received: from localhost (71-212-21-65.tukw.qwest.net [71.212.21.65]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 0988AFA09E; Fri, 8 Dec 2023 16:01:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1702051300; bh=voXyyWM4lxPqKdAzUQ9A0u753rzt8u0nuJleaD9d9T0=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=eQ+r0x31+HOo/hIf5cFSnsRZXdt05xUOhsGjLA4XPk9qzUVX82ZAQaXlMVlWP5Sac sklQuQiItJB2ZUrs40jmDY2UzKJNJVz05528T5ZOVTuz5yROc4i9lcAS4Sv82qNZ99 t9nNkAg+B/8pn931R/R42mTfYLBGeZK/SrBxvZgg= From: Eric Abrahamsen To: Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: (Daniel Semyonov via's message of "Fri, 08 Dec 2023 11:32:20 +0200") References: <87v8fhmgvw.fsf@dsemy.com> Date: Fri, 08 Dec 2023 08:01:38 -0800 Message-ID: <87v898of8d.fsf@ericabrahamsen.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=52.70.2.18; envelope-from=eric@ericabrahamsen.net; helo=mail.ericabrahamsen.net X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit Cc: Husain Alshehhi , 64202@debbugs.gnu.org, Daniel Semyonov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) Daniel Semyonov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" writes: >>>>>> Husain Alshehhi writes: > > > Hi Daniel. Just wondering about the state of this change. Is the > > atom method merged in into emacs? > > I've procrastinated on this for a while and then due to unforeseen > circumstances I've been able to use Emacs on more than my phone only a > few times in the past two months or so. I don't think these > circumstances will change soon, so I'm not sure when I'll be able to > work on this again unfortunately... > > FWIW it works in its current state, I'll see if I can find the time to > separate the code between an "nnfeed" backend and nnatom on my phone > (but still I don't want to promise anything), and after that there is no > reason IMO it shouldn't be merged; the backend already supports in > theory the features I still want to add so I should be able to just add > them later (and they aren't required to use the backend or anything). > If anyone feels inclined to do this before me I would be very grateful, > using an Android keyboard for an extended period can get frustrating. Maybe add a roadmap or TODO list to the sourcehut repo, in case anyone finds time to help out? From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 08 11:32:36 2023 Received: (at submit) by debbugs.gnu.org; 8 Dec 2023 16:32:36 +0000 Received: from localhost ([127.0.0.1]:45874 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rBdmE-0005X9-Rp for submit@debbugs.gnu.org; Fri, 08 Dec 2023 11:32:36 -0500 Received: from lists.gnu.org ([2001:470:142::17]:39160) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rBdmB-0005WR-0e for submit@debbugs.gnu.org; Fri, 08 Dec 2023 11:32:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rBdll-0006UF-HH for bug-gnu-emacs@gnu.org; Fri, 08 Dec 2023 11:32:05 -0500 Received: from dsemy.com ([46.23.89.208]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rBdlg-0003h3-9j for bug-gnu-emacs@gnu.org; Fri, 08 Dec 2023 11:32:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=d3K/0Sxl8twUT 6P+05LDNNNiMTclHG3QgRNzy1ZE6fU=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=f4vncukkT1FxBh6X0WHxUEGX2MN+YLVqa94 RWvpoeetTGJdM9WUdguHU/T6oOn6wqoUDpIwMZQ0/XgTF8+BVk+cl4sjgWanJK8MpG5PHE TaX2SVbtEIyCbyScthsTknMGjA7pt1urkWSs9R0CJ+ivUREjlQIM4V8J0N2l7tcoGa6ZkF NM42YZrD6ASa27UpugeqULTGQFfMPe91cJn5nVVEXjzaeim4TAGSJo/amVhESAVNcCTXbi iPaNLRDF9S93pnvSLreKy9+VzFyXvpspNlNJNiE0jmsH8wc1FPe6FciVdcGBsaoNNnIUsn co8QlvkK+Pf+Rk9rZ487xiQoZPg== Received: from coldharbour.local ( [147.235.213.125]) by dsemy.com (OpenSMTPD) with ESMTPSA id 79433df4 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 8 Dec 2023 17:31:56 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id ebb0bd47; Fri, 8 Dec 2023 16:26:48 +0000 (UTC) From: Daniel Semyonov To: 64202@debbugs.gnu.org Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: (Daniel Semyonov's message of "Fri, 08 Dec 2023 11:32:20 +0200") References: <87v8fhmgvw.fsf@dsemy.com> Date: Fri, 08 Dec 2023 18:26:48 +0200 Message-ID: <8734wcfynr.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=46.23.89.208; envelope-from=daniel@dsemy.com; helo=dsemy.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_FILL_THIS_FORM_SHORT=0.01, T_SCC_BODY_TEXT_LINE=-0.01, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit Cc: Eric Abrahamsen , Husain Alshehhi , bug-gnu-emacs@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) --=-=-= Content-Type: text/plain >>>>> Daniel Semyonov writes: > I've procrastinated on this for a while and then due to unforeseen > circumstances I've been able to use Emacs on more than my phone > only a few times in the past two months or so. I don't think > these circumstances will change soon, so I'm not sure when I'll be > able to work on this again unfortunately... Well as luck would have it I had a rare opportunity to spend a few hours at home today with no prior business I had to take care of (and this was also fresh on my mind for a change) so I decided to try finish this while I could while I can use a normal computer. Attached is a new version of the patch which is identical in functionality (more or less) to the last one but nnatom is implemented on top of an abstract nnfeed backend. I made a lot of effort to document nnfeed, hopefully everything is clear, but I'm honestly very tired and might have made some mistakes. The docs in nnfeed.el are supposed to be all that's needed for someone basing a backend on it, so I'd like to know if it's clear. I just realized this patch still can't be merged though, NEWS and the manual need updated. I'll do it on my phone tomorrow if I have the time. Hopefully this can be merged soon finally though --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch Content-Description: patch >From dbb989d9584bdefab2470cbe8ef86103e2d263aa Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnatom.el: * lisp/gnus/nnfeed.el: New file. * doc/misc/gnus.texi: * etc/NEWS: Document nnatom --- doc/misc/gnus.texi | 58 ++++ etc/NEWS | 7 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 277 ++++++++++++++++++ lisp/gnus/nnfeed.el | 683 ++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 1026 insertions(+) create mode 100644 lisp/gnus/nnatom.el create mode 100644 lisp/gnus/nnfeed.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 586e4b94ba1..6691b0fba7b 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -17250,6 +17251,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17494,6 +17496,62 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@pxref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds +as Gnus servers (with a single group), as you would with any other +method, by supplying the location of the feed as the server address. +Note, however, that the server address shouldn't be prefixed with +@url{http://} or @url{https://}. + +Features of @code{nnatom} include: + +@itemize @bullet + +@item +Server data is saved per-server in the @file{atom} sub-directory of +@file{gnus-directory}. + +@item +An article part is generated for both the summary and the content for +each entry in the feed. Content of all MIME types should be displayed +correctly through Gnus (as long as the feed specifies a MIME type), +even base64 encoded files (like images for example), which are +supported by Atom. + +@item +Article modification and publish dates are tracked, and articles are +updated if changed. + +@item +Designed with modularity in mind, the back end theoretically supports +features which aren't available in Atom (and thus aren't actually +implemented in practice). For example, the back end operates under +the assumption that ``servers'' (Atom feeds) can have multiple groups, +when in reality the parsing functions will only ever return servers +with a single group. In addition, all parsing steps and some of the +printing is done in functions stored in server variables, to allow +very easily defining new back ends for different kinds of feeds by +inheriting from @code{nnatom}; these are called +@code{nnatom-read-*-function} and +@code{nnatom-print-content-function}, and their requirements are +detailed in their docstrings. A macro is provided to ease creation of +new inheriting backends. + +@defmac nnatom-define-basic-backend-interface backend +Define server variables expected by @code{nnatom} and import its back +end functions for @var{backend}. This macro (or code similar to it) should +be used by any inheriting back end. +@end defmac + +@end itemize + @node Other Sources @section Other Sources diff --git a/etc/NEWS b/etc/NEWS index 03a40c11c5b..47ecbf3f8e0 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -882,6 +882,13 @@ distracting and easily confused with actual code, or a significant early aid that relieves you from moving the buffer or reaching for the mouse to consult an error message. +** Gnus + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + ** JS Mode The binding 'M-.' has been removed from the major mode keymaps in 'js-mode' and 'js-ts-mode', having it default to the global binding diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index 6bf66233101..de5567d1fc0 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1348,6 +1348,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..e92a6f09baa --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,277 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format . +;; Other types of feeds may be supported by providing custom parsing +;; functions. + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'subr-x)) + +(require 'nnfeed) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'nnfeed) + +(nnoo-declare nnatom nnfeed) + +(nnfeed-define-basic-backend-interface nnatom) + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "\\`https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-name-absolute-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (when-let ((data (if (libxml-available-p) + (libxml-parse-xml-region + (point-min) (point-max)) + (car (xml-parse-region + (point-min) (point-max))))) + (auth (list 'authors))) + (when (eq (car data) 'top) + (setq data (assq 'feed data))) + (dom-add-child-before data auth) + (catch :stop ; Collect feed authors, stop at first entry. + (dolist (child (cdddr data)) + (let ((tag (car-safe child))) + (if (eq tag 'entry) + (throw :stop data) + (and (or (eq tag 'author) + (eq tag 'contributor)) + (dom-add-child-before auth child))))) + data))))))) +(defvoo nnatom-read-feed-function #'nnatom--read-feed + nil nnfeed-read-feed-function) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) +(defvoo nnatom-read-group-function #'nnatom--read-group + nil nnfeed-read-group-function) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) (setq data (cddr data))) + (while (and data (not (eq (car-safe (car data)) 'entry))) (pop data)) + (when-let ((article (car data)) + (auths (list 'authors)) (links (list 'links))) + (dom-add-child-before article links) + (dom-add-child-before article auths) + (dolist (child (cddddr article) `(,article . ,(cdr data))) + (pcase (car-safe child) ; Authors and links can appear + ((or 'author 'contributor) ; anywhere in the entry so we + (dom-add-child-before auths child) ; collect them all here to + (dom-add-child-before links child)) ; avoid looping over the + ((or 'link ; entry multiple times later. + (and 'content (guard (assq 'src (dom-attributes child))))) + (dom-add-child-before links child)))))) +(defvoo nnatom-read-article-function #'nnatom--read-article + nil nnfeed-read-article-function) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) +(defvoo nnatom-read-title-function #'nnatom--read-title + nil nnfeed-read-title-function) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) +(defvoo nnatom-read-description-function #'nnatom--read-description + nil nnfeed-read-description-function) + +(defun nnatom--read-article-or-group-authors (article-or-group) + "Return the authors of ARTICLE-OR-GROUP, or nil." + (when-let + ((a (string-trim-right + (mapconcat (lambda (author) + (let* ((name + (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email + (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (concat + (or (and name email (format "%s <%s>" name email)) + name email) + ", "))) + (dom-children + (dom-child-by-tag article-or-group 'authors))) + ", ")) + ((not (string-blank-p a)))) + a)) +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-authors + nil nnfeed-read-author-function) +(defvoo nnatom-read-group-author-function #'nnatom--read-article-or-group-authors + nil nnfeed-read-group-author-function) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) +(defvoo nnatom-read-subject-function #'nnatom--read-subject + nil nnfeed-read-subject-function) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) +(defvoo nnatom-read-id-function #'nnatom--read-id + nil nnfeed-read-id-function) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + nil nnfeed-read-publish-date-function) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-update-date-function #'nnatom--read-update + nil nnfeed-read-update-date-function) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((alt 0) (rel 0) (sel 0) (enc 0) (via 0) (aut 0)) + (mapcan + (lambda (link) + (when-let ((l (car-safe link))) + (or + (when-let (((eq l 'content)) + (src (dom-attr link 'src)) + (label (concat "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt))))) + `(((("text/plain") . ,(format "%s: %s\n" label src)) + (("text/html") . ,(format "[%s] " + src label))))) + (when-let (((or (eq l 'author) (eq l 'contributor))) + (name (dom-text (dom-child-by-tag link 'name))) + (name (if (string-blank-p name) + (concat "Author" + (and (< 1 (cl-incf aut)) + (format " %s" aut))) + name)) + (uri (dom-text (dom-child-by-tag link 'uri))) + ((not (string-blank-p uri)))) + `(((("text/plain") . ,(format "%s: %s\n" name uri)) + (("text/html") . ,(format "[%s] " + uri name))))) + (when-let (((eq l 'link)) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf rel)) + (format " %s" rel)))) + ("self" + (concat "More" + (and (< 1 (cl-incf sel)) + (format " %s" sel)))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf enc)) + (format " %s" enc)))) + ("via" + (concat "Source" + (and (< 1 (cl-incf via)) + (format " %s" via)))) + (_ (if-let + ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat + "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt)))))))) + (link (cdr (assq 'href attrs)))) + `(((("text/plain") . ,(format "%s: %s\n" label link)) + (("text/html") . ,(format "[%s] " + link label)))))))) + (dom-children (dom-child-by-tag article 'links))))) +(defvoo nnatom-read-links-function #'nnatom--read-links + nil nnfeed-read-links-function) + +(defun nnatom--read-part (part type) + (let* ((atypes '("html" "plain")) + (mtypes '(("xhtml" . "text/html") ("text" . "text/plain"))) + (xsuff (concat "[+/]xml\\(-\\(dtd\\|external-parsed-entity\\)\\)?\\'" + "\\|^text")) + (part (if (string= type "xhtml") + (with-temp-buffer + (dom-print (dom-child-by-tag part 'div) nil t) + (buffer-substring-no-properties + (point-min) (point-max))) + (dom-text part))) + (type (if (member type atypes) (concat "text/" type) type)) + (type (or (cdr (assoc type mtypes)) type))) + (unless (string-blank-p part) + `(,part (Content-Type . ,(or type (setq type "text/plain"))) + ,(and (not (string-match-p xsuff type)) + '(Content-Transfer-Encoding . "base64")))))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (cdr (assq 'type (dom-attributes summary)))) + (summary (nnatom--read-part summary stype)) + (content (dom-child-by-tag article 'content)) + (ctype (cdr (assq 'type (dom-attributes content)))) + (content (nnatom--read-part content ctype)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push summary parts)) + (push (append content '(links)) parts) + (or st (push summary parts)) + parts) + ((setq parts (or summary content)) + `(,(append (or summary content) '(links)))) + (t '((nil (Content-Type . "text/html") links)))))) +(defvoo nnatom-read-parts-function #'nnatom--read-parts + nil nnfeed-read-parts-function) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +(provide 'nnatom) + +;;; nnatom.el ends here diff --git a/lisp/gnus/nnfeed.el b/lisp/gnus/nnfeed.el new file mode 100644 index 00000000000..021fc6bbb32 --- /dev/null +++ b/lisp/gnus/nnfeed.el @@ -0,0 +1,683 @@ +;;; nnfeed.el --- Generic feed backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnfeed is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnfeed is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnfeed. If not, see . + +;;; Commentary: + +;; Generic Gnus backend (intended) for implementing backends for web +;; feeds (Atom, RSS). +;; +;; This backend is abstract - it doesn't implement a parser for any +;; specific web feed type, and so can't be used independently. +;; +;; Instead, it implements a generic parser, feed data storage and most +;; of the Gnus backend interface; the intended use for this backend is +;; to be a source of inheritance for backends supporting new web feed +;; types. +;; +;; To implement new backends, use `nnfeed-define-basic-backend-interface': +;; +;; ... +;; (require 'nnfeed) +;; +;; (nnoo-declare nnfoo nnfeed) +;; +;; (nnfeed-define-basic-backend-interface nnfoo) +;; ... +;; [ definitions of parsing functions, see the "Feed parser interface" +;; section for more information. ] +;; +;; (defvoo nnfoo-read-feed-function #'nnfoo--read-feed +;; nil nnfeed-read-feed-function) +;; ... +;; (gnus-declare-backend (symbol-name nnfeed-backend) 'address) +;; +;; (provide 'nnfoo) +;; +;;; Code: + +(eval-when-compile + (require 'subr-x)) + +(require 'gnus) +(require 'nnoo) + +(defgroup nnfeed nil + "Generic feed backend for Gnus." + :group 'gnus) + +(nnoo-declare nnfeed) + +(defvoo nnfeed-backend nil + "Symbol which identifies this backend.") + +(defvoo nnfeed-status-string nil + "Last status message reported by this backend.") + +;; This macro should be used to define inheriting backends. + +(defmacro nnfeed-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnfeed-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnfeed-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnfeed-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnfeed-servers) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnfeed-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnfeed-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnfeed-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnfeed-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnfeed-open-server server defs ',backend)) + (nnoo-import ,backend (nnfeed)))) + +;;;; Feed parser interface: + +;; The following set of server variables define a parser for a +;; specific web feed type. +;; An inheriting backend should define all of these functions (ideally). +;; Apart from this set of variables there is also +;; `nnfeed-print-content-function' which can be optionally be defined +;; by an inheriting backend to allow more advanced control over the +;; printing of articles. + +(defvoo nnfeed-read-feed-function #'ignore + "Function returning a Lisp object representing a feed (or part of it). + +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnfeed-read-group-function').") + +(defvoo nnfeed-read-group-function #'ignore + "Function returning a cons cell of a group and remaining data from a feed. + +The returned group can be represented by any Lisp object. +It should accept a single argument, a Lisp object representing a feed +\(as can be returned by this function or `nnfeed-read-feed-function').") + +(defvoo nnfeed-read-article-function #'ignore + "Function returning a cons cell of an article and remaining data from a group. + +The returned article can be represented by any Lisp object. +It should accept two arguments, a Lisp object representing a group +\(as can be returned by this function or `nnfeed-read-group-function'), +and a flag indicating whether the last article was not new or updated.") + +(defvoo nnfeed-read-title-function #'ignore + "Function returning the title of a group (a string). + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +(defvoo nnfeed-read-description-function #'ignore + "Function returning the description of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +(defvoo nnfeed-read-group-author-function #'ignore + "Function returning the author of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function')..") + +(defvoo nnfeed-read-id-function #'ignore + "Function returning the ID of an article. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-subject-function #'ignore + "Function returning the subject of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-publish-date-function #'ignore + "Function returning the publish date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-update-date-function #'ignore + "Function returning the update date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-author-function #'ignore + "Function returning the author of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-headers-function #'ignore + "Function returning an alist of article-wide MIME headers. + +Each element of this alist maps a MIME header (a symbol, +i.e. `Content-Type') to its value. As a special case, `:boundary' +maps to a string which will serve as the boundary between article +parts. This must be supplied if a custom boundary is used in a +multipart content type header. The default boundary is \"-_nnfeed_-\", +and is automatically modified to match the name of the back end. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; As mentioned in their docstrings, the last two parsing functions +;; can optionally return any Lisp representation they want, provided +;; an appropriate `nnfeed-print-content-function' is defined. + +(defvoo nnfeed-read-links-function #'ignore + "Function returning all links contained in an article. + +With the default `nnfeed-print-content-function', it should return a +list of links, where each link is an alist mapping MIME content types +to links formatted for display in a part of that type. Each content +type may also be a list of content types. +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-parts-function #'ignore + "Function returning an alist associating parts of an article to their headers. + +With the default `nnfeed-print-content-function', each part should be a +string. Otherwise, it can be any Lisp object. The \"headers\" of +each part should be a list where each element is either a cons of a +MIME header (a symbol, i.e. `Content-Type') and its value (a string), +or any other Lisp object. MIME headers will be printed, the rest will +be passed on to `nnfeed-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;;;; Feed data storage: + +;; This section defines the data types used to store feed data, and +;; All variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. + +(defvoo nnfeed-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnfeed-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnfeed-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnfeed--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnfeed-backend) + "nn") + (gnus-newsgroup-savable-name server)) + gnus-directory)) + +(defun nnfeed--read-server (server) + "Read SERVER's information storage." + (if-let ((f (nnfeed--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnfeed-servers)) + (nnheader-report nnfeed-backend "Can't read %s" server))) + +(defun nnfeed--write-server (server) + "Write SERVER's information to storage." + (if-let ((f (nnfeed--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n") + (prin1 s (current-buffer)) + (insert "\n") + t) + t) + (nnheader-report nnfeed-backend "Can't write %s" f))) + +(defun nnfeed--server-address (server) + "Return SERVER's real address." + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnfeed-backend 'address) + (cddr (gnus-server-to-method + (concat + (symbol-name nnfeed-backend) ":" + server))))) + server)) + server)) + +;; This is the function which actually uses the functions defined in +;; the last section. None of those functions are called after parsing +;; a feed. + +(defun nnfeed--parse-feed (feed &optional group) + "Parse FEED into a new or existing server. +Optionally, only parse GROUP." + (let* ((feed (nnfeed--server-address feed)) + (s (or (gethash feed nnfeed-servers) (nnfeed--read-server feed))) + (g (or (and (hash-table-p s) (gethash group s)) `[,group])) + (name group) ; (Maybe) fake name + (group (or (aref g 0) name)) ; Real name + data) + (when (setq data (funcall nnfeed-read-feed-function feed group) + s (or (gethash feed nnfeed-servers) + (make-hash-table :test 'equal))) + (while-let ((cg (or (and name `(,data)) + (funcall nnfeed-read-group-function data))) + (cg (prog1 (car cg) (setq data (cdr cg))))) + (let* ((name (funcall nnfeed-read-title-function cg)) ; Real name + (group (gethash name nnfeed-group-names name)) ; (Maybe) fake name + (info (gnus-get-info + (concat (symbol-name nnfeed-backend) "+" feed ":" group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnfeed-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnfeed-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnfeed-read-article-function cg stale)) + (article (prog1 (car article) (setq cg (cdr article))))) + (when-let ((id (funcall nnfeed-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnfeed-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnfeed-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (setq max (1+ max))))) + (publish (funcall nnfeed-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnfeed-read-author-function article) + group-author group) + ,(or (funcall nnfeed-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnfeed-read-links-function article) + ,(funcall nnfeed-read-parts-function article) + ,(funcall nnfeed-read-headers-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnfeed-servers)))) + +;;;; Gnus backend functions: + +;; The following two sections define a Gnus backend interface based on +;; the parsed data from the last section. +;; All server variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. +;; For more information about these functions see the "Back End +;; Interface" section of the Gnus manual. + +(defvoo nnfeed-group nil + "Name of the current group.") + +(defvoo nnfeed-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnfeed-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS HEADERS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers. +- HEADERS is an alist associating article-wide MIME headers to their value.") + +(defvoo nnfeed-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnfeed-group-article-min-num 1 + "Minimum article number for the current group.") + +(defvar nnfeed-date-format "%F %X" + "Format of displayed dates (see `format-time-string').") + +(nnoo-define-basics nnfeed) + +(defun nnfeed--current-server-no-prefix () + "Remove the \"+\" prefix from the current server." + (string-remove-prefix (concat (symbol-name nnfeed-backend) "+") + (nnoo-current-server nnfeed-backend))) + +(defun nnfeed--group-data (group &optional server) + "Get parsed data for GROUP from SERVER (or the current server if omitted)." + (let ((s (gethash server nnfeed-servers)) c) + (or (and (hash-table-p s) (gethash group s)) + (and (setq c (nnfeed--current-server-no-prefix)) + (setq s (gethash c nnfeed-servers)) + (hash-table-p s) (gethash group s)) + (catch :stop (maphash (lambda (n s) + (or (string= n server) + (string= n c) + (when-let (((hash-table-p s)) + (g (gethash group s))) + (throw :stop g)))) + nnfeed-servers))))) + +(defun nnfeed-retrieve-article (article group) + "Retrieve headers for ARTICLE from GROUP." + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string nnfeed-date-format (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnfeed-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + `[ nil ,nnfeed-group-article-ids ,nnfeed-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnfeed-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnfeed-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnfeed-backend "Group %s not found" (or group "")))) + +(deffoo nnfeed-open-server (server &optional defs backend) + (let ((backend (or backend 'nnfeed)) + (a (nnfeed--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (or (gethash server nnfeed-servers) + (nnfeed--read-server a))) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash group g nnfeed-group-names))) + s)) + (or s (file-writable-p (nnfeed--server-file a)) + (and (nnoo-close-server nnfeed-backend server) + (nnheader-report + nnfeed-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnfeed-request-close () + (maphash (lambda (server _) + (nnfeed--write-server + (nnfeed--server-address server))) + nnfeed-servers) + (setq nnfeed-servers (make-hash-table :test 'equal) + nnfeed-status-string nil) + t) + +;; The default content printing function, which should be suitable for +;; most inheriting backends. + +(defun nnfeed--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, with LINKS possibly added." + (let ((links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and content (format "%s\n\n" content)) + (mapconcat (lambda (link) + (cdr (assoc (cdr (assq 'Content-Type attributes)) link + (lambda (types type) + (if (stringp types) (string= types type) + (member type types)))))) + links))))) + +(defvoo nnfeed-print-content-function #'nnfeed--print-content + "Function returning a single piece of content for an article (a string). + +It should accept three arguments, a part and its attributes (as returned +by `nnfeed-read-parts-function'), and links (as returned by +`nnfeed-read-links-function').") + +(defun nnfeed--print-part (content headers mime links) + "Print part of an article using its CONTENT, HEADERS, and LINKS. +Only HEADERS of a type included in MIME are considered." + (concat + (mapconcat (lambda (header) + (when-let ((m (car-safe header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnfeed-print-content-function content headers links))) + +(deffoo nnfeed-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + (and (setq group nnfeed-group) + `[ nil ,nnfeed-group-article-ids + ,nnfeed-group-articles + ,nnfeed-group-article-max-num + ,nnfeed-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((links (aref a 5)) + (parts (aref a 6)) + (headers (aref a 7)) + (boundary (or (cdr (assq :boundary headers)) + (format "-_%s_-" nnfeed-backend))) + (multi (length> parts 1)) + (mime '( Content-Type Content-Disposition + Content-Transfer-Encoding))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnfeed-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + (if (assq 'MIME-Version headers) "" "MIME-Version: 1.0\n") + (mapconcat (lambda (header) + (unless (keywordp (car header)) + (format "%s: %s\n" (car header) (cdr header)))) + headers) + (if multi + (if (assq 'Content-Type headers) "" + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary)) + (prog1 (nnfeed--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (format "--%s\n%s\n" boundary + (nnfeed--print-part + (car part) (cdr part) mime links))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnfeed-backend "No such article"))) + +(deffoo nnfeed-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (if fast (nnfeed--group-data group server) + (setq server (nnfeed--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnfeed-group group + nnfeed-group-article-ids (aref g 1) + nnfeed-group-articles (aref g 2) + nnfeed-group-article-max-num (aref g 3) + nnfeed-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnfeed-group-article-ids) + nnfeed-group-article-min-num + nnfeed-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnfeed-backend "Group %s not found" group)))) + +(deffoo nnfeed-close-group (group &optional server) + (and (string= group nnfeed-group) + (setq nnfeed-group nil + nnfeed-group-article-ids (make-hash-table :test 'equal) + nnfeed-group-articles (make-hash-table :test 'eql) + nnfeed-group-article-max-num 0 + nnfeed-group-article-min-num 1)) + (setq server (or server (nnfeed--current-server-no-prefix))) + (nnfeed--write-server server)) + +(deffoo nnfeed-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnfeed--parse-feed + (or server (nnfeed--current-server-no-prefix)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnfeed-request-post (&optional _server) + (nnheader-report nnfeed-backend "%s is a read only backend" nnfeed-backend)) + +;;;; Optional back end functions: + +(deffoo nnfeed-retrieve-groups (_groups &optional server) + (nnfeed-request-list (or server (nnfeed--current-server-no-prefix))) + 'active) + +(deffoo nnfeed-request-type (_group &optional _article) + 'unknown) + +(deffoo nnfeed-request-group-description (group &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (nnfeed--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnfeed-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnfeed-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (s (or (gethash server nnfeed-servers) + (and ; Open the server to add it to `nnfeed-servers' + (save-match-data + (nnfeed-open-server + server + (cdr ; Get defs and backend. + (assoc server (cdr (assq 'nnfeed nnoo-state-alist)) + (lambda (car key) + (and (stringp car) + (string-match + (concat + "\\(\\(nn[[:alpha:]]+\\)\\+\\)?" + "\\(" (regexp-quote key) "\\)") + car) + (setq server car))))) + (prog1 (if (stringp (match-string 1 server)) + (intern (match-string 2 server)) + 'nnfeed) + (setq server (match-string 3 server)))) + (gethash server nnfeed-servers))))) + (g (or (nnfeed--group-data group server) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnfeed-group-names) + (remhash group s) + (and (string= group nnfeed-group) + (setq nnfeed-group new-name)) + t)) + +(provide 'nnfeed) + +;;; nnfeed.el ends here -- 2.43.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 09 06:08:59 2023 Received: (at 64202) by debbugs.gnu.org; 9 Dec 2023 11:08:59 +0000 Received: from localhost ([127.0.0.1]:46875 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rBvCb-00011e-5g for submit@debbugs.gnu.org; Sat, 09 Dec 2023 06:08:59 -0500 Received: from dsemy.com ([46.23.89.208]:22814) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rBvCX-00011O-Hw for 64202@debbugs.gnu.org; Sat, 09 Dec 2023 06:08:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=6IkpxF0QhGQiz cnlzi4BwoKXuSMmh788vQucX/Ntr6o=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=bOo7EBu1XXyUH1C/VQR84h9FY8nxnrUxJgG bw8Zqws4LODJX5HN0GmqSBoPf12rJVPQjmrv+p0iSmlUBhz8ckJJ6rQg2PCvP/2cOqDGRW 9Rmlza71eP+PXFgKBy7imuNGW7L99iM0IURBDd3cKTuHFBULVumTQDpdzCJ4lJOSm1F5sM vHe1TALTRSq5ucZ6AmPlUNHza1Pi+zD879DlTxaIS7N+a2mcmrBbhvwt7dD6yRlhdV09z/ C37OeLUypEfSZFEWkIq7AYlpV2PXJklf2aqfmrYrK6I0tArYKLh4H/IrfznuZWYehPLYTP vtHaQL9nx6eSJMFhI+z/88W4Rfg== Received: from coldharbour.local ( [147.235.213.125]) by dsemy.com (OpenSMTPD) with ESMTPSA id 62a60a07 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sat, 9 Dec 2023 12:08:33 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id 78f97520; Sat, 9 Dec 2023 11:03:33 +0000 (UTC) From: Daniel Semyonov To: 64202@debbugs.gnu.org Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <8734wcfynr.fsf@dsemy.com> (Daniel Semyonov's message of "Fri, 08 Dec 2023 18:26:48 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> Date: Sat, 09 Dec 2023 13:03:33 +0200 Message-ID: <87wmtnbptm.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: Eric Abrahamsen , Husain Alshehhi , bug-gnu-emacs@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain >>>>> Daniel Semyonov writes: > I just realized this patch still can't be merged though, NEWS and > the manual need updated. I'll do it on my phone tomorrow if I > have the time. Hopefully this can be merged soon finally though I've added a section to the manual documenting nnfeed (without going too far into implementation details, which are available in the comments and docstrings of "nnfeed.el"), and amended the the section documenting nnatom. I've also documented nnfeed in NEWS. I think this can be merged now. Updated patch attached. Thanks, Daniel --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch Content-Description: patch >From e5707f38277c123a5f13fbc6d55f36247aa85a33 Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnatom.el: * lisp/gnus/nnfeed.el: New file. * doc/misc/gnus.texi: * etc/NEWS: Document nnatom and nnfeed. --- doc/misc/gnus.texi | 74 +++++ etc/NEWS | 10 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 277 ++++++++++++++++++ lisp/gnus/nnfeed.el | 683 ++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 1045 insertions(+) create mode 100644 lisp/gnus/nnatom.el create mode 100644 lisp/gnus/nnfeed.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 586e4b94ba1..e31ab2de557 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -17250,6 +17251,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17494,6 +17496,41 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@pxref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds +as Gnus servers (with a single group), as you would with any other +method, by supplying the location of the feed as the server address. +Note, however, that the server address shouldn't be prefixed with +@url{http://} or @url{https://}. + +Features of @code{nnatom} include: + +@itemize @bullet + +@item +Server data is saved per-server in the @file{atom} sub-directory of +@file{gnus-directory}. + +@item +An article part is generated for both the summary and the content for +each entry in the feed. Content of all MIME types should be displayed +correctly through Gnus (as long as the feed specifies a MIME type), +even base64 encoded files (like images for example), which are +supported by Atom. + +@item +Article modification and publish dates are tracked, and articles are +updated if changed. + +@end itemize + @node Other Sources @section Other Sources @@ -29992,6 +30029,7 @@ Back End Interface * Writing New Back Ends:: Extending old back ends. * Hooking New Back Ends Into Gnus:: What has to be done on the Gnus end. * Mail-like Back Ends:: Some tips on mail back ends. +* Web Feed Back Ends:: Easily defining back ends for web feeds. @end menu @@ -30764,6 +30802,42 @@ Mail-like Back Ends ("some-group" (34 . 39))) @end example +@node Web Feed Back Ends +@subsection Web Feed Back Ends + +If you want to write a back end for a new type of web feed (RSS, +Atom), or some other type of feed, an ``abstract'' back end +(@code{nnfeed}) exists to enable the creation of such back ends with +minimal knowledge of the Gnus back end interface. + +@code{nnfeed} defines a generic parser, which uses functions stored in +server variables to parse information from a feed (@code{nnfeed} +itself doesn't actually define any such functions though). + +The data parsed from the feed is stored in server variables (and +stored per-feed in a sub-directory of @option{gnus-directory} whose name +corresponds to the name of the back end). + +A Gnus back end interface is also defined, which uses the data parsed +from the feed. + +Therefore, a new back end only needs to inherit from @code{nnfeed}, +define (fairly) generic parsing functions for the feed type and setup +the required server variables. + +@code{nnfeed} was originally created to support Atom Syndication +Format feeds (@pxref{Atom}), but is very generic (as of writing this, +no standard web feed exists which can meaningfully use all the +features supported): it supports multiple groups contained in a single +feed, it allows for situations when the entire feed can't (or +shouldn't) be read ahead of time and it allows for very advanced +customization of the actual printing of articles from parsed data +(while providing a powerful and reasonable default method). + +Further implementation details are available in the documentation +strings of the various @code{nnfeed-*} server variables and +the commentary and other comments of @file{nnfeed.el}. + @node Score File Syntax @subsection Score File Syntax diff --git a/etc/NEWS b/etc/NEWS index 03a40c11c5b..7bf73a657ab 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -919,6 +919,16 @@ This keyword enables the user to install packages using 'package-vc'. ** Gnus ++++ +*** New back end 'nnfeed'. +This allows back end developers to easily create new back ends for web +feeds, as inheriting back ends of 'nnfeed'. + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + *** The 'nnweb-type' option 'gmane' has been removed. The gmane.org website is, sadly, down since a number of years with no prospect of it coming back. Therefore, it is no longer valid to set diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index 6bf66233101..de5567d1fc0 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1348,6 +1348,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..e92a6f09baa --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,277 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format . +;; Other types of feeds may be supported by providing custom parsing +;; functions. + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'subr-x)) + +(require 'nnfeed) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'nnfeed) + +(nnoo-declare nnatom nnfeed) + +(nnfeed-define-basic-backend-interface nnatom) + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "\\`https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-name-absolute-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (when-let ((data (if (libxml-available-p) + (libxml-parse-xml-region + (point-min) (point-max)) + (car (xml-parse-region + (point-min) (point-max))))) + (auth (list 'authors))) + (when (eq (car data) 'top) + (setq data (assq 'feed data))) + (dom-add-child-before data auth) + (catch :stop ; Collect feed authors, stop at first entry. + (dolist (child (cdddr data)) + (let ((tag (car-safe child))) + (if (eq tag 'entry) + (throw :stop data) + (and (or (eq tag 'author) + (eq tag 'contributor)) + (dom-add-child-before auth child))))) + data))))))) +(defvoo nnatom-read-feed-function #'nnatom--read-feed + nil nnfeed-read-feed-function) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) +(defvoo nnatom-read-group-function #'nnatom--read-group + nil nnfeed-read-group-function) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) (setq data (cddr data))) + (while (and data (not (eq (car-safe (car data)) 'entry))) (pop data)) + (when-let ((article (car data)) + (auths (list 'authors)) (links (list 'links))) + (dom-add-child-before article links) + (dom-add-child-before article auths) + (dolist (child (cddddr article) `(,article . ,(cdr data))) + (pcase (car-safe child) ; Authors and links can appear + ((or 'author 'contributor) ; anywhere in the entry so we + (dom-add-child-before auths child) ; collect them all here to + (dom-add-child-before links child)) ; avoid looping over the + ((or 'link ; entry multiple times later. + (and 'content (guard (assq 'src (dom-attributes child))))) + (dom-add-child-before links child)))))) +(defvoo nnatom-read-article-function #'nnatom--read-article + nil nnfeed-read-article-function) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) +(defvoo nnatom-read-title-function #'nnatom--read-title + nil nnfeed-read-title-function) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) +(defvoo nnatom-read-description-function #'nnatom--read-description + nil nnfeed-read-description-function) + +(defun nnatom--read-article-or-group-authors (article-or-group) + "Return the authors of ARTICLE-OR-GROUP, or nil." + (when-let + ((a (string-trim-right + (mapconcat (lambda (author) + (let* ((name + (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email + (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (concat + (or (and name email (format "%s <%s>" name email)) + name email) + ", "))) + (dom-children + (dom-child-by-tag article-or-group 'authors))) + ", ")) + ((not (string-blank-p a)))) + a)) +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-authors + nil nnfeed-read-author-function) +(defvoo nnatom-read-group-author-function #'nnatom--read-article-or-group-authors + nil nnfeed-read-group-author-function) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) +(defvoo nnatom-read-subject-function #'nnatom--read-subject + nil nnfeed-read-subject-function) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) +(defvoo nnatom-read-id-function #'nnatom--read-id + nil nnfeed-read-id-function) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + nil nnfeed-read-publish-date-function) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-update-date-function #'nnatom--read-update + nil nnfeed-read-update-date-function) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((alt 0) (rel 0) (sel 0) (enc 0) (via 0) (aut 0)) + (mapcan + (lambda (link) + (when-let ((l (car-safe link))) + (or + (when-let (((eq l 'content)) + (src (dom-attr link 'src)) + (label (concat "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt))))) + `(((("text/plain") . ,(format "%s: %s\n" label src)) + (("text/html") . ,(format "[%s] " + src label))))) + (when-let (((or (eq l 'author) (eq l 'contributor))) + (name (dom-text (dom-child-by-tag link 'name))) + (name (if (string-blank-p name) + (concat "Author" + (and (< 1 (cl-incf aut)) + (format " %s" aut))) + name)) + (uri (dom-text (dom-child-by-tag link 'uri))) + ((not (string-blank-p uri)))) + `(((("text/plain") . ,(format "%s: %s\n" name uri)) + (("text/html") . ,(format "[%s] " + uri name))))) + (when-let (((eq l 'link)) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf rel)) + (format " %s" rel)))) + ("self" + (concat "More" + (and (< 1 (cl-incf sel)) + (format " %s" sel)))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf enc)) + (format " %s" enc)))) + ("via" + (concat "Source" + (and (< 1 (cl-incf via)) + (format " %s" via)))) + (_ (if-let + ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat + "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt)))))))) + (link (cdr (assq 'href attrs)))) + `(((("text/plain") . ,(format "%s: %s\n" label link)) + (("text/html") . ,(format "[%s] " + link label)))))))) + (dom-children (dom-child-by-tag article 'links))))) +(defvoo nnatom-read-links-function #'nnatom--read-links + nil nnfeed-read-links-function) + +(defun nnatom--read-part (part type) + (let* ((atypes '("html" "plain")) + (mtypes '(("xhtml" . "text/html") ("text" . "text/plain"))) + (xsuff (concat "[+/]xml\\(-\\(dtd\\|external-parsed-entity\\)\\)?\\'" + "\\|^text")) + (part (if (string= type "xhtml") + (with-temp-buffer + (dom-print (dom-child-by-tag part 'div) nil t) + (buffer-substring-no-properties + (point-min) (point-max))) + (dom-text part))) + (type (if (member type atypes) (concat "text/" type) type)) + (type (or (cdr (assoc type mtypes)) type))) + (unless (string-blank-p part) + `(,part (Content-Type . ,(or type (setq type "text/plain"))) + ,(and (not (string-match-p xsuff type)) + '(Content-Transfer-Encoding . "base64")))))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (cdr (assq 'type (dom-attributes summary)))) + (summary (nnatom--read-part summary stype)) + (content (dom-child-by-tag article 'content)) + (ctype (cdr (assq 'type (dom-attributes content)))) + (content (nnatom--read-part content ctype)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push summary parts)) + (push (append content '(links)) parts) + (or st (push summary parts)) + parts) + ((setq parts (or summary content)) + `(,(append (or summary content) '(links)))) + (t '((nil (Content-Type . "text/html") links)))))) +(defvoo nnatom-read-parts-function #'nnatom--read-parts + nil nnfeed-read-parts-function) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +(provide 'nnatom) + +;;; nnatom.el ends here diff --git a/lisp/gnus/nnfeed.el b/lisp/gnus/nnfeed.el new file mode 100644 index 00000000000..021fc6bbb32 --- /dev/null +++ b/lisp/gnus/nnfeed.el @@ -0,0 +1,683 @@ +;;; nnfeed.el --- Generic feed backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnfeed is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnfeed is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnfeed. If not, see . + +;;; Commentary: + +;; Generic Gnus backend (intended) for implementing backends for web +;; feeds (Atom, RSS). +;; +;; This backend is abstract - it doesn't implement a parser for any +;; specific web feed type, and so can't be used independently. +;; +;; Instead, it implements a generic parser, feed data storage and most +;; of the Gnus backend interface; the intended use for this backend is +;; to be a source of inheritance for backends supporting new web feed +;; types. +;; +;; To implement new backends, use `nnfeed-define-basic-backend-interface': +;; +;; ... +;; (require 'nnfeed) +;; +;; (nnoo-declare nnfoo nnfeed) +;; +;; (nnfeed-define-basic-backend-interface nnfoo) +;; ... +;; [ definitions of parsing functions, see the "Feed parser interface" +;; section for more information. ] +;; +;; (defvoo nnfoo-read-feed-function #'nnfoo--read-feed +;; nil nnfeed-read-feed-function) +;; ... +;; (gnus-declare-backend (symbol-name nnfeed-backend) 'address) +;; +;; (provide 'nnfoo) +;; +;;; Code: + +(eval-when-compile + (require 'subr-x)) + +(require 'gnus) +(require 'nnoo) + +(defgroup nnfeed nil + "Generic feed backend for Gnus." + :group 'gnus) + +(nnoo-declare nnfeed) + +(defvoo nnfeed-backend nil + "Symbol which identifies this backend.") + +(defvoo nnfeed-status-string nil + "Last status message reported by this backend.") + +;; This macro should be used to define inheriting backends. + +(defmacro nnfeed-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnfeed-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnfeed-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnfeed-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnfeed-servers) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnfeed-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnfeed-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnfeed-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnfeed-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnfeed-open-server server defs ',backend)) + (nnoo-import ,backend (nnfeed)))) + +;;;; Feed parser interface: + +;; The following set of server variables define a parser for a +;; specific web feed type. +;; An inheriting backend should define all of these functions (ideally). +;; Apart from this set of variables there is also +;; `nnfeed-print-content-function' which can be optionally be defined +;; by an inheriting backend to allow more advanced control over the +;; printing of articles. + +(defvoo nnfeed-read-feed-function #'ignore + "Function returning a Lisp object representing a feed (or part of it). + +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnfeed-read-group-function').") + +(defvoo nnfeed-read-group-function #'ignore + "Function returning a cons cell of a group and remaining data from a feed. + +The returned group can be represented by any Lisp object. +It should accept a single argument, a Lisp object representing a feed +\(as can be returned by this function or `nnfeed-read-feed-function').") + +(defvoo nnfeed-read-article-function #'ignore + "Function returning a cons cell of an article and remaining data from a group. + +The returned article can be represented by any Lisp object. +It should accept two arguments, a Lisp object representing a group +\(as can be returned by this function or `nnfeed-read-group-function'), +and a flag indicating whether the last article was not new or updated.") + +(defvoo nnfeed-read-title-function #'ignore + "Function returning the title of a group (a string). + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +(defvoo nnfeed-read-description-function #'ignore + "Function returning the description of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +(defvoo nnfeed-read-group-author-function #'ignore + "Function returning the author of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function')..") + +(defvoo nnfeed-read-id-function #'ignore + "Function returning the ID of an article. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-subject-function #'ignore + "Function returning the subject of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-publish-date-function #'ignore + "Function returning the publish date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-update-date-function #'ignore + "Function returning the update date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-author-function #'ignore + "Function returning the author of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-headers-function #'ignore + "Function returning an alist of article-wide MIME headers. + +Each element of this alist maps a MIME header (a symbol, +i.e. `Content-Type') to its value. As a special case, `:boundary' +maps to a string which will serve as the boundary between article +parts. This must be supplied if a custom boundary is used in a +multipart content type header. The default boundary is \"-_nnfeed_-\", +and is automatically modified to match the name of the back end. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; As mentioned in their docstrings, the last two parsing functions +;; can optionally return any Lisp representation they want, provided +;; an appropriate `nnfeed-print-content-function' is defined. + +(defvoo nnfeed-read-links-function #'ignore + "Function returning all links contained in an article. + +With the default `nnfeed-print-content-function', it should return a +list of links, where each link is an alist mapping MIME content types +to links formatted for display in a part of that type. Each content +type may also be a list of content types. +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-parts-function #'ignore + "Function returning an alist associating parts of an article to their headers. + +With the default `nnfeed-print-content-function', each part should be a +string. Otherwise, it can be any Lisp object. The \"headers\" of +each part should be a list where each element is either a cons of a +MIME header (a symbol, i.e. `Content-Type') and its value (a string), +or any other Lisp object. MIME headers will be printed, the rest will +be passed on to `nnfeed-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;;;; Feed data storage: + +;; This section defines the data types used to store feed data, and +;; All variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. + +(defvoo nnfeed-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnfeed-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnfeed-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnfeed--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnfeed-backend) + "nn") + (gnus-newsgroup-savable-name server)) + gnus-directory)) + +(defun nnfeed--read-server (server) + "Read SERVER's information storage." + (if-let ((f (nnfeed--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnfeed-servers)) + (nnheader-report nnfeed-backend "Can't read %s" server))) + +(defun nnfeed--write-server (server) + "Write SERVER's information to storage." + (if-let ((f (nnfeed--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n") + (prin1 s (current-buffer)) + (insert "\n") + t) + t) + (nnheader-report nnfeed-backend "Can't write %s" f))) + +(defun nnfeed--server-address (server) + "Return SERVER's real address." + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnfeed-backend 'address) + (cddr (gnus-server-to-method + (concat + (symbol-name nnfeed-backend) ":" + server))))) + server)) + server)) + +;; This is the function which actually uses the functions defined in +;; the last section. None of those functions are called after parsing +;; a feed. + +(defun nnfeed--parse-feed (feed &optional group) + "Parse FEED into a new or existing server. +Optionally, only parse GROUP." + (let* ((feed (nnfeed--server-address feed)) + (s (or (gethash feed nnfeed-servers) (nnfeed--read-server feed))) + (g (or (and (hash-table-p s) (gethash group s)) `[,group])) + (name group) ; (Maybe) fake name + (group (or (aref g 0) name)) ; Real name + data) + (when (setq data (funcall nnfeed-read-feed-function feed group) + s (or (gethash feed nnfeed-servers) + (make-hash-table :test 'equal))) + (while-let ((cg (or (and name `(,data)) + (funcall nnfeed-read-group-function data))) + (cg (prog1 (car cg) (setq data (cdr cg))))) + (let* ((name (funcall nnfeed-read-title-function cg)) ; Real name + (group (gethash name nnfeed-group-names name)) ; (Maybe) fake name + (info (gnus-get-info + (concat (symbol-name nnfeed-backend) "+" feed ":" group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnfeed-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnfeed-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnfeed-read-article-function cg stale)) + (article (prog1 (car article) (setq cg (cdr article))))) + (when-let ((id (funcall nnfeed-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnfeed-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnfeed-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (setq max (1+ max))))) + (publish (funcall nnfeed-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnfeed-read-author-function article) + group-author group) + ,(or (funcall nnfeed-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnfeed-read-links-function article) + ,(funcall nnfeed-read-parts-function article) + ,(funcall nnfeed-read-headers-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnfeed-servers)))) + +;;;; Gnus backend functions: + +;; The following two sections define a Gnus backend interface based on +;; the parsed data from the last section. +;; All server variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. +;; For more information about these functions see the "Back End +;; Interface" section of the Gnus manual. + +(defvoo nnfeed-group nil + "Name of the current group.") + +(defvoo nnfeed-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnfeed-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS HEADERS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers. +- HEADERS is an alist associating article-wide MIME headers to their value.") + +(defvoo nnfeed-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnfeed-group-article-min-num 1 + "Minimum article number for the current group.") + +(defvar nnfeed-date-format "%F %X" + "Format of displayed dates (see `format-time-string').") + +(nnoo-define-basics nnfeed) + +(defun nnfeed--current-server-no-prefix () + "Remove the \"+\" prefix from the current server." + (string-remove-prefix (concat (symbol-name nnfeed-backend) "+") + (nnoo-current-server nnfeed-backend))) + +(defun nnfeed--group-data (group &optional server) + "Get parsed data for GROUP from SERVER (or the current server if omitted)." + (let ((s (gethash server nnfeed-servers)) c) + (or (and (hash-table-p s) (gethash group s)) + (and (setq c (nnfeed--current-server-no-prefix)) + (setq s (gethash c nnfeed-servers)) + (hash-table-p s) (gethash group s)) + (catch :stop (maphash (lambda (n s) + (or (string= n server) + (string= n c) + (when-let (((hash-table-p s)) + (g (gethash group s))) + (throw :stop g)))) + nnfeed-servers))))) + +(defun nnfeed-retrieve-article (article group) + "Retrieve headers for ARTICLE from GROUP." + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string nnfeed-date-format (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnfeed-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + `[ nil ,nnfeed-group-article-ids ,nnfeed-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnfeed-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnfeed-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnfeed-backend "Group %s not found" (or group "")))) + +(deffoo nnfeed-open-server (server &optional defs backend) + (let ((backend (or backend 'nnfeed)) + (a (nnfeed--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (or (gethash server nnfeed-servers) + (nnfeed--read-server a))) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash group g nnfeed-group-names))) + s)) + (or s (file-writable-p (nnfeed--server-file a)) + (and (nnoo-close-server nnfeed-backend server) + (nnheader-report + nnfeed-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnfeed-request-close () + (maphash (lambda (server _) + (nnfeed--write-server + (nnfeed--server-address server))) + nnfeed-servers) + (setq nnfeed-servers (make-hash-table :test 'equal) + nnfeed-status-string nil) + t) + +;; The default content printing function, which should be suitable for +;; most inheriting backends. + +(defun nnfeed--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, with LINKS possibly added." + (let ((links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and content (format "%s\n\n" content)) + (mapconcat (lambda (link) + (cdr (assoc (cdr (assq 'Content-Type attributes)) link + (lambda (types type) + (if (stringp types) (string= types type) + (member type types)))))) + links))))) + +(defvoo nnfeed-print-content-function #'nnfeed--print-content + "Function returning a single piece of content for an article (a string). + +It should accept three arguments, a part and its attributes (as returned +by `nnfeed-read-parts-function'), and links (as returned by +`nnfeed-read-links-function').") + +(defun nnfeed--print-part (content headers mime links) + "Print part of an article using its CONTENT, HEADERS, and LINKS. +Only HEADERS of a type included in MIME are considered." + (concat + (mapconcat (lambda (header) + (when-let ((m (car-safe header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnfeed-print-content-function content headers links))) + +(deffoo nnfeed-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + (and (setq group nnfeed-group) + `[ nil ,nnfeed-group-article-ids + ,nnfeed-group-articles + ,nnfeed-group-article-max-num + ,nnfeed-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((links (aref a 5)) + (parts (aref a 6)) + (headers (aref a 7)) + (boundary (or (cdr (assq :boundary headers)) + (format "-_%s_-" nnfeed-backend))) + (multi (length> parts 1)) + (mime '( Content-Type Content-Disposition + Content-Transfer-Encoding))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnfeed-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + (if (assq 'MIME-Version headers) "" "MIME-Version: 1.0\n") + (mapconcat (lambda (header) + (unless (keywordp (car header)) + (format "%s: %s\n" (car header) (cdr header)))) + headers) + (if multi + (if (assq 'Content-Type headers) "" + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary)) + (prog1 (nnfeed--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (format "--%s\n%s\n" boundary + (nnfeed--print-part + (car part) (cdr part) mime links))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnfeed-backend "No such article"))) + +(deffoo nnfeed-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (if fast (nnfeed--group-data group server) + (setq server (nnfeed--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnfeed-group group + nnfeed-group-article-ids (aref g 1) + nnfeed-group-articles (aref g 2) + nnfeed-group-article-max-num (aref g 3) + nnfeed-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnfeed-group-article-ids) + nnfeed-group-article-min-num + nnfeed-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnfeed-backend "Group %s not found" group)))) + +(deffoo nnfeed-close-group (group &optional server) + (and (string= group nnfeed-group) + (setq nnfeed-group nil + nnfeed-group-article-ids (make-hash-table :test 'equal) + nnfeed-group-articles (make-hash-table :test 'eql) + nnfeed-group-article-max-num 0 + nnfeed-group-article-min-num 1)) + (setq server (or server (nnfeed--current-server-no-prefix))) + (nnfeed--write-server server)) + +(deffoo nnfeed-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnfeed--parse-feed + (or server (nnfeed--current-server-no-prefix)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnfeed-request-post (&optional _server) + (nnheader-report nnfeed-backend "%s is a read only backend" nnfeed-backend)) + +;;;; Optional back end functions: + +(deffoo nnfeed-retrieve-groups (_groups &optional server) + (nnfeed-request-list (or server (nnfeed--current-server-no-prefix))) + 'active) + +(deffoo nnfeed-request-type (_group &optional _article) + 'unknown) + +(deffoo nnfeed-request-group-description (group &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (nnfeed--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnfeed-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnfeed-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (s (or (gethash server nnfeed-servers) + (and ; Open the server to add it to `nnfeed-servers' + (save-match-data + (nnfeed-open-server + server + (cdr ; Get defs and backend. + (assoc server (cdr (assq 'nnfeed nnoo-state-alist)) + (lambda (car key) + (and (stringp car) + (string-match + (concat + "\\(\\(nn[[:alpha:]]+\\)\\+\\)?" + "\\(" (regexp-quote key) "\\)") + car) + (setq server car))))) + (prog1 (if (stringp (match-string 1 server)) + (intern (match-string 2 server)) + 'nnfeed) + (setq server (match-string 3 server)))) + (gethash server nnfeed-servers))))) + (g (or (nnfeed--group-data group server) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnfeed-group-names) + (remhash group s) + (and (string= group nnfeed-group) + (setq nnfeed-group new-name)) + t)) + +(provide 'nnfeed) + +;;; nnfeed.el ends here -- 2.43.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 09 09:04:47 2023 Received: (at submit) by debbugs.gnu.org; 9 Dec 2023 14:04:47 +0000 Received: from localhost ([127.0.0.1]:47034 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rBxwi-0000aQ-Qz for submit@debbugs.gnu.org; Sat, 09 Dec 2023 09:04:47 -0500 Received: from lists.gnu.org ([2001:470:142::17]:50996) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rBxwe-0000Zz-Eu for submit@debbugs.gnu.org; Sat, 09 Dec 2023 09:04:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rBxwJ-0006XU-RP for bug-gnu-emacs@gnu.org; Sat, 09 Dec 2023 09:04:19 -0500 Received: from dsemy.com ([46.23.89.208]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rBxwE-0008Nb-LG for bug-gnu-emacs@gnu.org; Sat, 09 Dec 2023 09:04:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=Gh66jwpxqDGuu ajZZ0pb/bxHAm5KqLheDgOSLLbf3/4=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=eMJHiAWsuOuGvCiL9ptSkXpxEP4QOrjR3oi 3z+X5Jh6NQoP6BMAD7OenE8EpLMTOus3dGjsRzAV2AfjTPT86DbhudaunTa5rEmIXYHj6P 9NzNo1rAgcw81d/rYZAELg+w0TzmDlhDpHRe4skHablMxHbqEK3DjugGPEXirYnmB1mpio OhGaYlpFIBc3Zy/CgTCfOAm2yit2uIP1oVuvNWb41E11GcpB7AhWcIdivUL4uaT0p0TEgX J1fyIFEfNnuk0IPIBBwmXiuqoMu07HA2dbaQTAtmaceVad1X+xvfc6RRT1hd14wPkCxfTp jEYde88TWN4U57X6S+ZeqmasgJg== Received: from coldharbour.local ( [147.235.213.125]) by dsemy.com (OpenSMTPD) with ESMTPSA id e17901c0 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sat, 9 Dec 2023 15:04:11 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id 320a57ab; Sat, 9 Dec 2023 13:59:12 +0000 (UTC) From: Daniel Semyonov To: 64202@debbugs.gnu.org Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87wmtnbptm.fsf@dsemy.com> (Daniel Semyonov's message of "Sat, 09 Dec 2023 13:03:33 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> Date: Sat, 09 Dec 2023 15:59:12 +0200 Message-ID: <87zfyj8ojz.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=46.23.89.208; envelope-from=daniel@dsemy.com; helo=dsemy.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_FILL_THIS_FORM_SHORT=0.01, T_SCC_BODY_TEXT_LINE=-0.01, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit Cc: Eric Abrahamsen , Husain Alshehhi , bug-gnu-emacs@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) --=-=-= Content-Type: text/plain >>>>> Daniel Semyonov writes: > I think this can be merged now. Updated patch attached. Here's another version which fixes a bunch of small mistakes and clarifies some information in the manual, docstrings and comments. --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch Content-Description: patch >From 05bc03d95cb642066eb027edc5cfc4c1b33bdffd Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnfeed.el: New file implementing an abstract web feed back end. * lisp/gnus/nnatom.el: New file implementing a back end for Atom feeds. * doc/misc/gnus.texi (Browsing the Web, Back End Interface): * etc/NEWS (Gnus): Document nnatom and nnfeed. --- doc/misc/gnus.texi | 73 +++++ etc/NEWS | 10 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 276 ++++++++++++++++++ lisp/gnus/nnfeed.el | 690 ++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 1050 insertions(+) create mode 100644 lisp/gnus/nnatom.el create mode 100644 lisp/gnus/nnfeed.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 586e4b94ba1..4eb283720de 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -975,6 +976,7 @@ Top * Writing New Back Ends:: Extending old back ends. * Hooking New Back Ends Into Gnus:: What has to be done on the Gnus end. * Mail-like Back Ends:: Some tips on mail back ends. +* Web Feed Back Ends:: Easily defining back ends for web feeds. Various File Formats @@ -17250,6 +17252,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17494,6 +17497,40 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@pxref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds +as Gnus servers (with a single group), as you would with any other +method, by supplying the location of the feed as the server address. +Note, however, that the server address shouldn't be prefixed with +@url{http://} or @url{https://}. + +Features of @code{nnatom} include: + +@itemize @bullet + +@item +Server data is saved per-server in the @file{atom} sub-directory of +@file{gnus-directory}. + +@item +An article part is generated for both the summary and the content for +each entry in the feed. Content of all MIME types should be displayed +correctly through Gnus (as long as they are supported and the feed +specifies a MIME type). + +@item +Article modification and publish dates are tracked, and articles are +updated if changed. + +@end itemize + @node Other Sources @section Other Sources @@ -29992,6 +30029,7 @@ Back End Interface * Writing New Back Ends:: Extending old back ends. * Hooking New Back Ends Into Gnus:: What has to be done on the Gnus end. * Mail-like Back Ends:: Some tips on mail back ends. +* Web Feed Back Ends:: Easily defining back ends for web feeds. @end menu @@ -30764,6 +30802,41 @@ Mail-like Back Ends ("some-group" (34 . 39))) @end example +@node Web Feed Back Ends +@subsection Web Feed Back Ends + +If you want to write a back end for a new type of web feed (RSS, +Atom), or some other type of feed, an ``abstract'' back end +(@code{nnfeed}) exists to enable the creation of such back ends with +minimal knowledge of Gnus. + +@code{nnfeed} defines a generic parser, which uses functions stored in +server variables to parse information from a feed (@code{nnfeed} +itself doesn't actually define any such functions though). + +The data parsed from the feed is stored in server variables (and +stored per-feed in a sub-directory of @option{gnus-directory} whose name +corresponds to the name of the back end). + +A Gnus back end interface is also defined, which uses the data parsed +from the feed. + +Therefore, a new back end only needs to inherit from @code{nnfeed}, +define (fairly) generic parsing functions for the feed type and setup +the required server variables. + +@code{nnfeed} was originally created to support Atom Syndication +Format feeds (@pxref{Atom}), but is very generic (as of writing this, +no standard web feed exists which can meaningfully use all the +features supported): it supports multiple groups contained in a single +feed, it allows for situations when the entire feed can't (or +shouldn't) be read ahead of time and it allows for very advanced +customization of the actual printing of articles from parsed data +(while providing a reasonably powerful default method). + +Further implementation details are available in the documentation +strings of the various @code{nnfeed-*} server variables and +the commentary and other comments of @file{nnfeed.el}. @node Score File Syntax @subsection Score File Syntax diff --git a/etc/NEWS b/etc/NEWS index 03a40c11c5b..7bf73a657ab 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -919,6 +919,16 @@ This keyword enables the user to install packages using 'package-vc'. ** Gnus ++++ +*** New back end 'nnfeed'. +This allows back end developers to easily create new back ends for web +feeds, as inheriting back ends of 'nnfeed'. + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + *** The 'nnweb-type' option 'gmane' has been removed. The gmane.org website is, sadly, down since a number of years with no prospect of it coming back. Therefore, it is no longer valid to set diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index 6bf66233101..de5567d1fc0 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1348,6 +1348,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..2fcf59c1441 --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,276 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format . + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'subr-x)) + +(require 'nnfeed) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'nnfeed) + +(nnoo-declare nnatom nnfeed) + +(nnfeed-define-basic-backend-interface nnatom) + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "\\`https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-name-absolute-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (when-let ((data (if (libxml-available-p) + (libxml-parse-xml-region + (point-min) (point-max)) + (car (xml-parse-region + (point-min) (point-max))))) + (auth (list 'authors))) + (when (eq (car data) 'top) + (setq data (assq 'feed data))) + (dom-add-child-before data auth) + (catch :stop ; Collect feed authors, stop at first entry. + (dolist (child (cdddr data)) + (let ((tag (car-safe child))) + (if (eq tag 'entry) + (throw :stop data) + (and (or (eq tag 'author) + (eq tag 'contributor)) + (dom-add-child-before auth child))))) + data))))))) +(defvoo nnatom-read-feed-function #'nnatom--read-feed + nil nnfeed-read-feed-function) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) +(defvoo nnatom-read-group-function #'nnatom--read-group + nil nnfeed-read-group-function) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) (setq data (cddr data))) + (while (and data (not (eq (car-safe (car data)) 'entry))) (pop data)) + (when-let ((article (car data)) + (auths (list 'authors)) (links (list 'links))) + (dom-add-child-before article links) + (dom-add-child-before article auths) + (dolist (child (cddddr article) `(,article . ,(cdr data))) + (pcase (car-safe child) ; Authors and links can appear + ((or 'author 'contributor) ; anywhere in the entry so we + (dom-add-child-before auths child) ; collect them all here to + (dom-add-child-before links child)) ; avoid looping over the + ((or 'link ; entry multiple times later. + (and 'content (guard (assq 'src (dom-attributes child))))) + (dom-add-child-before links child)))))) +(defvoo nnatom-read-article-function #'nnatom--read-article + nil nnfeed-read-article-function) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) +(defvoo nnatom-read-title-function #'nnatom--read-title + nil nnfeed-read-title-function) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) +(defvoo nnatom-read-description-function #'nnatom--read-description + nil nnfeed-read-description-function) + +(defun nnatom--read-article-or-group-authors (article-or-group) + "Return the authors of ARTICLE-OR-GROUP, or nil." + (when-let + ((a (string-trim-right + (mapconcat (lambda (author) + (let* ((name + (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email + (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (concat + (or (and name email (format "%s <%s>" name email)) + name email) + ", "))) + (dom-children + (dom-child-by-tag article-or-group 'authors))) + ", ")) + ((not (string-blank-p a)))) + a)) +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-authors + nil nnfeed-read-author-function) +(defvoo nnatom-read-group-author-function + #'nnatom--read-article-or-group-authors + nil nnfeed-read-group-author-function) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) +(defvoo nnatom-read-subject-function #'nnatom--read-subject + nil nnfeed-read-subject-function) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) +(defvoo nnatom-read-id-function #'nnatom--read-id + nil nnfeed-read-id-function) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + nil nnfeed-read-publish-date-function) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-update-date-function #'nnatom--read-update + nil nnfeed-read-update-date-function) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((alt 0) (rel 0) (sel 0) (enc 0) (via 0) (aut 0)) + (mapcan + (lambda (link) + (when-let ((l (car-safe link))) + (or + (when-let (((eq l 'content)) + (src (dom-attr link 'src)) + (label (concat "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt))))) + `(((("text/plain") . ,(format "%s: %s\n" label src)) + (("text/html") . ,(format "[%s] " + src label))))) + (when-let (((or (eq l 'author) (eq l 'contributor))) + (name (dom-text (dom-child-by-tag link 'name))) + (name (if (string-blank-p name) + (concat "Author" + (and (< 1 (cl-incf aut)) + (format " %s" aut))) + name)) + (uri (dom-text (dom-child-by-tag link 'uri))) + ((not (string-blank-p uri)))) + `(((("text/plain") . ,(format "%s: %s\n" name uri)) + (("text/html") . ,(format "[%s] " + uri name))))) + (when-let (((eq l 'link)) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf rel)) + (format " %s" rel)))) + ("self" + (concat "More" + (and (< 1 (cl-incf sel)) + (format " %s" sel)))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf enc)) + (format " %s" enc)))) + ("via" + (concat "Source" + (and (< 1 (cl-incf via)) + (format " %s" via)))) + (_ (if-let + ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat + "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt)))))))) + (link (cdr (assq 'href attrs)))) + `(((("text/plain") . ,(format "%s: %s\n" label link)) + (("text/html") . ,(format "[%s] " + link label)))))))) + (dom-children (dom-child-by-tag article 'links))))) +(defvoo nnatom-read-links-function #'nnatom--read-links + nil nnfeed-read-links-function) + +(defun nnatom--read-part (part type) + (let* ((atypes '("html" "plain")) + (mtypes '(("xhtml" . "text/html") ("text" . "text/plain"))) + (xsuff (concat "[+/]xml\\(-\\(dtd\\|external-parsed-entity\\)\\)?\\'" + "\\|^text")) + (part (if (string= type "xhtml") + (with-temp-buffer + (dom-print (dom-child-by-tag part 'div) nil t) + (buffer-substring-no-properties + (point-min) (point-max))) + (dom-text part))) + (type (if (member type atypes) (concat "text/" type) type)) + (type (or (cdr (assoc type mtypes)) type))) + (unless (string-blank-p part) + `(,part (Content-Type . ,(or type (setq type "text/plain"))) + ,(and (not (string-match-p xsuff type)) + '(Content-Transfer-Encoding . "base64")))))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (cdr (assq 'type (dom-attributes summary)))) + (summary (nnatom--read-part summary stype)) + (content (dom-child-by-tag article 'content)) + (ctype (cdr (assq 'type (dom-attributes content)))) + (content (nnatom--read-part content ctype)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push summary parts)) + (push (append content '(links)) parts) + (or st (push summary parts)) + parts) + ((setq parts (or summary content)) + `(,(append (or summary content) '(links)))) + (t '((nil (Content-Type . "text/html") links)))))) +(defvoo nnatom-read-parts-function #'nnatom--read-parts + nil nnfeed-read-parts-function) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +(provide 'nnatom) + +;;; nnatom.el ends here diff --git a/lisp/gnus/nnfeed.el b/lisp/gnus/nnfeed.el new file mode 100644 index 00000000000..56a447ec1ec --- /dev/null +++ b/lisp/gnus/nnfeed.el @@ -0,0 +1,690 @@ +;;; nnfeed.el --- Generic feed backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnfeed is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnfeed is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnfeed. If not, see . + +;;; Commentary: + +;; Generic Gnus backend (intended) for implementing backends for web +;; feeds (Atom, RSS). +;; +;; This backend is abstract - it doesn't implement a parser for any +;; specific web feed type, and so can't be used independently. +;; +;; Instead, it implements a generic parser, feed data storage and most +;; of the Gnus backend interface; the intended use for this backend is +;; to be a source of inheritance for backends supporting new web feed +;; types. +;; +;; To implement new backends, use `nnfeed-define-basic-backend-interface': +;; +;; ... +;; (require 'nnfeed) +;; +;; (nnoo-declare nnfoo nnfeed) +;; +;; (nnfeed-define-basic-backend-interface nnfoo) +;; ... +;; [ definitions of parsing functions, see the "Feed parser interface" +;; section for more information. ] +;; +;; (defvoo nnfoo-read-feed-function #'nnfoo--read-feed +;; nil nnfeed-read-feed-function) +;; ... +;; (gnus-declare-backend (symbol-name nnfeed-backend) 'address) +;; +;; (provide 'nnfoo) +;; +;;; Code: + +(eval-when-compile + (require 'subr-x)) + +(require 'gnus) +(require 'nnoo) + +(defgroup nnfeed nil + "Generic feed backend for Gnus." + :group 'gnus) + +(nnoo-declare nnfeed) + +(defvoo nnfeed-backend nil + "Symbol which identifies this backend.") + +(defvoo nnfeed-status-string nil + "Last status message reported by this backend.") + +;; This macro should be used to define inheriting backends. + +(defmacro nnfeed-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnfeed-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnfeed-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnfeed-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnfeed-servers) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnfeed-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnfeed-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnfeed-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnfeed-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnfeed-open-server server defs ',backend)) + (nnoo-import ,backend (nnfeed)))) + +;;;; Feed parser interface: + +;; The following set of server variables define a parser for a +;; specific web feed type. +;; An inheriting backend doesn't necessarily have to define all of +;; these functions (see the comments below for more information). +;; Apart from this set of variables there is also +;; `nnfeed-print-content-function' which can optionally be defined +;; by an inheriting backend to allow more advanced control over the +;; printing of articles. + +(defvoo nnfeed-read-feed-function #'ignore + "Function returning a Lisp object representing a feed (or part of it). + +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnfeed-read-group-function').") + +(defvoo nnfeed-read-group-function #'ignore + "Function returning a cons cell of a group and remaining data from a feed. + +The returned group can be represented by any Lisp object. +It should accept a single argument, a Lisp object representing a feed +\(as can be returned by this function or `nnfeed-read-feed-function').") + +(defvoo nnfeed-read-article-function #'ignore + "Function returning a cons cell of an article and remaining data from a group. + +The returned article can be represented by any Lisp object. +It should accept two arguments, a Lisp object representing a group +\(as can be returned by this function or `nnfeed-read-group-function'), +and a flag indicating whether the last article was not new or updated.") + +(defvoo nnfeed-read-title-function #'ignore + "Function returning the title of a group (a string). + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +;; Optional. +(defvoo nnfeed-read-description-function #'ignore + "Function returning the description of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +;; Either this function or `nnfeed-read-author-function' is required. +(defvoo nnfeed-read-group-author-function #'ignore + "Function returning the author of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function')..") + +(defvoo nnfeed-read-id-function #'ignore + "Function returning the ID of an article. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-subject-function #'ignore + "Function returning the subject of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-update-date-function' is required. +(defvoo nnfeed-read-publish-date-function #'ignore + "Function returning the publish date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-publish-date-function' is required. +(defvoo nnfeed-read-update-date-function #'ignore + "Function returning the update date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-group-author-function' is required. +(defvoo nnfeed-read-author-function #'ignore + "Function returning the author of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-headers-function #'ignore + "Function returning an alist of article-wide MIME headers. + +Each element of this alist maps a MIME header (a symbol, +i.e. `Content-Type') to its value. As a special case, `:boundary' +maps to a string which will serve as the boundary between article +parts. This must be supplied if a custom boundary is used in a +multipart content type header. The default boundary is \"-_nnfeed_-\", +and is automatically modified to match the name of the back end. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; As mentioned in their docstrings, the last two parsing functions +;; can optionally return any Lisp representation they want, provided +;; an appropriate `nnfeed-print-content-function' is defined. This +;; means they are also not _strictly_ required. + +(defvoo nnfeed-read-links-function #'ignore + "Function returning all links contained in an article. + +With the default `nnfeed-print-content-function', it should return a +list of links, where each link is an alist mapping MIME content types +to links formatted for display in a part of that type. Each content +type may also be a list of content types. +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-parts-function #'ignore + "Function returning an alist associating parts of an article to their headers. + +With the default `nnfeed-print-content-function', each part should be a +string. Otherwise, it can be any Lisp object. The \"headers\" of +each part should be a list where each element is either a cons of a +MIME header (a symbol, i.e. `Content-Type') and its value (a string), +or any other Lisp object. MIME headers will be printed, the rest will +be passed on to `nnfeed-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;;;; Feed data storage: + +;; This section defines the data types used to store feed data, and +;; All variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. + +(defvoo nnfeed-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnfeed-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnfeed-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnfeed--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnfeed-backend) + "nn") + (gnus-newsgroup-savable-name server)) + gnus-directory)) + +(defun nnfeed--read-server (server) + "Read SERVER's information storage." + (if-let ((f (nnfeed--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnfeed-servers)) + (nnheader-report nnfeed-backend "Can't read %s" server))) + +(defun nnfeed--write-server (server) + "Write SERVER's information to storage." + (if-let ((f (nnfeed--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n") + (prin1 s (current-buffer)) + (insert "\n") + t) + t) + (nnheader-report nnfeed-backend "Can't write %s" f))) + +(defun nnfeed--server-address (server) + "Return SERVER's real address." + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnfeed-backend 'address) + (cddr (gnus-server-to-method + (concat + (symbol-name nnfeed-backend) ":" + server))))) + server)) + server)) + +;; This is the function which actually uses the functions defined in +;; the last section. None of those functions are called after parsing +;; a feed. + +(defun nnfeed--parse-feed (feed &optional group) + "Parse FEED into a new or existing server. +Optionally, only parse GROUP." + (let* ((feed (nnfeed--server-address feed)) + (s (or (gethash feed nnfeed-servers) (nnfeed--read-server feed))) + (g (or (and (hash-table-p s) (gethash group s)) `[,group])) + (name group) ; (Maybe) fake name + (group (or (aref g 0) name)) ; Real name + data) + (when (setq data (funcall nnfeed-read-feed-function feed group) + s (or (gethash feed nnfeed-servers) + (make-hash-table :test 'equal))) + (while-let ((cg (or (and name `(,data)) + (funcall nnfeed-read-group-function data))) + (cg (prog1 (car cg) (setq data (cdr cg))))) + (let* ((name (funcall nnfeed-read-title-function cg)) ; Real name + (group (gethash name nnfeed-group-names name)) ; (Maybe) fake name + (info (gnus-get-info + (concat (symbol-name nnfeed-backend) "+" feed ":" group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnfeed-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnfeed-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnfeed-read-article-function cg stale)) + (article (prog1 (car article) (setq cg (cdr article))))) + (when-let ((id (funcall nnfeed-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnfeed-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnfeed-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (setq max (1+ max))))) + (publish (funcall nnfeed-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnfeed-read-author-function article) + group-author group) + ,(or (funcall nnfeed-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnfeed-read-links-function article) + ,(funcall nnfeed-read-parts-function article) + ,(funcall nnfeed-read-headers-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnfeed-servers)))) + +;;;; Gnus backend functions: + +;; The following two sections define a Gnus backend interface based on +;; the parsed data from the last section. +;; All server variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. +;; For more information about these functions see the "Back End +;; Interface" section of the Gnus manual. + +(defvoo nnfeed-group nil + "Name of the current group.") + +(defvoo nnfeed-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnfeed-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS HEADERS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers. +- HEADERS is an alist associating article-wide MIME headers to their value.") + +(defvoo nnfeed-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnfeed-group-article-min-num 1 + "Minimum article number for the current group.") + +(defvar nnfeed-date-format "%F %X" + "Format of displayed dates (see `format-time-string').") + +(nnoo-define-basics nnfeed) + +(defun nnfeed--current-server-no-prefix () + "Remove the \"+\" prefix from the current server." + (string-remove-prefix (concat (symbol-name nnfeed-backend) "+") + (nnoo-current-server nnfeed-backend))) + +(defun nnfeed--group-data (group &optional server) + "Get parsed data for GROUP from SERVER (or the current server if omitted)." + (let ((s (gethash server nnfeed-servers)) c) + (or (and (hash-table-p s) (gethash group s)) + (and (setq c (nnfeed--current-server-no-prefix)) + (setq s (gethash c nnfeed-servers)) + (hash-table-p s) (gethash group s)) + (catch :stop (maphash (lambda (n s) + (or (string= n server) + (string= n c) + (when-let (((hash-table-p s)) + (g (gethash group s))) + (throw :stop g)))) + nnfeed-servers))))) + +(defun nnfeed-retrieve-article (article group) + "Retrieve headers for ARTICLE from GROUP." + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string nnfeed-date-format (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnfeed-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + `[ nil ,nnfeed-group-article-ids ,nnfeed-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnfeed-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnfeed-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnfeed-backend "Group %s not found" (or group "")))) + +(deffoo nnfeed-open-server (server &optional defs backend) + (let ((backend (or backend 'nnfeed)) + (a (nnfeed--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (or (gethash server nnfeed-servers) + (nnfeed--read-server a))) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash group g nnfeed-group-names))) + s)) + (or s (file-writable-p (nnfeed--server-file a)) + (and (nnoo-close-server nnfeed-backend server) + (nnheader-report + nnfeed-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnfeed-request-close () + (maphash (lambda (server _) + (nnfeed--write-server + (nnfeed--server-address server))) + nnfeed-servers) + (setq nnfeed-servers (make-hash-table :test 'equal) + nnfeed-status-string nil) + t) + +;; The default content printing function, which should be suitable for +;; most inheriting backends. + +(defun nnfeed--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, with LINKS possibly added." + (let ((links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and content (format "%s\n\n" content)) + (mapconcat (lambda (link) + (cdr (assoc (cdr (assq 'Content-Type attributes)) link + (lambda (types type) + (if (stringp types) (string= types type) + (member type types)))))) + links))))) + +(defvoo nnfeed-print-content-function #'nnfeed--print-content + "Function returning a single piece of content for an article (a string). + +It should accept three arguments, a part and its attributes (as returned +by `nnfeed-read-parts-function'), and links (as returned by +`nnfeed-read-links-function').") + +(defun nnfeed--print-part (content headers mime links) + "Print part of an article using its CONTENT, HEADERS, and LINKS. +Only HEADERS of a type included in MIME are considered." + (concat + (mapconcat (lambda (header) + (when-let ((m (car-safe header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnfeed-print-content-function content headers links))) + +(deffoo nnfeed-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + (and (setq group nnfeed-group) + `[ nil ,nnfeed-group-article-ids + ,nnfeed-group-articles + ,nnfeed-group-article-max-num + ,nnfeed-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((links (aref a 5)) + (parts (aref a 6)) + (headers (aref a 7)) + (boundary (or (cdr (assq :boundary headers)) + (format "-_%s_-" nnfeed-backend))) + (multi (length> parts 1)) + (mime '( Content-Type Content-Disposition + Content-Transfer-Encoding))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnfeed-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + (if (assq 'MIME-Version headers) "" "MIME-Version: 1.0\n") + (mapconcat (lambda (header) + (unless (keywordp (car header)) + (format "%s: %s\n" (car header) (cdr header)))) + headers) + (if multi + (if (assq 'Content-Type headers) "" + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary)) + (prog1 (nnfeed--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (format "--%s\n%s\n" boundary + (nnfeed--print-part + (car part) (cdr part) mime links))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnfeed-backend "No such article"))) + +(deffoo nnfeed-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (if fast (nnfeed--group-data group server) + (setq server (nnfeed--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnfeed-group group + nnfeed-group-article-ids (aref g 1) + nnfeed-group-articles (aref g 2) + nnfeed-group-article-max-num (aref g 3) + nnfeed-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnfeed-group-article-ids) + nnfeed-group-article-min-num + nnfeed-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnfeed-backend "Group %s not found" group)))) + +(deffoo nnfeed-close-group (group &optional server) + (and (string= group nnfeed-group) + (setq nnfeed-group nil + nnfeed-group-article-ids (make-hash-table :test 'equal) + nnfeed-group-articles (make-hash-table :test 'eql) + nnfeed-group-article-max-num 0 + nnfeed-group-article-min-num 1)) + (setq server (or server (nnfeed--current-server-no-prefix))) + (nnfeed--write-server server)) + +(deffoo nnfeed-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnfeed--parse-feed + (or server (nnfeed--current-server-no-prefix)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnfeed-request-post (&optional _server) + (nnheader-report nnfeed-backend "%s is a read only backend" nnfeed-backend)) + +;;;; Optional back end functions: + +(deffoo nnfeed-retrieve-groups (_groups &optional server) + (nnfeed-request-list (or server (nnfeed--current-server-no-prefix))) + 'active) + +(deffoo nnfeed-request-type (_group &optional _article) + 'unknown) + +(deffoo nnfeed-request-group-description (group &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (nnfeed--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnfeed-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnfeed-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (s (or (gethash server nnfeed-servers) + (and ; Open the server to add it to `nnfeed-servers' + (save-match-data + (nnfeed-open-server + server + (cdr ; Get defs and backend. + (assoc server (cdr (assq 'nnfeed nnoo-state-alist)) + (lambda (car key) + (and (stringp car) + (string-match + (concat + "\\(\\(nn[[:alpha:]]+\\)\\+\\)?" + "\\(" (regexp-quote key) "\\)") + car) + (setq server car))))) + (prog1 (if (stringp (match-string 1 server)) + (intern (match-string 2 server)) + 'nnfeed) + (setq server (match-string 3 server)))) + (gethash server nnfeed-servers))))) + (g (or (nnfeed--group-data group server) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnfeed-group-names) + (remhash group s) + (and (string= group nnfeed-group) + (setq nnfeed-group new-name)) + t)) + +(provide 'nnfeed) + +;;; nnfeed.el ends here -- 2.43.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Dec 12 13:08:46 2023 Received: (at 64202) by debbugs.gnu.org; 12 Dec 2023 18:08:46 +0000 Received: from localhost ([127.0.0.1]:57494 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rD7BW-0001Y5-Al for submit@debbugs.gnu.org; Tue, 12 Dec 2023 13:08:46 -0500 Received: from mail.ericabrahamsen.net ([52.70.2.18]:53102) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rD7BT-0001Xj-Ks for 64202@debbugs.gnu.org; Tue, 12 Dec 2023 13:08:44 -0500 Received: from localhost (71-212-21-65.tukw.qwest.net [71.212.21.65]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 27085FA02E; Tue, 12 Dec 2023 18:08:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1702404502; bh=BBcgDl8Yb9vBtU0aeOa5N6ftHGi/y9lWUyNjv/KkdDE=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=Dgd/pEtuveftxa/DsPXy41MfJSz+xjpqPzAmALo/ausMMaYiqIbyKyay7L5k7NQxR b9ZjpjweU+BWhHSx59obmRpcogozA6WDpJDSBkg5S7foYowmR9gdEs3OC5K7Q9uf5d iyeazxr5Js82SHVZJlV6L+t5c6p67lOgRld2Mks0= From: Eric Abrahamsen To: Daniel Semyonov Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87zfyj8ojz.fsf@dsemy.com> (Daniel Semyonov's message of "Sat, 09 Dec 2023 15:59:12 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> Date: Tue, 12 Dec 2023 10:08:20 -0800 Message-ID: <87o7evjnu3.fsf@ericabrahamsen.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64202 Cc: husain@alshehhi.io, 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Daniel Semyonov writes: >>>>>> Daniel Semyonov writes: > > > I think this can be merged now. Updated patch attached. > > Here's another version which fixes a bunch of small mistakes and > clarifies some information in the manual, docstrings and comments. Thanks very much for working on this! I'll need to find a little bit of time to review, but that will happen in the next week. Thanks, Eric From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 16 10:15:32 2023 Received: (at submit) by debbugs.gnu.org; 16 Dec 2023 15:15:33 +0000 Received: from localhost ([127.0.0.1]:55713 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEWNv-00023G-6Z for submit@debbugs.gnu.org; Sat, 16 Dec 2023 10:15:32 -0500 Received: from lists.gnu.org ([2001:470:142::17]:44612) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEWNt-0001jQ-K6 for submit@debbugs.gnu.org; Sat, 16 Dec 2023 10:15:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rEWNn-00011g-1v for bug-gnu-emacs@gnu.org; Sat, 16 Dec 2023 10:15:15 -0500 Received: from mail.fitzsim.org ([69.165.165.189]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rEWNl-000581-55 for bug-gnu-emacs@gnu.org; Sat, 16 Dec 2023 10:15:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=fitzsim.org ; s=20220430; h=Content-Transfer-Encoding:Content-Type:MIME-Version: Message-ID:Date:References:In-Reply-To:Subject:Cc:To:From:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=ValCmJFG3BTKcPLEeCF36weujcdSltAIVKKfmn0PtKU=; b=ldeEBaIPn6DmiwUUivqJPlhoNx qryqLed1/FFA7yK8IodKTwznVkyH947x0Vbfuaof6xE8D3yrhHTAAESZ+pWPUvb2eP6iOsb6jhwv/ leyx06m0THg2PJJ+XNBW8a31YaVD8ZsW41CH0ZsIQXxfh7I8fIX8+8a9LGA9Z/cokA1kHJNZjJGSg 8b923aKtTD4D919cveJqqFpgpD61qCVjVbT5Diloz3ceQw1LQkXAeTJhuIqq61tpJAhYDe9rEBN/c UCEf17MPcPVDyDN1KWR4zIIxIWqwunIZ37Fo6g284j/wVAtMxHVJDCbK8Tzd9Q8EZNr8EGgJlZ1Dv 4K6EyDwg==; Received: from [192.168.1.1] (helo=localhost.localdomain) by mail.fitzsim.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1rEWNT-000cy9-Id; Sat, 16 Dec 2023 10:14:55 -0500 From: Thomas Fitzsimmons To: Daniel Semyonov via Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87zfyj8ojz.fsf@dsemy.com> (Daniel Semyonov via's message of "Sat, 09 Dec 2023 15:59:12 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> Date: Sat, 16 Dec 2023 10:14:54 -0500 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=69.165.165.189; envelope-from=fitzsim@fitzsim.org; helo=mail.fitzsim.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: submit Cc: eric@ericabrahamsen.net, husain@alshehhi.io, 64202@debbugs.gnu.org, Daniel Semyonov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hi Daniel, Daniel Semyonov via writes: >>>>>> Daniel Semyonov writes: > > > I think this can be merged now. Updated patch attached. > > Here's another version which fixes a bunch of small mistakes and > clarifies some information in the manual, docstrings and comments. I am trying this, but am having no luck. I applied the patch to Emacs master tip (2a591b228aaa8c66c27cc5b7cb3033aa6625bc0b) and rebuilt. I am starting from an empty home directory. I customize gnus-select-method to: (nnatom "www.fitzsim.org/blog/?feed=3Datom") Then M-x gnus RET produces: nnatom (www.fitzsim.org/blog/?feed=3Datom) open error: =E2=80=98Server = file www.fitzsim.org/blog/?feed=3Datom not readable or writable=E2=80=99. Continue? (y or n) I instead configure gnus-select-method to (nnnil "") and restart Gnus. Then in the *Group* buffer, I press 'B', then select nnatom, then paste in: www.fitzsim.org/blog/?feed=3Datom and I see in the minibuffer: Unable to contact server www.fitzsim.org/blog/?feed=3Datom: Server file www.fitzsim.org/blog/?feed=3Datom not readable or writable EWW loads both of the following without issue: https://www.fitzsim.org/blog/?feed=3Datom www.fitzsim.org/blog/?feed=3Datom Thomas From debbugs-submit-bounces@debbugs.gnu.org Sat Dec 16 19:13:40 2023 Received: (at 64202) by debbugs.gnu.org; 17 Dec 2023 00:13:40 +0000 Received: from localhost ([127.0.0.1]:56351 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEemq-0000Qf-Bz for submit@debbugs.gnu.org; Sat, 16 Dec 2023 19:13:40 -0500 Received: from dsemy.com ([46.23.89.208]:34566) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEemm-0000QK-4n for 64202@debbugs.gnu.org; Sat, 16 Dec 2023 19:13:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=gQKmF7FFsFM7B esV3E+4ZVRrT+7eaDtXN1XBx4jY4SU=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=sGPHeQ9lPW+l4guVLCN46XQ3t/jghIBHru3 9WYekOa5x1OYMz9IiP/RpXofyEciCKXPPtJYd8sqPnxh/qG7FueUsi4iSBJf1b6E0yMaaG dqSVWy3PRQsgAVkDVSTS1/WzvRML5sIVdDKu9iJmxX83veM/DgZAouy1rJYi/x64Iez2lo 1xSZC3AMCQ+chFp8rqdqAs/sOS58BzABRwV4r+nvvocHM6DiAgS3gpxycbcRAm04OX//kB EQ4GHWlfSdMnw9ReyONVIQQ36cWTiebVZgkpmARwSfC68sVQi65TbB/qeY3n7a6TsNH82h Th8ZWds/TOgg3a6d8jymrZTaXdw== Received: from localhost (CUST-29.202.102.5.018.net.il [5.102.202.29]) by dsemy.com (OpenSMTPD) with ESMTPSA id 7b7c3e46 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sun, 17 Dec 2023 01:13:28 +0100 (CET) From: Daniel Semyonov To: Thomas Fitzsimmons Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: (Thomas Fitzsimmons's message of "Sat, 16 Dec 2023 10:14:54 -0500") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> Date: Sun, 17 Dec 2023 02:13:25 +0200 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: eric@ericabrahamsen.net, husain@alshehhi.io, Daniel Semyonov via , 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >>>>> Thomas Fitzsimmons writes: > Hi Daniel, Daniel Semyonov via writes: >>>>>>> Daniel Semyonov writes: >>=20 >> > I think this can be merged now. Updated patch attached. >>=20 >> Here's another version which fixes a bunch of small mistakes and >> clarifies some information in the manual, docstrings and >> comments. > I am trying this, but am having no luck. > I applied the patch to Emacs master tip > (2a591b228aaa8c66c27cc5b7cb3033aa6625bc0b) and rebuilt. > I am starting from an empty home directory. I customize > gnus-select-method to: > (nnatom "www.fitzsim.org/blog/?feed=3Datom") > Then M-x gnus RET produces: > nnatom (www.fitzsim.org/blog/?feed=3Datom) open error: =E2=80=98S= erver > file www.fitzsim.org/blog/?feed=3Datom not readable or writable=E2=80= =99. > Continue? (y or n) > I instead configure gnus-select-method to (nnnil "") and restart > Gnus. Then in the *Group* buffer, I press 'B', then select > nnatom, then paste in: > www.fitzsim.org/blog/?feed=3Datom > and I see in the minibuffer: > Unable to contact server www.fitzsim.org/blog/?feed=3Datom: > Server file www.fitzsim.org/blog/?feed=3Datom not readable or > writable > EWW loads both of the following without issue: > https://www.fitzsim.org/blog/?feed=3Datom > www.fitzsim.org/blog/?feed=3Datom Huh, I could've sworn I fixed this already. This happens because you don't have an "atom" subdirectory in your 'gnus-directory'. The backend *should* create it automatically, but looking at the code it seems I forgot to implement this. If you create the directory manually it should work. I'll send a patch with a fix as soon as I can. Thanks a lot for testing, I really appreciate it. Daniel From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 17 07:14:42 2023 Received: (at submit) by debbugs.gnu.org; 17 Dec 2023 12:14:42 +0000 Received: from localhost ([127.0.0.1]:56681 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEq2c-0002XL-7S for submit@debbugs.gnu.org; Sun, 17 Dec 2023 07:14:42 -0500 Received: from lists.gnu.org ([2001:470:142::17]:48332) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEq2a-0002X1-Fw for submit@debbugs.gnu.org; Sun, 17 Dec 2023 07:14:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rEq2T-0004TR-JW for bug-gnu-emacs@gnu.org; Sun, 17 Dec 2023 07:14:33 -0500 Received: from dsemy.com ([46.23.89.208]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rEq2O-0000BV-12 for bug-gnu-emacs@gnu.org; Sun, 17 Dec 2023 07:14:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=l6+yaLm73g5V1 OcMZoVTeHaLHmLpdUuHVMTiuPEB4Y8=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=sK6QVZJKBFpmI0KHpJMQ7uoEx+uGaSyrUJm btEelHf3WT5MXSTwJCzMlmtY+J7VvbaY3XOCZOmbBnDV6soKI894Ztey5s4fQxz2CwCB4H zFln/kmD/Ry7onDxYb5ziFEG/xBthPozWcXIDKnQRwVVDL6zdCrsmdEslY2v3aNaM9Q8w4 6VaBTJrh1Tyc2iV7PrLiN5PU9RvmlWfk/FXfrLS19rj8XiwLSHxrw4Su85WrZUmbZVoiFj WjM82FOLleD7fy80mRSRxeVvxPCv8Pb8SUMvjwR1XrFQjwKqc4ueNFBzc6vO2S08LJNPsh naY58dsGB3VxVF7645th0av8K5A== Received: from coldharbour.local ( [147.235.216.58]) by dsemy.com (OpenSMTPD) with ESMTPSA id 06a3024b (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sun, 17 Dec 2023 13:14:25 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id a53c0d08; Sun, 17 Dec 2023 12:10:45 +0000 (UTC) From: Daniel Semyonov To: Thomas Fitzsimmons Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: (Daniel Semyonov's message of "Sun, 17 Dec 2023 02:13:25 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> Date: Sun, 17 Dec 2023 14:10:45 +0200 Message-ID: <87ttohxc56.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=46.23.89.208; envelope-from=daniel@dsemy.com; helo=dsemy.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_HELO_TEMPERROR=0.01, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit Cc: eric@ericabrahamsen.net, husain@alshehhi.io, Daniel Semyonov via , 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) >>>>> Daniel Semyonov writes: > I'll send a patch with a fix as soon as I can. I've decided to setup a mirror of the Emacs repo for further development, so I've pushed the fix there. It can be found at: https://git.sr.ht/~dsemy/emacs-nnatom. From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 17 10:56:11 2023 Received: (at submit) by debbugs.gnu.org; 17 Dec 2023 15:56:11 +0000 Received: from localhost ([127.0.0.1]:58917 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEtUx-0005zD-0n for submit@debbugs.gnu.org; Sun, 17 Dec 2023 10:56:11 -0500 Received: from lists.gnu.org ([2001:470:142::17]:41154) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEtUs-0005yR-R9 for submit@debbugs.gnu.org; Sun, 17 Dec 2023 10:56:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rEtUi-0003V7-O5 for bug-gnu-emacs@gnu.org; Sun, 17 Dec 2023 10:55:57 -0500 Received: from mail.fitzsim.org ([69.165.165.189]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rEtUg-0004SM-2F for bug-gnu-emacs@gnu.org; Sun, 17 Dec 2023 10:55:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=fitzsim.org ; s=20220430; h=Content-Type:MIME-Version:Message-ID:Date:References: In-Reply-To:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=HdcRhEzyQOWKXGN4uuS+Hg0eIu58wv7i4GUYKopBIvQ=; b=TZwEprG6sm95DEjc167u2St4pC i+e5i2qZiDe6UAQBEhdyZqDGKhJqE8lIBSvPYBtixWkchScKMi7bB+GtG0u2ZrF8TaSzFnjTlCrXu 3ouhC5bGvwNmgbL7mGsUPrQDfM0z0RgaalZsXvTtc6d/0JM4vYCkSAsfq5SpecRAcO6Qktqn/SEPo JnruU4fMRf/iI60msmAPMUgvRxQH97aQcwur4q5nBwAuALX+nwNsSPFHeTUOGw7o0AsD+Y4vNiZ0h z5/YMkStEwzHuSOp6Xz2qNiaCWKC01rBX557wNc+4fMYqi9nq7TekjrvJYE9JvsUYSV7TtOhuGAC+ EQY+bobg==; Received: from [192.168.1.1] (helo=localhost.localdomain) by mail.fitzsim.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1rEtUT-000ddE-4B; Sun, 17 Dec 2023 10:55:41 -0500 From: Thomas Fitzsimmons To: Daniel Semyonov Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87ttohxc56.fsf@dsemy.com> (Daniel Semyonov's message of "Sun, 17 Dec 2023 14:10:45 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> Date: Sun, 17 Dec 2023 10:55:40 -0500 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=69.165.165.189; envelope-from=fitzsim@fitzsim.org; helo=mail.fitzsim.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: submit Cc: eric@ericabrahamsen.net, husain@alshehhi.io, Daniel Semyonov via , 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) Hi Daniel, Daniel Semyonov writes: >>>>>> Daniel Semyonov writes: > > > I'll send a patch with a fix as soon as I can. > > I've decided to setup a mirror of the Emacs repo for further > development, so I've pushed the fix there. > > It can be found at: https://git.sr.ht/~dsemy/emacs-nnatom. It works now, thank you. I tested my blog's Atom feed. I also reviewed the patch and it looks good to me, other than "checkdoc" reporting that `nnatom--read-part' needs a docstring. I think Eric should review it too before it goes in. Thomas From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 17 11:16:19 2023 Received: (at submit) by debbugs.gnu.org; 17 Dec 2023 16:16:19 +0000 Received: from localhost ([127.0.0.1]:58965 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEtoR-0003Kx-Ab for submit@debbugs.gnu.org; Sun, 17 Dec 2023 11:16:19 -0500 Received: from lists.gnu.org ([2001:470:142::17]:48264) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEtoP-0003Gu-BA for submit@debbugs.gnu.org; Sun, 17 Dec 2023 11:16:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rEtoG-0007d9-TN for bug-gnu-emacs@gnu.org; Sun, 17 Dec 2023 11:16:08 -0500 Received: from mail.ericabrahamsen.net ([52.70.2.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rEtoF-0003u1-Aq for bug-gnu-emacs@gnu.org; Sun, 17 Dec 2023 11:16:08 -0500 Received: from localhost (c-24-16-170-249.hsd1.wa.comcast.net [24.16.170.249]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id D5AECFA02E; Sun, 17 Dec 2023 16:15:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1702829758; bh=EQhzaV/LD8HtU+lNbdOzFfTzVhBTDKcRD6EOH0FoIZw=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=cuBBYXKw/aGsjsfoDt37XzTQiE9E1Eb+UCo84RHpnDj7TAdAhO/ckIccdWvgcVzFL 3T/b7M6GUoGSYtrV6Dw18WeYjaOXLA6wgj5zlpLyBMGYZO0W9q6ZOL5a41IXUBu3Q4 oPfa42HXI/VBvlg7eO0LeLEXbCrtHZ1oTTcRv+0s= From: Eric Abrahamsen To: Thomas Fitzsimmons Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: (Thomas Fitzsimmons's message of "Sun, 17 Dec 2023 10:55:40 -0500") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> Date: Sun, 17 Dec 2023 08:15:56 -0800 Message-ID: <874jggst37.fsf@ericabrahamsen.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=52.70.2.18; envelope-from=eric@ericabrahamsen.net; helo=mail.ericabrahamsen.net X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit Cc: husain@alshehhi.io, Daniel Semyonov via , 64202@debbugs.gnu.org, Daniel Semyonov X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) On 12/17/23 10:55 AM, Thomas Fitzsimmons wrote: > Hi Daniel, > > Daniel Semyonov writes: > >>>>>>> Daniel Semyonov writes: >> >> > I'll send a patch with a fix as soon as I can. >> >> I've decided to setup a mirror of the Emacs repo for further >> development, so I've pushed the fix there. >> >> It can be found at: https://git.sr.ht/~dsemy/emacs-nnatom. > > It works now, thank you. I tested my blog's Atom feed. I also reviewed > the patch and it looks good to me, other than "checkdoc" reporting that > `nnatom--read-part' needs a docstring. I think Eric should review it > too before it goes in. I will soon! Thanks for setting up the mirror, that makes things easier. From debbugs-submit-bounces@debbugs.gnu.org Sun Dec 17 15:27:34 2023 Received: (at submit) by debbugs.gnu.org; 17 Dec 2023 20:27:34 +0000 Received: from localhost ([127.0.0.1]:59062 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rExja-0001CZ-FT for submit@debbugs.gnu.org; Sun, 17 Dec 2023 15:27:34 -0500 Received: from lists.gnu.org ([2001:470:142::17]:47072) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rExjZ-0001CI-Hn for submit@debbugs.gnu.org; Sun, 17 Dec 2023 15:27:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rExjS-0003ls-Ms for bug-gnu-emacs@gnu.org; Sun, 17 Dec 2023 15:27:26 -0500 Received: from ciao.gmane.io ([116.202.254.214]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rExjR-0002xn-6G for bug-gnu-emacs@gnu.org; Sun, 17 Dec 2023 15:27:26 -0500 Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1rExjM-0007sa-MB for bug-gnu-emacs@gnu.org; Sun, 17 Dec 2023 21:27:20 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: bug-gnu-emacs@gnu.org From: Eric Abrahamsen Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) Date: Sun, 17 Dec 2023 12:27:12 -0800 Message-ID: <87jzpcr2vz.fsf@ericabrahamsen.net> References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <874jggst37.fsf@ericabrahamsen.net> Mime-Version: 1.0 Content-Type: text/plain User-Agent: Gnus/5.13 (Gnus v5.13) Cancel-Lock: sha1:IDFzt4hGe7IUYvvMfnWeXZam1sc= Received-SPF: pass client-ip=116.202.254.214; envelope-from=geb-bug-gnu-emacs@m.gmane-mx.org; helo=ciao.gmane.io X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) Eric Abrahamsen writes: > On 12/17/23 10:55 AM, Thomas Fitzsimmons wrote: >> Hi Daniel, >> >> Daniel Semyonov writes: >> >>>>>>>> Daniel Semyonov writes: >>> >>> > I'll send a patch with a fix as soon as I can. >>> >>> I've decided to setup a mirror of the Emacs repo for further >>> development, so I've pushed the fix there. >>> >>> It can be found at: https://git.sr.ht/~dsemy/emacs-nnatom. >> >> It works now, thank you. I tested my blog's Atom feed. I also reviewed >> the patch and it looks good to me, other than "checkdoc" reporting that >> `nnatom--read-part' needs a docstring. I think Eric should review it >> too before it goes in. > > I will soon! Thanks for setting up the mirror, that makes things easier. I just took this for a whirl; so far so good! TBH I forget how nnoo.el works if I don't look at it for a couple of weeks, so I don't immediately have any comments there. In general I prefer using structs instead vectors-as-structs, though mostly that's for ease of update and change, and if the article vectors aren't being written to disk as such, maybe it doesn't matter that much. Something to consider, anyway. I'll keep trying to break it. Thanks! Eric From debbugs-submit-bounces@debbugs.gnu.org Thu Feb 01 20:36:05 2024 Received: (at 64202) by debbugs.gnu.org; 2 Feb 2024 01:36:05 +0000 Received: from localhost ([127.0.0.1]:43361 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rViTN-0005XJ-34 for submit@debbugs.gnu.org; Thu, 01 Feb 2024 20:36:05 -0500 Received: from mout.gmx.net ([212.227.15.19]:57447) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rViTH-0005Wl-Lc for 64202@debbugs.gnu.org; Thu, 01 Feb 2024 20:36:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=s31663417; t=1706837731; x=1707442531; i=jimjoe@gmx.net; bh=3Ltym0LSoOelCYLTMLT6GyqviS3W+heq88mEE2ODFYs=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References: Date; b=a0CJNaXdjNLsLy7HrLAjAVuxge9JgoQPDBMr8xlYQ/SrjOdoBd0rf9qR7P/P83JT x7vt8p1Z141d+sA2U+z0sEPJ776lmJVKZCt+/2Ud/hlcOAHhVxjOzxk1O1aHuWAaH 9Vn4FI4e/yKG9B4h86caaKcTwp5F+IqOXoLm7YChKE3+8fsCTrU6dhBd69+TSsCUu 2vwTJnF+yvEArWfWlKLeKvjkWGqeGH472B4xyVyY57+lgPTkV49XHTpSF1NIpbVWt YVCa32omREzxUWuhIilQ3sFHe0+JJDxKFZbrh3NbU+hUqTFMUI/QWKrGRG7O2Km7J tRjLcb2ShiCsOa7+aw== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from user-Inspiron-15-5518 ([117.194.144.89]) by mail.gmx.net (mrgmx005 [212.227.17.184]) with ESMTPSA (Nemesis) id 1MqJm5-1qkKcT3LhD-00nOSv; Fri, 02 Feb 2024 02:35:31 +0100 From: James Thomas To: Daniel Semyonov Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87ttohxc56.fsf@dsemy.com> (Daniel Semyonov's message of "Sun, 17 Dec 2023 14:10:45 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> Date: Fri, 02 Feb 2024 07:05:27 +0530 Message-ID: <87le83hcm8.fsf@outlook.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:uWns5HVoF8Y7sSUHXZEynzg5Wod3v52+ztVbFdok5oxCt4vjOXS 3w35LS62ilEZLWwXsPR4Xuw/5yHSBZyQg9gv/pb99z7wDHP3YLQIt2lbwru/BzzagX9ACAH JFxLkkgI0o38t0ofCXKONcuAIG3cZDbehPdw7bOKqF9vSv9owrl+AmnNmcoEuvVUc0/qL4S 6dRBYK6poUWzgmdzLU1BA== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:kf7Wub5D/ZM=;kTzVTJShl928sLQkWFzM0eaj8w1 wVUTrnjLa66HUW2xvSiYD6wg9sr40d3GvoaGLs47asM0XN8nY86Aw3cF4BRsL0fuQJj4Bdc9W 2EDJGG7gw7smm9DWYTWGpLRBFG7MmP6iRI521XesMjAZhpboh8hz8e26cCXc6jUWw8KMiq88b i2kBUJ3+Vjo8SJ1SXYO5vTGcjGRJmY+KCA++3Dk2a3wMq2ijSk1iioNJ9RG797+RokZZVAxa6 8Rso400OYbk7wLzUmiGQQN93/57Y7PwBFknKyp7r/eoKCsCL8Zx/Pf7VE0gj700W2DSY8xEU2 T0GqJ4I5g2WhIKGHalyq/kB90fs8ms78EIfh83ffmcOEjTK5xCn6/dz0GF+cjTyljxC93WGDr 3ejsLDR7Acj8CDXpjnLlyDe1m6m1SSxWsIdxxatDEMxkhz71C7gFUwjKdYaV+m2RvkjlQknRV 8VHt0dreQAkXPAU7gmho/vxM8MRzD1zSbEMvaWroVKqz7k+mgIUO33zEocHSTmNK3IzMO05Tb wU2tH244O9cGXE7S47q1dcI7vsty5/dJDe6hDm3dqPKVmtq3e+zhP+8n+SIPxb4qiPQt007+L 5vZBswVmBr/trFaoWQxhTccE36CDcuaACzstjSKtejqyfkVx89wDdZ69tBAI7w+K+f27hIBuI 6wkCKNAVVOnLCyNq2Xx9pEAqYKoUvktItygw3Siwjn0t06WxvRd5HBp+XR/J9H9Mi00hsGBkf 9cAeStz2MVzNIqmiqP2wiyiyG3bd2KwzrRQs8twe2sgTfr2YI8ysUGiC3Rb1zqqfINNA3SIGm x9SYHZrIrft5a0h6Ccm3UU3YIATmH4gSRXuaG9s+Ovur0= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Daniel Semyonov wrote: >>>>>> Daniel Semyonov writes: > > > I'll send a patch with a fix as soon as I can. > > I've decided to setup a mirror of the Emacs repo for further > development, so I've pushed the fix there. > > It can be found at: https://git.sr.ht/~dsemy/emacs-nnatom. I've been trying it for a few days now, and one thing I notice here is that the time of the post isn't displayed accurately. For instance, even if it's 3:30 PM, the summary buffer simply shows 3:30 in my 24-hr clock. I've no issues other than this. I don't think it's because of my config, but will check again when I have time. -- From debbugs-submit-bounces@debbugs.gnu.org Sun Feb 04 07:45:55 2024 Received: (at 64202) by debbugs.gnu.org; 4 Feb 2024 12:45:55 +0000 Received: from localhost ([127.0.0.1]:48372 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWbsh-00072t-HG for submit@debbugs.gnu.org; Sun, 04 Feb 2024 07:45:55 -0500 Received: from dsemy.com ([46.23.89.208]:23854) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWbse-00072c-5j for 64202@debbugs.gnu.org; Sun, 04 Feb 2024 07:45:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=fIxpek+7PWrBl MXruntFpli7C2rK+tqUv6pZw6cqfNc=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=FT8501yE6Gn5sTQUHBTkjFydobvWcY2Cbhq 72gjPjSTKV9lOO96NvL7mQBAUYdD7UpcRVLmBkLAkOmpCdVT+o+YVPhC++gjHHZHdJkwiI MhJsfioMBT3Peletoh7ofLxxYCMFDiSh86Tre1exQcUYyFhdhqSnriWvzvRFSe47P6QXZn whvu7VzJ9kn0sgtYnLB510sEViqbfHp/81Xfk7rM/7moplP2bCSLRwufplX42UzXuifKmT i+BlWK8nv9YqxOBxygDbRVkX258Owx9x1ZTUAMxYmCyrFhgdAzjbtSVNETV7HjGx393XXy UVOK8cCG/CEf1gVcf6hX5SgN1SQ== Received: from coldharbour.local ( [147.235.218.19]) by dsemy.com (OpenSMTPD) with ESMTPSA id 0034a86f (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sun, 4 Feb 2024 13:45:32 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id 12816030; Sun, 4 Feb 2024 12:38:22 +0000 (UTC) From: Daniel Semyonov To: James Thomas Subject: Re: bug#64202: [PATCH] Gnus: Add back end for Atom feeds (nnatom) In-Reply-To: <87le83hcm8.fsf@outlook.com> (James Thomas's message of "Fri, 02 Feb 2024 07:05:27 +0530") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> Date: Sun, 04 Feb 2024 14:38:22 +0200 Message-ID: <87h6io5rr5.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 3.5 (+++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: >>>>> James Thomas writes: > I've been trying it for a few days now, and one thing I notice > here is that the time of the post isn't displayed accurately. For > instance, even if it's 3:30 PM, the summary buffer simply shows [...] Content analysis details: (3.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.218.19 listed in zen.spamhaus.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 2.5 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: >>>>> James Thomas writes: > I've been trying it for a few days now, and one thing I notice > here is that the time of the post isn't displayed accurately. For > instance, even if it's 3:30 PM, the summary buffer simply shows [...] Content analysis details: (2.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.218.19 listed in zen.spamhaus.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager >>>>> James Thomas writes: > I've been trying it for a few days now, and one thing I notice > here is that the time of the post isn't displayed accurately. For > instance, even if it's 3:30 PM, the summary buffer simply shows > 3:30 in my 24-hr clock. > I've no issues other than this. I don't think it's because of my > config, but will check again when I have time. The hour and minutes should be displayed formatted to your 'locale=E2=80=99s "preferred" time format' (according to `format-time-string's docstring). I've tested with a few feeds, and this seems to display correctly for me (with a 24 hour clock). I the issue is that your locale's "preferred clock" is a 12 hour clock and I didn't realize that I need to also add "%p" to the format string to show the locale's version of "AM" or "PM" (or nothing). I'll push a fix soon. Thanks for testing, Daniel From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 07 12:57:03 2024 Received: (at 64202) by debbugs.gnu.org; 7 Feb 2024 17:57:03 +0000 Received: from localhost ([127.0.0.1]:57878 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rXmAR-0004n3-38 for submit@debbugs.gnu.org; Wed, 07 Feb 2024 12:57:03 -0500 Received: from dsemy.com ([46.23.89.208]:37392) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rXmAO-0004mT-RP; Wed, 07 Feb 2024 12:57:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=A3GJmAaq/Rm/F 3JW8x/olG/accZhcW3D5Mzdx95UA4s=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=sO7uoTYmrq3kZMIWYfSWySBQCu/AlHdZiB2 Un1HOs/SHexmhZLYuEq3L2nEDM417titsVkPQfLx6Ip6K4BO7w2DmJjkHjbyUfrPWvEdeU ex2wT4/FCdhIOhSWWvAY0nxnXoqFbVR2Y//KnIZJPjzcYdV+/9AwtOnIrqQGlKuxXUIrrl KDMOPFIJxMSv/DUp/p9XKdnBM0Kgtj+V/AO5aaDgU8to0rc1RZbsQVEmW6eziwId6z6xPb DYdsfhAkeqIlh5w1GO0IRReYaUh4qdVfWSLzWk9rxY1SyFPOGLZatwx7qTsbvvs/w1kHtY Czy+cZSlSbY4/FomRC1jhopijvg== Received: from coldharbour.local ( [147.235.218.19]) by dsemy.com (OpenSMTPD) with ESMTPSA id 52c2ee42 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Wed, 7 Feb 2024 18:56:39 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id 8cce0e2e; Wed, 7 Feb 2024 17:49:26 +0000 (UTC) From: Daniel Semyonov To: James Thomas Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87h6io5rr5.fsf@dsemy.com> (Daniel Semyonov's message of "Sun, 04 Feb 2024 14:38:22 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> Date: Wed, 07 Feb 2024 19:49:26 +0200 Message-ID: <87r0hof9ll.fsf_-_@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 3.5 (+++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: >>>>> Daniel Semyonov writes: > I the issue is that your locale's "preferred clock" is a 12 hour > clock and I didn't realize that I need to also add "%p" to the > format string to show the locale's version of "AM" or "PM" (or > [...] Content analysis details: (3.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.218.19 listed in zen.spamhaus.org] 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: 64202 Cc: 66188@debbugs.gnu.org, 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 2.5 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: >>>>> Daniel Semyonov writes: > I the issue is that your locale's "preferred clock" is a 12 hour > clock and I didn't realize that I need to also add "%p" to the > format string to show the locale's version of "AM" or "PM" (or > [...] Content analysis details: (2.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.218.19 listed in zen.spamhaus.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager >>>>> Daniel Semyonov writes: > I the issue is that your locale's "preferred clock" is a 12 hour > clock and I didn't realize that I need to also add "%p" to the > format string to show the locale's version of "AM" or "PM" (or > nothing). > I'll push a fix soon. I've pushed a commit which turns the displayed date format into a user option (and fixes the default value, and uses a simple date format when it isn't displayed). From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 07 23:27:56 2024 Received: (at 64202) by debbugs.gnu.org; 8 Feb 2024 04:27:56 +0000 Received: from localhost ([127.0.0.1]:58774 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rXw0y-0005Rv-0m for submit@debbugs.gnu.org; Wed, 07 Feb 2024 23:27:56 -0500 Received: from mout.gmx.net ([212.227.17.20]:44663) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rXw0w-0005Rb-Ga for 64202@debbugs.gnu.org; Wed, 07 Feb 2024 23:27:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=s31663417; t=1707366442; x=1707971242; i=jimjoe@gmx.net; bh=oW6bCG936AWBdZXRy//+/FQRJh2Qh8ktUzx/wKkFskA=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References: Date; b=SbVgQIUiLZZU4NWRM/Ex9hL0xx3oA99GhY2UfIZ88N7dJG3/Cz7DZGioeGZ03fab U9R3+BqE4XLS55ccch2P6/ZD0DhwQuK1Uy29BQyDrtNRqeBDdQLQ6CVUhrjB0AVQ/ 76+U3XD1lw5uXxPRrTbx4OKYMd47IVFdjFFHB46zKzc8tilZgJ4IXP2Yh2Xfbz3qF yatWDl9zL4FgAsWH+X8RUonmuQC7CF2O73Rw8ZODJhSMSOmQP7r0JDnHhXKF9jQuy t1Cq7n3MHL/2aqaD65wkjuFwm3Vtqwjgjzq2ugrUMDm9n66Y/3Wk3ajFHCKCgLxut dK4q+Xq/ApnPa+erfQ== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from user-Inspiron-15-5518 ([117.194.148.126]) by mail.gmx.net (mrgmx105 [212.227.17.174]) with ESMTPSA (Nemesis) id 1MQMuX-1rKvSy2uMj-00MPC2; Thu, 08 Feb 2024 05:27:22 +0100 From: James Thomas To: Daniel Semyonov Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87r0hof9ll.fsf_-_@dsemy.com> (Daniel Semyonov's message of "Wed, 07 Feb 2024 19:49:26 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) Date: Thu, 08 Feb 2024 09:57:18 +0530 Message-ID: <87jznfpom1.fsf@gmx.net> MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:ckZven1rrRIQO/W/WN/trOwI5DcJpL6GH2rCjaFnJW4rcsrD27u dS+wYCRAhTCdrjyaRlFQ9ktyNQebis+lAD6domUOHHUBHm9HAfvpeMMsfDmdH5/QJ/XxCTF WWvsXzw3ELZ382iYsdM1S++j6LNBApFhOWyYJrb0Fh+1FqiXeUImgRYGTNA8Y/Z3ex6RunG w5X540p15PezjHfwrlPvg== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:3vECDGMtOTY=;ELuu5hL7ZE/nt9c5keOaxFQrIfJ ZFaJi7ZfTvl56TJ12JpQnL4PTTW7mdhoK3sBDg4Fobzg3A0JfIOP8afTAg4UuIJiV/KBhRT8x Wp5YcbjL+0Ty9VJAPk1zvI4017A+Dnv9pji45VHh6GSXAiG1jOjo9qg2V8rUldphiYd6NVyEA cgHeGjta0g5SdK9fqWJcatjDqxrsp4dGmVmUHVgknom8xuPUXVs8Tl7J705/YFPdSVzmYm7Gr 3H4B7KWr/vsrHdxLtuDGp92lUPzeQ1jZhVc60OLJkwMWP4UlLpgxWKybRcdtdzba+LRzJ6Lqz g6mf+RwrDwClbk5qJaJoC0uuXUPtKj9edTdNpMjSa1kDXRKp/cUVEP9Kob6QT+qvbfydtTekh RjF0wO+DVOqF83VkG/0BdL348YzD7CSKgMxHSohiP4I9DxvT9KhdD14C/jN2YKedxsewN4WVW RCF3HNm113nafqVk1alHp+FsDgfIcibCQNkJRYMQu9EGRYqOyaWTho35QRQ6dqUnQvKeSOe1D xdHHTWBv4rtDK4HeU+0jhlsD4khqg+52AyBy7o8CHbCLLescCezvrURPUtgBiOE3E0blE1fnL s2vvZK0ehKzBYO3rasOYZhq4JfdKPblRrY+kJjKt2TC1aGjasSxvYsJ25eLZLiu6jv91OCprC vCkDFpwOQfduyrw6mfDOxuxXv+D9AXJzWavkpnTmLJLSSkR9K2xcErvaEbWf84WF+9bff+Z/v YCEbs8L7D14IGN/TvH1psIq3RKPUSFMrOc++yCdzIT6JlnVIez8ogncSs6JpKaihb8IrMvJcq 3o4z+02xhrBfsu5mcVWr7TjTGS2H22JgLRIjwyDKQKG6c= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Daniel Semyonov wrote: >>>>>> Daniel Semyonov writes: > > > I the issue is that your locale's "preferred clock" is a 12 hour > > clock and I didn't realize that I need to also add "%p" to the > > format string to show the locale's version of "AM" or "PM" (or > > nothing). > > > I'll push a fix soon. > > I've pushed a commit which turns the displayed date format into a user > option (and fixes the default value, and uses a simple date format when > it isn't displayed). Works now. Thanks! Another thing: The article numbers seem to be assigned in the reverse order of datetime on each fetch: ie. the later posts seem to have a smaller number. Is this intentional? I'm asking because I use `-sort-by-number' and this kinda messes it up. I know I can use `-by-date', but still... Regards, James From debbugs-submit-bounces@debbugs.gnu.org Thu Feb 08 06:50:03 2024 Received: (at 64202) by debbugs.gnu.org; 8 Feb 2024 11:50:03 +0000 Received: from localhost ([127.0.0.1]:59277 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rY2up-0001vW-4c for submit@debbugs.gnu.org; Thu, 08 Feb 2024 06:50:03 -0500 Received: from dsemy.com ([46.23.89.208]:31238) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rY2uk-0001us-SZ for 64202@debbugs.gnu.org; Thu, 08 Feb 2024 06:50:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=bHZ5TOlxrfv7N XxT1+zdbnyHQL0unG2Kpa38bpVASq0=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=mbpPbA1JyfOLFtZByaSXyDzrgspmrgTw+R8 CTuYntwyivvU5a6a2om6OWP2EqColIpzkK5hQ70xPRqMSfwRyS11Fgk3UYcEn6oRBGQ5FZ gyaJVHLOxU4CMGjTp2mf4wFlq0g+71xkBhf71+CBbs1TNn54OQI9Qi5FxpASdCWhdwUf9Y XqenQPYN5vjj8Tr+Kd+mNSZcqW2E+izL4TYGqXFJQ40QhlZnel+GPbWHNegTWrPrVSTRDv KEaYRoGiwYD9Rwnh+2su3Xc0JB9FZKrrUFLKAESlxNsE0gKoKzOLrKl90b4iNeulO/DgAP 2K5yhbzr+wqf8IF21pjdlf3D+QA== Received: from coldharbour.local ( [147.235.218.19]) by dsemy.com (OpenSMTPD) with ESMTPSA id 81e8320c (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 8 Feb 2024 12:49:37 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id ee97dc6e; Thu, 8 Feb 2024 11:42:23 +0000 (UTC) From: Daniel Semyonov To: James Thomas Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87jznfpom1.fsf@gmx.net> (James Thomas's message of "Thu, 08 Feb 2024 09:57:18 +0530") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> Date: Thu, 08 Feb 2024 13:42:23 +0200 Message-ID: <874jejb2sg.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 3.5 (+++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: >>>>> James Thomas writes: > Another thing: The article numbers seem to be assigned in the > reverse order of datetime on each fetch: ie. the later posts seem > to have a smaller number. Is this intentional? Content analysis details: (3.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.218.19 listed in zen.spamhaus.org] 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 2.5 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: >>>>> James Thomas writes: > Another thing: The article numbers seem to be assigned in the > reverse order of datetime on each fetch: ie. the later posts seem > to have a smaller number. Is this intentional? Content analysis details: (2.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.218.19 listed in zen.spamhaus.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager >>>>> James Thomas writes: > Another thing: The article numbers seem to be assigned in the > reverse order of datetime on each fetch: ie. the later posts seem > to have a smaller number. Is this intentional? The articles are numbered based on the order they appear in the feed. This could be changed, however there is no requirement to place feed items in a chronological order in Atom (or other feed formats AFAIK), so this would "fix" this for some feeds while "breaking" it for others. I think it would be better to just explicitly sort by date in this case. Daniel From debbugs-submit-bounces@debbugs.gnu.org Thu Feb 08 22:54:41 2024 Received: (at 64202) by debbugs.gnu.org; 9 Feb 2024 03:54:41 +0000 Received: from localhost ([127.0.0.1]:57201 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rYHyL-0006Zp-Al for submit@debbugs.gnu.org; Thu, 08 Feb 2024 22:54:41 -0500 Received: from mout.gmx.net ([212.227.17.21]:53139) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rYHyI-0006ZL-Gc for 64202@debbugs.gnu.org; Thu, 08 Feb 2024 22:54:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=s31663417; t=1707450855; x=1708055655; i=jimjoe@gmx.net; bh=SpemEzX0ts8Bc121EKO1Jv9WXuNHsM1+J3MAbMrLMvs=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References: Date; b=X16L5e/qyqyCMG0p3RTdaXCEqGTzVE21cpxP6gDK3cOeydg1qKNU3ZllykMwoFMg 41dRT14g4yZW4NdHIoHldheuDIZ+z+11q78u58G1mmZ22dzg5OCfXLUg6bMS7XZT2 Q6qQX4JlDP25KWUt6mGWL9JPtDsuAUOiVfTJ+RXnTVU9z1Trl5vuHfUGJe0MkA+eU cLgvFmPAsfHoHPQ0aHbkLDv/eCfJSisYYvPCIKWxkMl2WVRDqbR9xiTM1muugPQPU BbeIWai8edQUoTscuHQtRmo1Cp60pOVFHQeTBmBPB7qh+IPWq/YygVtqRoVd2Bx/L Yt8NwHggT3qd0tjv/A== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from user-Inspiron-15-5518 ([117.216.27.81]) by mail.gmx.net (mrgmx104 [212.227.17.174]) with ESMTPSA (Nemesis) id 1MDysm-1rfxSd0lYi-00A1Zi; Fri, 09 Feb 2024 04:54:15 +0100 From: James Thomas To: Daniel Semyonov Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <874jejb2sg.fsf@dsemy.com> (Daniel Semyonov's message of "Thu, 08 Feb 2024 13:42:23 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> Date: Fri, 09 Feb 2024 09:24:12 +0530 Message-ID: <87fry2pa1n.fsf@gmx.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:kEPvPo84PtFKhbwLrq4jXt0aCbNL+TbKRYAbKyWvUrcTXvCRIId e1Fz53bGjeOPIi8B57QtqJq2cZtKEawl4Y3IMSs3Cn60WBwb2d9+ttDf7+0KZF17qESAVa7 +TH8LvzxK8/lzzNreb6FzSnDswTA+wTknlKKoGYsjkz1yT7s2a0UR1aRA7/mPf+582DFwWm vJE3wUqaOvN3rsPxN1NMg== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:FQCMvTqbnug=;S7CtrcyucQmpChNOXGSJxBGys6I SYfau1iPbDv2G0YqTl8RFy90oiVb0NBptA3kjrkc9Wk2zYh/Q4woUTJgoQ6JVmqSE7TIRGUaW tPYvt3RsYuka5aTBUkHSRlcxTjNZVAdaratytQIu4O6iq05w3TdqSg1EqBtMR6y0yIs+C8d75 XLXeZHegbFpS3jkpdWZBOcYLPG/+dlrUOyGGSSrUOZhOhqMGNqB1HxDjCGwMcKVYxKxVZLXf1 qoB6kqwCuhYnYlXzkcs/m3lyXQ/jKZXZFXRe67BsbKeXcrolmIz7HkJkWeWeMCRoZZMGi4P2g x0gBFaygpBdpQRvaucUTzrGpsctapwUbgfLnm///cowmKq5WSZhAKVtPWkKaU0Ky+Ox5MwJL7 8CqLFdpxkHDos2lpCTZenu+9vFwUkE4taqYulbRfUeYcZGgEhdIMYb1Q8tPxeKBgluJvOC1iV iTUAhvrMBiFA3qUEsbOUi5eoGCw/1wLdy0g2h2AfuVYYlPhsE3KweXefQBieoCmxdv2y4i55x g9q7c09DsxQZW9yGBxWbzrHp+zZFQH/N1B6dd/fyaL8cUDbDgTEE/a3b9IuEdjyeXBcgK15nm HnOtiQqGN1n8Od5RiidlR/k+jQak5VPIXNYHJqNY95kOZjlt4nHQhVlZzk7yCyJ4ufTWpzgAg hbphxg98rMKn079ddLOW+54In8YnyaYBAj7fyPLOYbAFvIb2Ywvexs0qEurtobi5Im203v6EW dceBdyBSickz8tdzmJbOasL01WwAi7MB3qS9HiIVFPu56VZWFUaOr7Vt1j6aiYEA5f+Rw9J/H a/u+rKE3tiEENJaEX+gt9hoFFGXpSXuLy8s5YdsX/kgyM= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Daniel Semyonov wrote: >>>>>> James Thomas writes: > > > Another thing: The article numbers seem to be assigned in the > > reverse order of datetime on each fetch: ie. the later posts seem > > to have a smaller number. Is this intentional? > > The articles are numbered based on the order they appear in the feed. Of course, but my point was different. Let me try and explain it better with an example: Let's say a fetch in the early morning returns these: 6am : Post 1: article number 3 7am : Post 2: article number 2 8am : Post 3: article number 1 (note that the order is reverse) And a second fetch later on returns these: 9am : Post 4: article number 6 10am: Post 5: article number 5 11am: Post 6: article number 4 When sorting them later on (by number), the order ends up being: 8am : Post 3: article number 1 7am : Post 2: article number 2 6am : Post 1: article number 3 10am: Post 5: article number 5 11am: Post 6: article number 4 9am : Post 4: article number 6 ...which is neither descending nor ascending (by time). This could be fixed if the number generation order (on each fetch) were simply reversed. (info "(gnus) Back End Interface") uses the words 'later' and 'order of arrival'. WDYT? > This could be changed, however there is no requirement to place feed > items in a chronological order in Atom (or other feed formats AFAIK), so > this would "fix" this for some feeds while "breaking" it for others. > I think it would be better to just explicitly sort by date in this case. Regards, James From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 09 09:09:22 2024 Received: (at 64202) by debbugs.gnu.org; 9 Feb 2024 14:09:22 +0000 Received: from localhost ([127.0.0.1]:38700 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rYRZB-0006iG-PK for submit@debbugs.gnu.org; Fri, 09 Feb 2024 09:09:22 -0500 Received: from dsemy.com ([46.23.89.208]:22815) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rYRZ9-0006hr-27 for 64202@debbugs.gnu.org; Fri, 09 Feb 2024 09:09:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=60guEUWLhyX2p m04x+iDawG/QovcwbDgrtjhTcf1Fmo=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=pYU8CZ3dCepcxhKBh6odq4hodDbWDAxT3nP GEy/92UmMBK9qVE8Wurvv2divrX7M+kO7z2+V4Q3RxyEfprfoSltXbDjhbCDoTTD75YTRd Opg4JXhfxkeTejnW2mTinsSSWf47bQbDsshYbkfOWBPULn4J18lZx5qlK6Q3b1pq9OFXTp T9IOP+1GPPN6OolACXRcqcodJHUxXAAfJkY+UHi4YzhpNYJJVi+lzcvWmvD3UibC0aW/0C 4okinpWv+f2DPpdgTJkdJ/nxK9Y/4r93xr58MriDWygvBMalyix2hxOnhQf/TdAVSNG24V rKhCw8D6MeI1/+8wwHpiHCv79eQ== Received: from coldharbour.local ( [147.235.215.121]) by dsemy.com (OpenSMTPD) with ESMTPSA id cc0bf1fc (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 9 Feb 2024 15:08:55 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id 2ac69444; Fri, 9 Feb 2024 14:01:37 +0000 (UTC) From: Daniel Semyonov To: James Thomas Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87fry2pa1n.fsf@gmx.net> (James Thomas's message of "Fri, 09 Feb 2024 09:24:12 +0530") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> Date: Fri, 09 Feb 2024 16:01:37 +0200 Message-ID: <87ttmhwxby.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 3.5 (+++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: >>>>> James Thomas writes: > Daniel Semyonov wrote: >>>>>>> James Thomas writes: >> >> > Another thing: The article numbers seem to be assigned in the >> > reverse order of datetime on each fetch: ie. the later posts seem >> > [...] Content analysis details: (3.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.215.121 listed in zen.spamhaus.org] 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 2.5 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: >>>>> James Thomas writes: > Daniel Semyonov wrote: >>>>>>> James Thomas writes: >> >> > Another thing: The article numbers seem to be assigned in the >> > reverse order of datetime on each fetch: ie. the later posts seem >> > [...] Content analysis details: (2.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.215.121 listed in zen.spamhaus.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager >>>>> James Thomas writes: > Daniel Semyonov wrote: >>>>>>> James Thomas writes: >> >> > Another thing: The article numbers seem to be assigned in the >> > reverse order of datetime on each fetch: ie. the later posts seem >> > to have a smaller number. Is this intentional? >> >> The articles are numbered based on the order they appear in the feed. > Of course, but my point was different. Let me try and explain it better > with an example: Sorry, but I don't understand this example, you seem to be describing the exact opposite of what should be happening. > Let's say a fetch in the early morning returns these: > 6am : Post 1: article number 3 > 7am : Post 2: article number 2 > 8am : Post 3: article number 1 > (note that the order is reverse) If the 6am post is the first post in the feed, it should get article number 1 (and the 8am post should get number 3). > And a second fetch later on returns these: > 9am : Post 4: article number 6 > 10am: Post 5: article number 5 > 11am: Post 6: article number 4 Same for the 9am post getting article number 4. > When sorting them later on (by number), the order ends up being: > 8am : Post 3: article number 1 > 7am : Post 2: article number 2 > 6am : Post 1: article number 3 > 10am: Post 5: article number 5 > 11am: Post 6: article number 4 > 9am : Post 4: article number 6 > ...which is neither descending nor ascending (by time). This could be > fixed if the number generation order (on each fetch) were simply > reversed. If the posts appear in a chronological order, it should end up being correct. I think what's happening is that the feed you're testing actually uses a reverse chronological order for posts. If not, send me the feed, I might be misunderstanding something. Also there is no "number generation order", the number is just incremented when a new entry is encountered. You can't know ahead of time how many new entries you will encounter, so you can't easily "reverse" this process (it is possible though). > (info "(gnus) Back End Interface") uses the words 'later' and 'order of > arrival'. > WDYT? I don't think it's really possible to know the order in which entries arrived in a feed. I've seen feeds with posts in chronological, reverse-chronological and (seemingly) random orders. Each entry in each feed might contain a date and an "update" date (in Atom feeds). IME the relation between these dates and the dates the corresponding posts were actually published or updated is non-existent. I still think your best bet is to just sort by date manually. From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 09 15:06:51 2024 Received: (at 64202) by debbugs.gnu.org; 9 Feb 2024 20:06:52 +0000 Received: from localhost ([127.0.0.1]:34628 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rYX99-0003Yr-6W for submit@debbugs.gnu.org; Fri, 09 Feb 2024 15:06:51 -0500 Received: from mout.gmx.net ([212.227.17.20]:55779) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rYX96-0003YR-DT for 64202@debbugs.gnu.org; Fri, 09 Feb 2024 15:06:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=s31663417; t=1707509185; x=1708113985; i=jimjoe@gmx.net; bh=0FwY6bW4gTrSoGIbgWxM225GKhNmT97rwNbh5WEmmjM=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References: Date; b=WrIjho2RNG/VoUOL7+YWaIsWyl5ybCHc3DdyUbtZ83JtK6LDX+rVcyojUtuvPDHX WQLELz9OWD8gnjiek+gZy4/uUuo9YdTewIykzLpSSLoHY2HDwZpvXmJ6zlRTJvjAy Md7EHAyejoqbgUxOWVAJlQqL6ZOHciqRTj7yyc0+AKdjalXrPwuXriE2OX/sX8g2X 2aYz4Ow0YDAN/rMuERRGGw4nIMHpisTQ3wzkD6FKQwN+nS2pub5FImNI1WTHthyaI uQ10iduuvRDThG7PoYgsKhVLJ3ghL3j+ixaEOjf0wfnNsoFl+ISbp+5fgoEfHWnwA CIR6fYb9qj7a5Iz/PA== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from user-Inspiron-15-5518 ([117.194.147.10]) by mail.gmx.net (mrgmx104 [212.227.17.174]) with ESMTPSA (Nemesis) id 1MNbox-1rNM3G0pEh-00P2BD; Fri, 09 Feb 2024 21:06:25 +0100 From: James Thomas To: Daniel Semyonov Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87ttmhwxby.fsf@dsemy.com> (Daniel Semyonov's message of "Fri, 09 Feb 2024 16:01:37 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> Date: Sat, 10 Feb 2024 01:36:21 +0530 Message-ID: <87wmrd9zcy.fsf@outlook.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:lB8s7RSnh9cbFiFj94F0s/aCAfSvairHLMz18e0UW2v8MZhQUHr CErvvNpl0fKuofRUftAh2DKcdK9UMkOElgg9j6IOCLTd3nvk5IrrTCWNpkXx+5+mPn4UBto ZTfzN8pybzTINtNAPLH+m4tBUGp9MpXFbiWGBHG6JCeIaRsIcKg55y1bSe0nx7RRiAJl25M 0VuOVy8QD/cj2iECLAxyg== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:MjQ87elZP4Y=;R3r6KMhPa6JYm65R5A21D28lCZy xXe73i8pfPapALlVqEg44iA59/XsuZ4NaXwM76afN5jYgmd1rjR8yHG0fOTJybecSi9eHbplJ ZqmiQSgGdX4yH14XerIYBeTE5D/5KL+KpM7rrfKz/VfagA0BWmqcgSBehAgoONjToF5InHwa2 BNwwbnJtsxDSdO/3yuSwFzVRJfPSdw6V0dZncxF2QLlr2mtKbf3dFpmaWO7kyy4Xnzywka5+I VZ89H6pGdlj8TUFpItSJYy6xkuwZMwYl1zorEJVBbhQmN9NfeUGOC0Xxn/etrr0zMEMAayBfi +RulCqLaXletI4E8HDEHqVNu6g8Z4or0fbXQYMeWZuMvD+kDtrzxZJthpbU5ThecHf9cvqtcg 1+W5Sztxkza6Y36i3kbavGteHyoENru2ghq+VGIured5z2wvnURAw+P3HJGfRuQXjKMndJjue BfPpGIbIlVb9B4tD6+DRTmmPPbH1oI8puc9cAfZaklCXh+tYLd4x2JBIA123XTaPdQZf6BkCO vc1SiCLGgCp7dCPtvMCD1929D6dqm/9C79jsuyQN5dgrVCi9TpS/RcZ5dmlkc/ZNdRq4kBDEf a+oQ08Ed0QkNDojQ59q329UKTam4DhcNyKg8V9V2jObyK1DFCVOjmK3FQv6hLT+vZGf8DczOD AvjbuyA2mvFrQFyBQ9wMtD9oB35a68B11xzL/mNw6rtwWxJ0I9gT0gPR8LwBWdoVjPrNGU1y1 5CMvPbUCQ3KjHLgd4RE4a1KJC8LIWn7l2H35S0yEImDR5jhJskLO+sb80tcYvB2OjjWUQr7ZP nSlT5mqGtzAtgICV8oloIkVPCSWgPx6VB2wiyq4cOp48A= Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Daniel Semyonov wrote: >>>>>> James Thomas writes: > > > Daniel Semyonov wrote: > >>>>>>> James Thomas writes: > >> > >> > Another thing: The article numbers seem to be assigned in the > >> > reverse order of datetime on each fetch: ie. the later posts se= em > >> > to have a smaller number. Is this intentional? > >> > >> The articles are numbered based on the order they appear in the f= eed. > > > Of course, but my point was different. Let me try and explain it b= etter > > with an example: > > Sorry, but I don't understand this example, you seem to be describing > the exact opposite of what should be happening. > > > Let's say a fetch in the early morning returns these: > > > 6am : Post 1: article number 3 > > 7am : Post 2: article number 2 > > 8am : Post 3: article number 1 > > > (note that the order is reverse) > > If the 6am post is the first post in the feed, it should get article > number 1 (and the 8am post should get number 3). > > > And a second fetch later on returns these: > > > 9am : Post 4: article number 6 > > 10am: Post 5: article number 5 > > 11am: Post 6: article number 4 > > Same for the 9am post getting article number 4. > > > When sorting them later on (by number), the order ends up being: > > > 8am : Post 3: article number 1 > > 7am : Post 2: article number 2 > > 6am : Post 1: article number 3 > > 10am: Post 5: article number 5 > > 11am: Post 6: article number 4 > > 9am : Post 4: article number 6 > > > ...which is neither descending nor ascending (by time). This could= be > > fixed if the number generation order (on each fetch) were simply > > reversed. > > If the posts appear in a chronological order, it should end up being > correct. I think what's happening is that the feed you're testing > actually uses a reverse chronological order for posts. If not, send me > the feed, I might be misunderstanding something. Hmm.. It seems to be happening that way for every server I used: git.savannah.gnu.org/cgit/emacs.git/atom/?h=3Dmaster rss.slashdot.org/Slashdot/slashdotMainatom:Slashdot github.com/emacs-mirror/emacs/commits.atom (the article numbers are displayed in the Summary buffer modeline) > > Also there is no "number generation order", the number is just > incremented when a new entry is encountered. You can't know ahead of > time how many new entries you will encounter, so you can't easily > "reverse" this process (it is possible though). > > > (info "(gnus) Back End Interface") uses the words 'later' and 'ord= er of > > arrival'. > > > WDYT? > > I don't think it's really possible to know the order in which entries > arrived in a feed. I've seen feeds with posts in chronological, > reverse-chronological and (seemingly) random orders. Each entry in each > feed might contain a date and an "update" date (in Atom feeds). IME the > relation between these dates and the dates the corresponding posts were > actually published or updated is non-existent. > I still think your best bet is to just sort by date manually. OK. I think I'll use the datetime; for the time being at least. Regards, James From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 09 17:42:28 2024 Received: (at 64202) by debbugs.gnu.org; 9 Feb 2024 22:42:28 +0000 Received: from localhost ([127.0.0.1]:44285 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rYZZj-0001zn-V8 for submit@debbugs.gnu.org; Fri, 09 Feb 2024 17:42:28 -0500 Received: from dsemy.com ([2a03:6000:6e64:605::208]:41154) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rYZZe-0001zG-03 for 64202@debbugs.gnu.org; Fri, 09 Feb 2024 17:42:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=MMkdKgeU/er+U NGdKtbw6MmGG5BkuS2EFitn7Ww7Q8M=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=O66l4DcJnt0eL9WtwigATu1ymw9SybO/hvz 4KJvyP1u1hgMpvNGBbiNkE9cZIFl2nlTAOnKuW3T+q0hN8xQyhB9anOBLGeWVwXQBL+hI+ VRVuN053BR2SgV5xSdJgZKW1xNl/C5QNCoe8JYHraHnWQFCpZT5XnIGSUatePI+JXLhCbT 8N1F7eOaOhtWeNeKJ1zWnOkt5UMTYgSMxzQ/jLjuyV1iZ2puW7o4R21dEdejhk7lR7l3uy xz4NbrQ2olOtodlf8Jg84uL2N0k0s8XPgBkoz7iRbvPymkUUCVjiq7DBjrrTuODcxJf6hP vOKxya2J1ahxvsN0t4V3ZWdh32g== Received: from coldharbour.local ( [147.235.215.121]) by dsemy.com (OpenSMTPD) with ESMTPSA id f476f84a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 9 Feb 2024 23:41:53 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id 1b17a4ac; Fri, 9 Feb 2024 22:34:37 +0000 (UTC) From: Daniel Semyonov To: James Thomas Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87wmrd9zcy.fsf@outlook.com> (James Thomas's message of "Sat, 10 Feb 2024 01:36:21 +0530") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> Date: Sat, 10 Feb 2024 00:34:37 +0200 Message-ID: <875xyx5ksi.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 3.5 (+++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: >>>>> James Thomas writes: >> If the posts appear in a chronological order, it should end up >> being correct. I think what's happening is that the feed you're >> testing actually uses a reverse chronological order for posts. [...] Content analysis details: (3.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.215.121 listed in zen.spamhaus.org] 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 2.5 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: >>>>> James Thomas writes: >> If the posts appear in a chronological order, it should end up >> being correct. I think what's happening is that the feed you're >> testing actually uses a reverse chronological order for posts. [...] Content analysis details: (2.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.215.121 listed in zen.spamhaus.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager >>>>> James Thomas writes: >> If the posts appear in a chronological order, it should end up >> being correct. I think what's happening is that the feed you're >> testing actually uses a reverse chronological order for posts. >> If not, send me the feed, I might be misunderstanding something. > Hmm.. It seems to be happening that way for every server I used: > git.savannah.gnu.org/cgit/emacs.git/atom/?h=master > rss.slashdot.org/Slashdot/slashdotMainatom:Slashdot > github.com/emacs-mirror/emacs/commits.atom These feeds (except for the second one which I can't access) are in reverse-chronological order. From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 09 19:25:17 2024 Received: (at 64202) by debbugs.gnu.org; 10 Feb 2024 00:25:17 +0000 Received: from localhost ([127.0.0.1]:51536 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rYbBE-0006wf-4d for submit@debbugs.gnu.org; Fri, 09 Feb 2024 19:25:16 -0500 Received: from mout.gmx.net ([212.227.17.21]:53741) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rYb2M-0006VS-8e for 64202@debbugs.gnu.org; Fri, 09 Feb 2024 19:16:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=s31663417; t=1707524143; x=1708128943; i=jimjoe@gmx.net; bh=pDKq918d/d8EsiH1mi9xFPCTeRK3YbgY6LnVzCN3pu0=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References: Date; b=ndoTfRDgpf5R4tKSGifnKRreeld892NizDVP4sWhgwh20gUAhu1jmIezxQApfIzT okbYBoqVV6eCmgTlzKpmilpk/gCQP/66oX8q+ujlixGoJWJJznB0q6XDwA7f8b0OM S9FduYDgs6x1v9ZXwZjgdYNeX4WcuXIyKJoSSs3sr6lgqfLgwEOYC8VNuCX+hlzf5 0eyaAsvf92zGRJhbunNTX0JcsrrruBZIaRZbJt0aEnlFXbMKIq1wTXw7c/gM3wphL qqAhM2iviiHxTdJIq6FH4l9BgHY/5mhC/D9CNpxEveUoFrB4NfAXRimKlqfdkEVaq C2I72TCK7dUZ7gn3PQ== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from user-Inspiron-15-5518 ([117.194.147.10]) by mail.gmx.net (mrgmx104 [212.227.17.174]) with ESMTPSA (Nemesis) id 1MXGr8-1rUkDo3SPO-00Yh7T; Sat, 10 Feb 2024 01:15:43 +0100 From: James Thomas To: Daniel Semyonov Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <875xyx5ksi.fsf@dsemy.com> (Daniel Semyonov's message of "Sat, 10 Feb 2024 00:34:37 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> Date: Sat, 10 Feb 2024 05:45:40 +0530 Message-ID: <87sf219ntf.fsf@gmx.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:iyMhlUQMQ//vi3gbdngVFusUK7ezhVDqN5ATgYIMs/Vx/DwxK2x qaBuKWptvKZIZ+LgvETlZ2nh0HXHJzeLJQQkC/mv4QOdKIk1lJQ0qP2tpompbReRX/6XJrp k+nRUrGAMQthqhDXie+GxiFP8l6pi+6FtfhEADBYwJaqdKK+eA3pHMrDsEgju1HK1m2m6RU /AKBUwLjmEdd8Xpfv3L1A== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:txekTtEYtjg=;Mfa9z1HGt3FZKgwdprwImav24Tm vk7rcukjLXBLz+KDx9lXMrlMUnV0Gd1pbA82K7hwHoDjOGTjEz4tPnKAv0rxC1OgSCHrhdlCR ONJOF3QEkBweElCCy9ojHAA4Mx/PAzYqK7A4y2J8ENCr4hUIjV1GT66ZSTAN9cddqh1mTGDOY 4Ds25BZ5PCzOQmcR5BtvjYj2+vKKzhZmJifKQX2OtXH3fy6VvWHZRlBdYi0r2drFvs5U3qsGS sIbgP5Hm7g2D4ITO0rvJAIBpe/8dh9NYi8n6mfUhsEzKjbuR/Y4euHRPrazKFjEsVHOFxNLul 973gncf+RYwxZbap1A3h+Pu0QFIRxakYnwlaBoPG2aXi9rSmPnP2+etga+6AdSB78XU6TEWpB sYGMpkb3697XU59ld8QiytXGIuMgnxu640JqATJfN/da6dFoa7GMKuPaSktpyl9TCJfOqFfQA wGNA0o2KsJdwrumnE5WbJ+Lqc9IMmCjEWFExwMMdqNx6S07OeBvUwFtBKrFG8i+LxFuHPOohC 8MCWoDb5Sg0Ce79d4H7YkeJNB/lk+XvlvJqT3exTO2Qu0JZ/1H8m266d5ZaJvPExbtnFOB2D3 bDy0XU5Nd4FQjP6D652NiuyCuwavZqssyZHC4gQdbNe9oo6R94vaWzGjBHCxSrnh6LY/kvToe WkfxZu/6YTHts4wJq2PZ1LO5ihhsjLS7OHyjJ+GNutHDjX2vXjBSbfUoQhmI/0dgaZ4oQ3E0y nzDfP19vYMyR8J5U8Bj6Ab0756FE2jSvWkDl20opUHJ2YSs1zBkq8jLr0HGNVm29Gcv6yFc/f jO5mdlG4HZOorvbcFit9Ujm7RSZOVtAfB5T2hHG0OlSzE= Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Daniel Semyonov wrote: >>>>>> James Thomas writes: > > >> If the posts appear in a chronological order, it should end up > >> being correct. I think what's happening is that the feed you're > >> testing actually uses a reverse chronological order for posts. > >> If not, send me the feed, I might be misunderstanding something. > > > Hmm.. It seems to be happening that way for every server I used: > > > git.savannah.gnu.org/cgit/emacs.git/atom/?h=3Dmaster > > > rss.slashdot.org/Slashdot/slashdotMainatom:Slashdot > > > github.com/emacs-mirror/emacs/commits.atom > > These feeds (except for the second one which I can't access) are in > reverse-chronological order. Right. I was hoping that that could be made the default. But I think the patch is fine to be included as it is; from my 1-2 weeks of testing. Cheers, James From debbugs-submit-bounces@debbugs.gnu.org Wed Feb 21 21:18:08 2024 Received: (at 64202) by debbugs.gnu.org; 22 Feb 2024 02:18:08 +0000 Received: from localhost ([127.0.0.1]:37290 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rcyf1-0007bY-Og for submit@debbugs.gnu.org; Wed, 21 Feb 2024 21:18:08 -0500 Received: from mout.gmx.net ([212.227.15.15]:49233) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rcyey-0007ar-C4 for 64202@debbugs.gnu.org; Wed, 21 Feb 2024 21:18:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=s31663417; t=1708568244; x=1709173044; i=jimjoe@gmx.net; bh=riMBAX5/2NyjqE+R945Wjr0vAc80O3WBAZLbkXiI14U=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References: Date; b=jSJpfPAQ6RJntTZom1b9MOwi59SlPU9kE72hlg1gAAIn+uHEe056k6M2XJB9anSK Lx4v2i+wfmMyR9A0hqarz+Giet1CQvL+l0ipdYgrBu6PwehhvdgDMbwUWCuQlE3De qCe+ZRsOlKkazDLKzOrrqBZXLiEF52hkeZ18vONy/Zz6Msd8wCFfVe7OiKS8mOx14 c7nEo+u07QPN2f2HtBlyyC5HxI1vpwovpUk1daOqwAWxRStvkLmTJ4+zjzcSHY6f0 6Tp5A+O+H2/xtnSfU9wB8FAJDK/XypbG1gGwcPNGzILhEjpZqDEca83jNrODMg8eV asl6kYmO5702wttSkg== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from user-Inspiron-15-5518 ([59.92.162.6]) by mail.gmx.net (mrgmx005 [212.227.17.184]) with ESMTPSA (Nemesis) id 1M1psI-1rfAVf1Q2i-002E09; Thu, 22 Feb 2024 03:17:23 +0100 From: James Thomas To: Daniel Semyonov Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87sf219ntf.fsf@gmx.net> (James Thomas's message of "Sat, 10 Feb 2024 05:45:40 +0530") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> Date: Thu, 22 Feb 2024 07:47:15 +0530 Message-ID: <87msrtkztg.fsf@outlook.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:YatBzcj0onUf8Xe2qiMe3PtLqxflv5rXT2l3hJFch72LBKQ442S m3MwceyIARon1iSLcZMKeTOdk4uq4JVN1b+zYrOA2JWqLL8fPCVs5011ttL36Yikzg8Mm5N QW8IUuUyAbbr1to6Wl5umzta9qnTANN4DYCGSfiMDyh4qG5Nnna63rwI07dLSd6WMFH75j8 6nIYBPNTWZHb852/xpofg== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:XTS/4Imsf7g=;Q4JXXyd0GLkYc7pdVoNrmmSTqgP sQ8ioqns/wi4OuFOB9DWaL7j//aNl6zUAXkUrFjy4AfKaNKv5qynGo/7jRwFbX9KQ1VQI31sU tmxF9MIXXKPb3n9C/V/oDiawTDX1BaQfb4VkeB5CEB+mHE6vb1ZL/jPLvy4QS+A7LAv3cOzYj UgUDJi5uNTZ6A6eS48Vyrp8QuW+4fKASLHnzGS9o3PRmRRFWXzA0Po9XtSd4jHlBGQK/cQvzK 6zTY8Ex48VnpwlG/8I6RcOzKf795PXVAMeVTFHh27uS6XOMD/t1XwBssTPsE9wRKb3AQ1gSF7 qFxBpsIGEBtBSu+UAtWiMR+Ti/9VKfRcfc/P3xmLbqTG44Qub0pVlJ/7wt/By6H22gH1HbkJb zeDwkgEk5Br028K2/SZoD200WpeIEmSLdKh7X3QTBqtpQTdW9xPIrnVYMrKvZW5VtovW8eSaR drJCdjbvihUj9ny1yT4JWTj3YOeUEtxMGkPbofgb3UIzg0GaAJ+iM+yoGZfPZBJ0VO+QkNwYt u/CfQsntzSk8bdvfzaxbonAL9TnRBkzKZv7EdlZQcdJ4GHCToIVC36EWb012lcI7+yNjSuZ6M gIajCHH5NKx18sAfKQxpcphqlMn+z0Q/0m0lYKw2Vt5poyOF2PPbqXRYwVX8XKqDY26zefBdx VBWEuzcVjhlSFLZNZIllc5CUCFjxa+OE8z2IlXbSfYRzPV02wKMmufQSpBHr153Y3bq0Xv47Z Y9LTnaF7lqPlRJ/QPjPuP3NrJfRrEjoaAZBFoM5+ggNdgFIzWPTWzHVwDbvcQ/+JeEAMn4fod wfTcI4hEBb0aUbXZwALFspuZmeiIykFNhMA8r8j490qis= Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.6 (--) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.6 (---) James Thomas wrote: > Daniel Semyonov wrote: > >>>>>>> James Thomas writes: >> >> >> If the posts appear in a chronological order, it should end up >> >> being correct. I think what's happening is that the feed you're >> >> testing actually uses a reverse chronological order for posts. >> >> If not, send me the feed, I might be misunderstanding something. >> >> > Hmm.. It seems to be happening that way for every server I used: >> >> > git.savannah.gnu.org/cgit/emacs.git/atom/?h=3Dmaster >> >> > rss.slashdot.org/Slashdot/slashdotMainatom:Slashdot >> >> > github.com/emacs-mirror/emacs/commits.atom >> >> These feeds (except for the second one which I can't access) are in >> reverse-chronological order. > > Right. I was hoping that that could be made the default. > > But I think the patch is fine to be included as it is; from my 1-2 weeks > of testing. > > Cheers, > James This doesn't seem to work for some feeds such as: en.wikipedia.org/w/index.php?title=3DSpecial:RecentChanges&feed=3Datom thehackernews.com/feeds/posts/default In the first one above, is the colon character to blame? Another thing: I've been using your branch also with my addition below for assuming reverse-chronological as a default (Sorry! But it was really bugging me! :-)). diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el index 2fcf59c1441..9f2b9c359aa 100644 =2D-- a/lisp/gnus/nnatom.el +++ b/lisp/gnus/nnatom.el @@ -64,14 +64,16 @@ nnatom--read-feed (when (eq (car data) 'top) (setq data (assq 'feed data))) (dom-add-child-before data auth) - (catch :stop ; Collect feed authors, stop at first en= try. - (dolist (child (cdddr data)) - (let ((tag (car-safe child))) - (if (eq tag 'entry) - (throw :stop data) - (and (or (eq tag 'author) - (eq tag 'contributor)) - (dom-add-child-before auth child))))) + (let ((prev (cddr data))) + (while-let ((next (cdr prev)) + (child (car-safe next)) + (tag (car-safe child)) + (_ (not (eq tag 'entry)))) + (and (or (eq tag 'author) + (eq tag 'contributor)) + (dom-add-child-before auth child)) + (setq prev next)) + (setcdr prev (nreverse (cdr prev))) data))))))) (defvoo nnatom-read-feed-function #'nnatom--read-feed nil nnfeed-read-feed-function) My case for such a change: - Reverse-chronological makes sense as a default because a feed link only guarantees to provide the latest updates, not an unbroken chain of all events since the beginning. - nnrss.el does it that way: see the 'nreverse' in line #643. - Performance impact should be minimal because nreverse is O(n) and perhaps having to sort by date afterward (rather than simply use the existing number) would more than compensate for that. - Running -catchup-to-here (on a datetime-sorted Summary buffer) will only have to record a single article number range in the newsrc. (My code is probably not as beautiful as many would like it to be) Would like to know your thoughts. Regards, James From debbugs-submit-bounces@debbugs.gnu.org Thu Feb 22 12:25:14 2024 Received: (at 64202) by debbugs.gnu.org; 22 Feb 2024 17:25:14 +0000 Received: from localhost ([127.0.0.1]:43460 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rdCoq-0001OK-Pk for submit@debbugs.gnu.org; Thu, 22 Feb 2024 12:25:14 -0500 Received: from dsemy.com ([46.23.89.208]:6314) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rdCiJ-00011I-Je for 64202@debbugs.gnu.org; Thu, 22 Feb 2024 12:18:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=2Zd4IAZ7wX1+a W4Ef3lFtCG21bYfodqWR1BJAxwGF/0=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=SmBHw6XkBECjffAtJ+08g6VcMbOz19GEhA1 WME6JtGt8uosNPtFmOX+pUuC5/Q5QOFWFoW3jkns82s5qNU1lK3SQLXxmKR2Eirus/pkjo ynPJG+9OyXCfO0GC1iSflAfhWfNBNy3hXrFKcfotSfgF/gmcsoUWbSpm55bG03bEfGBJwH 45242JH/F47vFTE5p/lCgaYffctcohSzOsF29zXe6qKnAq9TKCiIXbZ7UAVnLu9ePPM+vh BALVZF4QWkrDtd75cNQiaIEwwAu3AWAIbI3IMfzet7COF16Ndf0jn23MEsqtGuXH66d7zY JRJx0SH/arRykPI5+5SDa1AV43g== Received: from coldharbour.local ( [147.235.213.103]) by dsemy.com (OpenSMTPD) with ESMTPSA id 82936819 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 22 Feb 2024 18:17:57 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id 8f95c0e0; Thu, 22 Feb 2024 17:10:57 +0000 (UTC) From: Daniel Semyonov To: James Thomas Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87msrtkztg.fsf@outlook.com> (James Thomas's message of "Thu, 22 Feb 2024 07:47:15 +0530") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> Date: Thu, 22 Feb 2024 19:10:57 +0200 Message-ID: <87ttm0mnku.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -1.9 (-) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.9 (--) >>>>> James Thomas writes: > This doesn't seem to work for some feeds such as: > en.wikipedia.org/w/index.php?title=Special:RecentChanges&feed=atom > thehackernews.com/feeds/posts/default > In the first one above, is the colon character to blame? Yes. I'm planning to fix this (this is a Gnus limitation), but I have a a few too many work in progress projects at the moment. The second feed is an RSS feed. > Another thing: I've been using your branch also with my addition below > for assuming reverse-chronological as a default (Sorry! But it was > really bugging me! :-)). > > My case for such a change: > - Reverse-chronological makes sense as a default because a feed link > only guarantees to provide the latest updates, not an unbroken chain > of all events since the beginning. > - nnrss.el does it that way: see the 'nreverse' in line #643. > - Performance impact should be minimal because nreverse is O(n) and > perhaps having to sort by date afterward (rather than simply use the > existing number) would more than compensate for that. > - Running -catchup-to-here (on a datetime-sorted Summary buffer) will > only have to record a single article number range in the newsrc. > (My code is probably not as beautiful as many would like it to be) Honestly, your code is much more beautiful than the code it replaces. > Would like to know your thoughts. I think your arguments make sense, and you solved the issue in a way I didn't consider (and is much simpler than what I had in mind). I'll install this change (probably add a comment explaining the rationale too, since I was hard to convince) when I get home. Thanks, Daniel From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 23 11:34:43 2024 Received: (at 64202) by debbugs.gnu.org; 23 Feb 2024 16:34:43 +0000 Received: from localhost ([127.0.0.1]:52201 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rdYVW-0004Kc-EG for submit@debbugs.gnu.org; Fri, 23 Feb 2024 11:34:42 -0500 Received: from dsemy.com ([46.23.89.208]:30236) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rdYVU-0004K8-81 for 64202@debbugs.gnu.org; Fri, 23 Feb 2024 11:34:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=YvfxJaVaWhRtK 9VAEVIn8O9ayCMa93cjGgrOjLZBwtU=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=Si9s809rwZGbku2tSxBeqd0jp51I4rV9evY bvGkNo4D4O/ytZd1XPhlILkODUMiPL2c050c1hmmibwhYBL3dISvh0NA+3Q9z+rINvcadG BwLx0qlhPrHvaux3uPFBbvKERSaZflivvr5KQRhaMDZwZdkoKOiQHJwyoN7nNgP7hAlOOO lB7vyvfC5+GSMOKcVjlhbXYUld2SBM4c4ZSnJIjZdX7zpB2nIONNmJOKTl9TESbg6qfOTp c9byeuKuIyUZtZ5KrIUQ+vIEsX3zz2FjUx6n2A++bGd3zVPVlr9Qq64M6qxo3IR2thNxc3 q5g6H7q/vQnoXfQwdQEvnbgGf1A== Received: from coldharbour.local ( [147.235.213.103]) by dsemy.com (OpenSMTPD) with ESMTPSA id 648636cb (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 23 Feb 2024 17:34:09 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id 0bcbb1c6; Fri, 23 Feb 2024 16:27:20 +0000 (UTC) From: Daniel Semyonov To: James Thomas Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87ttm0mnku.fsf@dsemy.com> (Daniel Semyonov's message of "Thu, 22 Feb 2024 19:10:57 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> Date: Fri, 23 Feb 2024 18:27:20 +0200 Message-ID: <87o7c7b0yf.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >>>>> Daniel Semyonov writes: > I'll install this change (probably add a comment explaining the > rationale too, since I was hard to convince) when I get home. I installed the change and also took the chance to make this function and a few others more readable. Please let me know if you have any more suggestions or find more issues. Daniel From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 24 09:50:43 2024 Received: (at 64202) by debbugs.gnu.org; 24 Feb 2024 14:50:43 +0000 Received: from localhost ([127.0.0.1]:32929 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rdtMR-0002Qw-8U for submit@debbugs.gnu.org; Sat, 24 Feb 2024 09:50:43 -0500 Received: from mout.gmx.net ([212.227.17.22]:38943) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rdtMP-0002Qc-3Y for 64202@debbugs.gnu.org; Sat, 24 Feb 2024 09:50:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=s31663417; t=1708786199; x=1709390999; i=jimjoe@gmx.net; bh=G+V2/MYUd02S+rQ8q7a3A1hwj41vaYd8hS2MbHW68wo=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:Date: References; b=pPCvv4HwCVqhjC80sveyhTVUjjH22+DrVZMMZkZuGFQg+dhdgaA5us4uyE2G+Oau XBMBh8BtXQbAVFW29jELYJAy13nH8R3VoFzegr3iCgQc/sKOeIQbQg3huEIoaLd5B L+uazlW6PjvhyQQ758AF0pShoi06/V/5QqC4QwwnvGRlTCBl0YYSlCzipTelJ6H9E 2CjAEQ+KWQDh5YMks9WYp7j+0/p+5FZjZsLyLjlfjYUrwI8/yi33CilvMVTN4X1Z5 /jVVb3yla8FaF8M48Cl/uraofkyMwltYMud2AMCDeeAWo8WRr6kS3OPU7o+TGSH13 K7fRK+L9qtc/+ck6Jg== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from user-Inspiron-15-5518 ([117.194.144.213]) by mail.gmx.net (mrgmx105 [212.227.17.174]) with ESMTPSA (Nemesis) id 1MQeA2-1rIX0S29Li-00Nj3u; Sat, 24 Feb 2024 15:49:59 +0100 From: James Thomas To: Daniel Semyonov Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87o7c7b0yf.fsf@dsemy.com> (Daniel Semyonov's message of "Fri, 23 Feb 2024 18:27:20 +0200") Date: Sat, 24 Feb 2024 20:19:46 +0530 Message-ID: <87le79yl11.fsf@gmx.net> References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:cPzjSFGez5ZzY2bxIJZYUXE4+35bE+z51BMQLcVaxbixITJnmA/ EEtcbfRiQxFgWzo3YC7dXsdinjhif6l0aOC2xtZoBCFI2xxX3DLNI8UcncEh2uu1/hdnTn4 3PjH+N7t70Q1BfF709N7VLjayfz0+eHenEdmqDgukF5UlGaTamhZZbAzcalNO4CQBpWdVlO 31YDhsjgFR3zfLdr7UmIQ== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:tBg6YEYD7Fw=;LjUsyW5unbIfy4LlVBFqKoJVImf v3L3hcs3nPK071BJkGoy9xbbilpdC3zig3Pz4+8LpDVWxIWp3/19RWknAAfhrXIUdLZ6/8sii v7SRXzdSs+eigDw8IhWDbQisCG0ZjumshbK9hSn0gDP4I3ZHcbmN0tFrkCxKZWCHz2qkgUtek eOjdU2Qj13mTi/RC4aINimSAMI8aE3/wMXlNn3BnvnQuJw7B0tYH4O7F4yQVmiQRW0bEi5Rwm 5pvQpDvH45svi77K5jGg70YaXwFMD1FKSNmE8/SJhjZpeTsJFwwV/ql1y5In9b9QWRhebhcd1 MEwNNkOu9SBkJThRX9rh5FYLhBwJmzzxAxcgHUeBBedijCyIBtnsZrrXizlk260i7fOTmFdsX sYZEli6moPwx/ZK84AmGfnL83mA13ivwvLAxgDt2KyFkGFVSuXYgVPN/SFKMJIYu1wldZP5XO 4Ngm3Kytu/GJWwumeVkCUXaGz74RaZ4G7T/uKbjyTasvQtXYh34y6BzmZ3k6HM8dhlLBaGgN/ AtslR4qE9wjOARUJLq1sDRuf0DtKmCzFOdlqJBzUErt0B2usfdMb3VORGs4CkE4Z3oAK2d4XJ Yz2uaAVkM1YljTDR8nEScc5hmbsurngMYkEonNdBTiWGy1Z+8VLcypyLPy6RbQlrrlABMbFck SBib6zKEuE0FEM9PH/Wrbz/4IKeuBDEx6AxOkiLZ4Gp9sxoThSJ4geRN9LTuzrS8Eo7BkFr3I x+OPzb3uLideLMwkGA04GkQeYcSKeBjCNowe0yiiJGA5rBRGCHaZNJdEqVM+uH1PjHXAQFrnp dEHhjB5DRTxFIFHDL0HomHIudp28tjgPDrweouh/PmZ1w= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Daniel Semyonov wrote: >>>>>> Daniel Semyonov writes: > > > I'll install this change (probably add a comment explaining the > > rationale too, since I was hard to convince) when I get home. > > I installed the change and also took the chance to make this function > and a few others more readable. Thanks! > Please let me know if you have any more suggestions or find more issues. Sure thing. Meanwhile, I think it would be better if you could post the entire patchset here so that others may test it easily. Regards, James From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 27 00:10:48 2024 Received: (at 64202) by debbugs.gnu.org; 27 Feb 2024 05:10:48 +0000 Received: from localhost ([127.0.0.1]:47264 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1repjo-0006xE-PE for submit@debbugs.gnu.org; Tue, 27 Feb 2024 00:10:48 -0500 Received: from dsemy.com ([46.23.89.208]:27970) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1repjk-0006wU-6U for 64202@debbugs.gnu.org; Tue, 27 Feb 2024 00:10:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=P0ZA+vbEoaj2p yVPksmA9lDt2XxjQZRWdVLve5z/N7o=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=T5k5hop8zCYrbMJFza0o1l44blPJx8x5qNL hAK+bemvyCOYl4MJ8mRn5j5HIyyZV4pbDXv+Y9QXb3dzeQ8BOjRgfHkupAk6tj4BZiQQee MayWxtR8FNo189LKHM5siLCwu1HptdSI3neiBmJY4q8PGIS2ewIs0nZYqak6SvHpV/YEuK UYmQ9Idp/Dd7S7zkKHZZo8iu51J0rpqKK4xuJv+MCOH4YikNxd+jfh+fdziwIgY1l6ZcHy qGUQvbq37UDOsdFu2o66y6Vkf1dgtfv7zJMBJwgmYJIqsW1jetRQbnWNBS+tXLMDCknyIa vuLOdkOuypIJwgzamhym/IkTEVQ== Received: from coldharbour.local ( [147.235.213.103]) by dsemy.com (OpenSMTPD) with ESMTPSA id cddd3370 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 27 Feb 2024 06:10:07 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id b9892d74; Tue, 27 Feb 2024 05:02:34 +0000 (UTC) From: Daniel Semyonov To: James Thomas Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87le79yl11.fsf@gmx.net> (James Thomas's message of "Sat, 24 Feb 2024 20:19:46 +0530") References: <87v8fhmgvw.fsf@dsemy.com> <8734wcfynr.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> Date: Tue, 27 Feb 2024 07:02:34 +0200 Message-ID: <87sf1eh53p.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 3.6 (+++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: >>>>> James Thomas writes: > Meanwhile, I think it would be better if you could post the entire > patchset here so that others may test it easily. Attached. (with all commits squashed into the first) Content analysis details: (3.6 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.213.103 listed in zen.spamhaus.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. 0.0 T_FILL_THIS_FORM_SHORT Fill in a short form with personal information X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 2.5 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: >>>>> James Thomas writes: > Meanwhile, I think it would be better if you could post the entire > patchset here so that others may test it easily. Attached. (with all commits squashed into the first) Content analysis details: (2.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.213.103 listed in zen.spamhaus.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager --=-=-= Content-Type: text/plain >>>>> James Thomas writes: > Meanwhile, I think it would be better if you could post the entire > patchset here so that others may test it easily. Attached. (with all commits squashed into the first) --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch Content-Description: patch >From 19c65a1f9dc422f311d39e9d28bea41cf2742e38 Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnfeed.el: New file implementing an abstract web feed back end. * lisp/gnus/nnatom.el: New file implementing a back end for Atom feeds. * doc/misc/gnus.texi (Browsing the Web, Back End Interface): * etc/NEWS (Gnus): Document nnatom and nnfeed. --- doc/misc/gnus.texi | 73 +++++ etc/NEWS | 10 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 276 ++++++++++++++++++ lisp/gnus/nnfeed.el | 696 ++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 1056 insertions(+) create mode 100644 lisp/gnus/nnatom.el create mode 100644 lisp/gnus/nnfeed.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 419a5390374..49a85576a37 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -975,6 +976,7 @@ Top * Writing New Back Ends:: Extending old back ends. * Hooking New Back Ends Into Gnus:: What has to be done on the Gnus end. * Mail-like Back Ends:: Some tips on mail back ends. +* Web Feed Back Ends:: Easily defining back ends for web feeds. Various File Formats @@ -17252,6 +17254,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17496,6 +17499,40 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@pxref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds +as Gnus servers (with a single group), as you would with any other +method, by supplying the location of the feed as the server address. +Note, however, that the server address shouldn't be prefixed with +@url{http://} or @url{https://}. + +Features of @code{nnatom} include: + +@itemize @bullet + +@item +Server data is saved per-server in the @file{atom} sub-directory of +@file{gnus-directory}. + +@item +An article part is generated for both the summary and the content for +each entry in the feed. Content of all MIME types should be displayed +correctly through Gnus (as long as they are supported and the feed +specifies a MIME type). + +@item +Article modification and publish dates are tracked, and articles are +updated if changed. + +@end itemize + @node Other Sources @section Other Sources @@ -29997,6 +30034,7 @@ Back End Interface * Writing New Back Ends:: Extending old back ends. * Hooking New Back Ends Into Gnus:: What has to be done on the Gnus end. * Mail-like Back Ends:: Some tips on mail back ends. +* Web Feed Back Ends:: Easily defining back ends for web feeds. @end menu @@ -30769,6 +30807,41 @@ Mail-like Back Ends ("some-group" (34 . 39))) @end example +@node Web Feed Back Ends +@subsection Web Feed Back Ends + +If you want to write a back end for a new type of web feed (RSS, +Atom), or some other type of feed, an ``abstract'' back end +(@code{nnfeed}) exists to enable the creation of such back ends with +minimal knowledge of Gnus. + +@code{nnfeed} defines a generic parser, which uses functions stored in +server variables to parse information from a feed (@code{nnfeed} +itself doesn't actually define any such functions though). + +The data parsed from the feed is stored in server variables (and +stored per-feed in a sub-directory of @option{gnus-directory} whose name +corresponds to the name of the back end). + +A Gnus back end interface is also defined, which uses the data parsed +from the feed. + +Therefore, a new back end only needs to inherit from @code{nnfeed}, +define (fairly) generic parsing functions for the feed type and setup +the required server variables. + +@code{nnfeed} was originally created to support Atom Syndication +Format feeds (@pxref{Atom}), but is very generic (as of writing this, +no standard web feed exists which can meaningfully use all the +features supported): it supports multiple groups contained in a single +feed, it allows for situations when the entire feed can't (or +shouldn't) be read ahead of time and it allows for very advanced +customization of the actual printing of articles from parsed data +(while providing a reasonably powerful default method). + +Further implementation details are available in the documentation +strings of the various @code{nnfeed-*} server variables and +the commentary and other comments of @file{nnfeed.el}. @node Score File Syntax @subsection Score File Syntax diff --git a/etc/NEWS b/etc/NEWS index b4a1c887f2e..214e0f093d5 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -1140,6 +1140,16 @@ This keyword enables the user to install packages using 'package-vc'. ** Gnus ++++ +*** New back end 'nnfeed'. +This allows back end developers to easily create new back ends for web +feeds, as inheriting back ends of 'nnfeed'. + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + *** The 'nnweb-type' option 'gmane' has been removed. The gmane.org website is, sadly, down since a number of years with no prospect of it coming back. Therefore, it is no longer valid to set diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index dab66b60205..bc8819dc967 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1360,6 +1360,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..e8dfa12aff5 --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,276 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format . + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'subr-x)) + +(require 'nnfeed) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'nnfeed) + +(nnoo-declare nnatom nnfeed) + +(nnfeed-define-basic-backend-interface nnatom) + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "\\`https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-name-absolute-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (when-let ((data (if (libxml-available-p) + (libxml-parse-xml-region + (point-min) (point-max)) + (car (xml-parse-region + (point-min) (point-max))))) + (authors (list 'authors))) + (when (eq (car data) 'top) + (setq data (assq 'feed data))) + (dom-add-child-before data authors) + (let ((all (dom-children data))) + (while-let ((rest (cdr all)) + (child (car-safe rest)) + (type (car-safe child)) + ((not (eq type 'entry)))) + (and (or (eq type 'author) + (eq type 'contributor)) + (dom-add-child-before authors child)) + (setq all rest)) + ;; Order of entries is reversed as most Atom feeds + ;; list only the "most recent" entries, in reverse + ;; chronological order. + (setcdr all (nreverse (cdr all)))) + data)))))) +(defvoo nnatom-read-feed-function #'nnatom--read-feed + nil nnfeed-read-feed-function) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) +(defvoo nnatom-read-group-function #'nnatom--read-group + nil nnfeed-read-group-function) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) (setq data (dom-children data))) + ;; Discard any children between/after entries. + (while (and data (not (eq (car-safe (car data)) 'entry))) (pop data)) + (when-let ((article (car data)) + (auths (list 'authors)) (links (list 'links))) + (dom-add-child-before article links) + (dom-add-child-before article auths) + (dolist (child (cddddr article) `(,article . ,(cdr data))) + (pcase (car-safe child) ; Authors and links can appear + ((or 'author 'contributor) ; anywhere in the entry so we + (dom-add-child-before auths child) ; collect them all here to + (dom-add-child-before links child)) ; avoid looping over the + ((or 'link ; entry multiple times later. + (and 'content (guard (assq 'src (dom-attributes child))))) + (dom-add-child-before links child)))))) +(defvoo nnatom-read-article-function #'nnatom--read-article + nil nnfeed-read-article-function) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) +(defvoo nnatom-read-title-function #'nnatom--read-title + nil nnfeed-read-title-function) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) +(defvoo nnatom-read-description-function #'nnatom--read-description + nil nnfeed-read-description-function) + +(defun nnatom--read-article-or-group-authors (article-or-group) + "Return the authors of ARTICLE-OR-GROUP, or nil." + (when-let + ((a (mapconcat + (lambda (author) + (let* ((name (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (or (and name email (format "%s <%s>" name email)) name email))) + (dom-children (dom-child-by-tag article-or-group 'authors)) + ", ")) + ((not (string-blank-p a)))) + a)) +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-authors + nil nnfeed-read-author-function) +(defvoo nnatom-read-group-author-function + #'nnatom--read-article-or-group-authors + nil nnfeed-read-group-author-function) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) +(defvoo nnatom-read-subject-function #'nnatom--read-subject + nil nnfeed-read-subject-function) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) +(defvoo nnatom-read-id-function #'nnatom--read-id + nil nnfeed-read-id-function) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + nil nnfeed-read-publish-date-function) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-update-date-function #'nnatom--read-update + nil nnfeed-read-update-date-function) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((alt 0) (rel 0) (sel 0) (enc 0) (via 0) (aut 0)) + (mapcan + (lambda (link) + (when-let ((l (car-safe link))) + (or + (when-let (((eq l 'content)) + (src (dom-attr link 'src)) + (label (concat "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt))))) + `(((("text/plain") . ,(format "%s: %s\n" label src)) + (("text/html") . ,(format "[%s] " + src label))))) + (when-let (((or (eq l 'author) (eq l 'contributor))) + (name (dom-text (dom-child-by-tag link 'name))) + (name (if (string-blank-p name) + (concat "Author" + (and (< 1 (cl-incf aut)) + (format " %s" aut))) + name)) + (uri (dom-text (dom-child-by-tag link 'uri))) + ((not (string-blank-p uri)))) + `(((("text/plain") . ,(format "%s: %s\n" name uri)) + (("text/html") . ,(format "[%s] " + uri name))))) + (when-let (((eq l 'link)) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf rel)) + (format " %s" rel)))) + ("self" + (concat "More" + (and (< 1 (cl-incf sel)) + (format " %s" sel)))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf enc)) + (format " %s" enc)))) + ("via" + (concat "Source" + (and (< 1 (cl-incf via)) + (format " %s" via)))) + (_ (if-let + ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat + "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt)))))))) + (link (cdr (assq 'href attrs)))) + `(((("text/plain") . ,(format "%s: %s\n" label link)) + (("text/html") . ,(format "[%s] " + link label)))))))) + (dom-children (dom-child-by-tag article 'links))))) +(defvoo nnatom-read-links-function #'nnatom--read-links + nil nnfeed-read-links-function) + +(defun nnatom--read-part (part type) + (let* ((atypes '("html" "plain")) + (mtypes '(("xhtml" . "text/html") ("text" . "text/plain"))) + (xsuff (concat "[+/]xml\\(-\\(dtd\\|external-parsed-entity\\)\\)?\\'" + "\\|^text")) + (part (if (string= type "xhtml") + (with-temp-buffer + (dom-print (dom-child-by-tag part 'div) nil t) + (buffer-substring-no-properties + (point-min) (point-max))) + (dom-text part))) + (type (if (member type atypes) (concat "text/" type) type)) + (type (or (cdr (assoc type mtypes)) type))) + (unless (string-blank-p part) + `(,part (Content-Type . ,(or type (setq type "text/plain"))) + ,(and (not (string-match-p xsuff type)) + '(Content-Transfer-Encoding . "base64")))))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (cdr (assq 'type (dom-attributes summary)))) + (summary (nnatom--read-part summary stype)) + (content (dom-child-by-tag article 'content)) + (ctype (cdr (assq 'type (dom-attributes content)))) + (content (nnatom--read-part content ctype)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push summary parts)) + (push (append content '(links)) parts) + (or st (push summary parts))) + ((setq content (or summary content)) + (push (append content '(links)) parts)) + (t (push '((nil (Content-Type . "text/html") links)) parts))) + parts)) +(defvoo nnatom-read-parts-function #'nnatom--read-parts + nil nnfeed-read-parts-function) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +(provide 'nnatom) + +;;; nnatom.el ends here diff --git a/lisp/gnus/nnfeed.el b/lisp/gnus/nnfeed.el new file mode 100644 index 00000000000..c413be5611f --- /dev/null +++ b/lisp/gnus/nnfeed.el @@ -0,0 +1,696 @@ +;;; nnfeed.el --- Generic feed backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnfeed is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnfeed is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnfeed. If not, see . + +;;; Commentary: + +;; Generic Gnus backend (intended) for implementing backends for web +;; feeds (Atom, RSS). +;; +;; This backend is abstract - it doesn't implement a parser for any +;; specific web feed type, and so can't be used independently. +;; +;; Instead, it implements a generic parser, feed data storage and most +;; of the Gnus backend interface; the intended use for this backend is +;; to be a source of inheritance for backends supporting new web feed +;; types. +;; +;; To implement new backends, use `nnfeed-define-basic-backend-interface': +;; +;; ... +;; (require 'nnfeed) +;; +;; (nnoo-declare nnfoo nnfeed) +;; +;; (nnfeed-define-basic-backend-interface nnfoo) +;; ... +;; [ definitions of parsing functions, see the "Feed parser interface" +;; section for more information. ] +;; +;; (defvoo nnfoo-read-feed-function #'nnfoo--read-feed +;; nil nnfeed-read-feed-function) +;; ... +;; (gnus-declare-backend (symbol-name nnfeed-backend) 'address) +;; +;; (provide 'nnfoo) +;; +;;; Code: + +(eval-when-compile + (require 'subr-x)) + +(require 'gnus) +(require 'nnoo) + +(defgroup nnfeed nil + "Generic feed backend for Gnus." + :group 'gnus) + +(defcustom nnfeed-date-format "%F %X%p" + "Format of displayed dates (see function `format-time-string')." + :type 'string) + +(nnoo-declare nnfeed) + +(defvoo nnfeed-backend nil + "Symbol which identifies this backend.") + +(defvoo nnfeed-status-string nil + "Last status message reported by this backend.") + +;; This macro should be used to define inheriting backends. + +(defmacro nnfeed-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnfeed-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnfeed-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnfeed-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnfeed-servers) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnfeed-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnfeed-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnfeed-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnfeed-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnfeed-open-server server defs ',backend)) + (nnoo-import ,backend (nnfeed)))) + +;;;; Feed parser interface: + +;; The following set of server variables define a parser for a +;; specific web feed type. +;; An inheriting backend doesn't necessarily have to define all of +;; these functions (see the comments below for more information). +;; Apart from this set of variables there is also +;; `nnfeed-print-content-function' which can optionally be defined +;; by an inheriting backend to allow more advanced control over the +;; printing of articles. + +(defvoo nnfeed-read-feed-function #'ignore + "Function returning a Lisp object representing a feed (or part of it). + +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnfeed-read-group-function').") + +(defvoo nnfeed-read-group-function #'ignore + "Function returning a cons cell of a group and remaining data from a feed. + +The returned group can be represented by any Lisp object. +It should accept a single argument, a Lisp object representing a feed +\(as can be returned by this function or `nnfeed-read-feed-function').") + +(defvoo nnfeed-read-article-function #'ignore + "Function returning a cons cell of an article and remaining data from a group. + +The returned article can be represented by any Lisp object. +It should accept two arguments, a Lisp object representing a group +\(as can be returned by this function or `nnfeed-read-group-function'), +and a flag indicating whether the last article was not new or updated.") + +(defvoo nnfeed-read-title-function #'ignore + "Function returning the title of a group (a string). + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +;; Optional. +(defvoo nnfeed-read-description-function #'ignore + "Function returning the description of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +;; Either this function or `nnfeed-read-author-function' is required. +(defvoo nnfeed-read-group-author-function #'ignore + "Function returning the author of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function')..") + +(defvoo nnfeed-read-id-function #'ignore + "Function returning the ID of an article. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-subject-function #'ignore + "Function returning the subject of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-update-date-function' is required. +(defvoo nnfeed-read-publish-date-function #'ignore + "Function returning the publish date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-publish-date-function' is required. +(defvoo nnfeed-read-update-date-function #'ignore + "Function returning the update date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-group-author-function' is required. +(defvoo nnfeed-read-author-function #'ignore + "Function returning the author of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-headers-function #'ignore + "Function returning an alist of article-wide MIME headers. + +Each element of this alist maps a MIME header (a symbol, +i.e. `Content-Type') to its value. As a special case, `:boundary' +maps to a string which will serve as the boundary between article +parts. This must be supplied if a custom boundary is used in a +multipart content type header. The default boundary is \"-_nnfeed_-\", +and is automatically modified to match the name of the back end. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; As mentioned in their docstrings, the last two parsing functions +;; can optionally return any Lisp representation they want, provided +;; an appropriate `nnfeed-print-content-function' is defined. This +;; means they are also not _strictly_ required. + +(defvoo nnfeed-read-links-function #'ignore + "Function returning all links contained in an article. + +With the default `nnfeed-print-content-function', it should return a +list of links, where each link is an alist mapping MIME content types +to links formatted for display in a part of that type. Each content +type may also be a list of content types. +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-parts-function #'ignore + "Function returning an alist associating parts of an article to their headers. + +With the default `nnfeed-print-content-function', each part should be a +string. Otherwise, it can be any Lisp object. The \"headers\" of +each part should be a list where each element is either a cons of a +MIME header (a symbol, i.e. `Content-Type') and its value (a string), +or any other Lisp object. MIME headers will be printed, the rest will +be passed on to `nnfeed-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;;;; Feed data storage: + +;; This section defines the data types used to store feed data, and +;; functions to read and write it. +;; All variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. + +(defvoo nnfeed-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnfeed-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnfeed-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnfeed--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnfeed-backend) + "nn") + (gnus-newsgroup-savable-name server)) + gnus-directory)) + +(defun nnfeed--read-server (server) + "Read SERVER's information from storage." + (if-let ((f (nnfeed--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnfeed-servers)) + (nnheader-report nnfeed-backend "Can't read %s" server))) + +(defun nnfeed--write-server (server) + "Write SERVER's information to storage." + (if-let ((f (nnfeed--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n") + (prin1 s (current-buffer)) + (insert "\n") + t) + t) + (nnheader-report nnfeed-backend "Can't write %s" f))) + +(defun nnfeed--server-address (server) + "Return SERVER's real address." + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnfeed-backend 'address) + (cddr (gnus-server-to-method + (concat + (symbol-name nnfeed-backend) ":" + server))))) + server)) + server)) + +;; This is the function which actually uses the functions defined in +;; the last section. None of those functions are called after parsing +;; a feed. + +(defun nnfeed--parse-feed (feed &optional group) + "Parse FEED into a new or existing server. +Optionally, only parse GROUP." + (let* ((feed (nnfeed--server-address feed)) + (s (or (gethash feed nnfeed-servers) (nnfeed--read-server feed))) + (g (or (and (hash-table-p s) (gethash group s)) `[,group])) + (name group) ; (Maybe) fake name + (group (or (aref g 0) name)) ; Real name + data) + (when (setq data (funcall nnfeed-read-feed-function feed group) + s (or (gethash feed nnfeed-servers) + (make-hash-table :test 'equal))) + (while-let ((cg (or (and name `(,data)) + (funcall nnfeed-read-group-function data))) + (cg (prog1 (car cg) (setq data (cdr cg))))) + (let* ((name (funcall nnfeed-read-title-function cg)) ; Real name + (group (gethash name nnfeed-group-names name)) ; (Maybe) fake name + (info (gnus-get-info + (concat (symbol-name nnfeed-backend) "+" feed ":" group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnfeed-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnfeed-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnfeed-read-article-function cg stale)) + (article (prog1 (car article) (setq cg (cdr article))))) + (when-let ((id (funcall nnfeed-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnfeed-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnfeed-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (setq max (1+ max))))) + (publish (funcall nnfeed-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnfeed-read-author-function article) + group-author group) + ,(or (funcall nnfeed-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnfeed-read-links-function article) + ,(funcall nnfeed-read-parts-function article) + ,(funcall nnfeed-read-headers-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnfeed-servers)))) + +;;;; Gnus backend functions: + +;; The following two sections define a Gnus backend interface based on +;; the parsed data from the last section. +;; All server variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. +;; For more information about these functions see the "Back End +;; Interface" section of the Gnus manual. + +(defvoo nnfeed-group nil + "Name of the current group.") + +(defvoo nnfeed-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnfeed-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS HEADERS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers. +- HEADERS is an alist associating article-wide MIME headers to their value.") + +(defvoo nnfeed-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnfeed-group-article-min-num 1 + "Minimum article number for the current group.") + +(nnoo-define-basics nnfeed) + +(defun nnfeed--current-server-no-prefix () + "Remove the \"+\" prefix from the current server." + (string-remove-prefix (concat (symbol-name nnfeed-backend) "+") + (nnoo-current-server nnfeed-backend))) + +(defun nnfeed--group-data (group &optional server) + "Get parsed data for GROUP from SERVER (or the current server if omitted)." + (let ((s (gethash server nnfeed-servers)) c) + (or (and (hash-table-p s) (gethash group s)) + (and (setq c (nnfeed--current-server-no-prefix)) + (setq s (gethash c nnfeed-servers)) + (hash-table-p s) (gethash group s)) + (catch :stop (maphash (lambda (n s) + (or (string= n server) + (string= n c) + (when-let (((hash-table-p s)) + (g (gethash group s))) + (throw :stop g)))) + nnfeed-servers))))) + +(defun nnfeed-retrieve-article (article group) + "Retrieve headers for ARTICLE from GROUP." + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string "%F %H:%M" (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnfeed-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + `[ nil ,nnfeed-group-article-ids ,nnfeed-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnfeed-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnfeed-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnfeed-backend "Group %s not found" (or group "")))) + +(deffoo nnfeed-open-server (server &optional defs backend) + (let ((backend (or backend 'nnfeed)) + (a (nnfeed--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (or (gethash server nnfeed-servers) + (nnfeed--read-server a))) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash group g nnfeed-group-names))) + s)) + (setq a (nnfeed--server-file a)) + (or s (condition-case _ (make-directory (file-name-parent-directory a) t) + (:success (file-writable-p a)) + (t nil)) + (and (nnoo-close-server nnfeed-backend server) + (nnheader-report + nnfeed-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnfeed-request-close () + (when (hash-table-p nnfeed-servers) + (maphash (lambda (server _) + (nnfeed--write-server + (nnfeed--server-address server))) + nnfeed-servers) + (setq nnfeed-servers (make-hash-table :test 'equal))) + (setq nnfeed-status-string nil) + t) + +;; The default content printing function, which should be suitable for +;; most inheriting backends. + +(defun nnfeed--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, with LINKS possibly added." + (let ((links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and content (format "%s\n\n" content)) + (mapconcat (lambda (link) + (cdr (assoc (cdr (assq 'Content-Type attributes)) link + (lambda (types type) + (if (stringp types) (string= types type) + (member type types)))))) + links))))) + +(defvoo nnfeed-print-content-function #'nnfeed--print-content + "Function returning a single piece of content for an article (a string). + +It should accept three arguments, a part and its attributes (as returned +by `nnfeed-read-parts-function'), and links (as returned by +`nnfeed-read-links-function').") + +(defun nnfeed--print-part (content headers mime links) + "Print part of an article using its CONTENT, HEADERS, and LINKS. +Only HEADERS of a type included in MIME are considered." + (concat + (mapconcat (lambda (header) + (when-let ((m (car-safe header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnfeed-print-content-function content headers links))) + +(deffoo nnfeed-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + (and (setq group nnfeed-group) + `[ nil ,nnfeed-group-article-ids + ,nnfeed-group-articles + ,nnfeed-group-article-max-num + ,nnfeed-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((links (aref a 5)) + (parts (aref a 6)) + (headers (aref a 7)) + (boundary (or (cdr (assq :boundary headers)) + (format "-_%s_-" nnfeed-backend))) + (multi (length> parts 1)) + (mime '( Content-Type Content-Disposition + Content-Transfer-Encoding))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnfeed-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + (if (assq 'MIME-Version headers) "" "MIME-Version: 1.0\n") + (mapconcat (lambda (header) + (unless (keywordp (car header)) + (format "%s: %s\n" (car header) (cdr header)))) + headers) + (if multi + (if (assq 'Content-Type headers) "" + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary)) + (prog1 (nnfeed--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (format "--%s\n%s\n" boundary + (nnfeed--print-part + (car part) (cdr part) mime links))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnfeed-backend "No such article"))) + +(deffoo nnfeed-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (if fast (nnfeed--group-data group server) + (setq server (nnfeed--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnfeed-group group + nnfeed-group-article-ids (aref g 1) + nnfeed-group-articles (aref g 2) + nnfeed-group-article-max-num (aref g 3) + nnfeed-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnfeed-group-article-ids) + nnfeed-group-article-min-num + nnfeed-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnfeed-backend "Group %s not found" group)))) + +(deffoo nnfeed-close-group (group &optional server) + (and (string= group nnfeed-group) + (setq nnfeed-group nil + nnfeed-group-article-ids (make-hash-table :test 'equal) + nnfeed-group-articles (make-hash-table :test 'eql) + nnfeed-group-article-max-num 0 + nnfeed-group-article-min-num 1)) + (setq server (or server (nnfeed--current-server-no-prefix))) + (nnfeed--write-server server)) + +(deffoo nnfeed-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnfeed--parse-feed + (or server (nnfeed--current-server-no-prefix)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnfeed-request-post (&optional _server) + (nnheader-report nnfeed-backend "%s is a read only backend" nnfeed-backend)) + +;;;; Optional back end functions: + +(deffoo nnfeed-retrieve-groups (_groups &optional server) + (nnfeed-request-list (or server (nnfeed--current-server-no-prefix))) + 'active) + +(deffoo nnfeed-request-type (_group &optional _article) + 'unknown) + +(deffoo nnfeed-request-group-description (group &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (nnfeed--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnfeed-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnfeed-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (s (or (gethash server nnfeed-servers) + (and ; Open the server to add it to `nnfeed-servers' + (save-match-data + (nnfeed-open-server + server + (cdr ; Get defs and backend. + (assoc server (cdr (assq 'nnfeed nnoo-state-alist)) + (lambda (car key) + (and (stringp car) + (string-match + (concat + "\\(\\(nn[[:alpha:]]+\\)\\+\\)?" + "\\(" (regexp-quote key) "\\)") + car) + (setq server car))))) + (prog1 (if (stringp (match-string 1 server)) + (intern (match-string 2 server)) + 'nnfeed) + (setq server (match-string 3 server)))) + (gethash server nnfeed-servers))))) + (g (or (nnfeed--group-data group server) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnfeed-group-names) + (remhash group s) + (and (string= group nnfeed-group) + (setq nnfeed-group new-name)) + t)) + +(provide 'nnfeed) + +;;; nnfeed.el ends here -- 2.43.2 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Mar 02 21:41:26 2024 Received: (at 64202) by debbugs.gnu.org; 3 Mar 2024 02:41:26 +0000 Received: from localhost ([127.0.0.1]:39499 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rgbn4-0002I0-L7 for submit@debbugs.gnu.org; Sat, 02 Mar 2024 21:41:26 -0500 Received: from mout.gmx.net ([212.227.17.22]:54975) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rgbn0-0002Hl-8v for 64202@debbugs.gnu.org; Sat, 02 Mar 2024 21:41:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=s31663417; t=1709433635; x=1710038435; i=jimjoe@gmx.net; bh=oaOqSOCpmBvBISfAyBDdZbY2jDYJmv/eo1RsmJ5/s/w=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References: Date; b=CT1LsB2wTkYFXwR9wEiDWUu1YDxXNR2y8Kb0L89I+E5wgsWtTNylfwdm9uBQu9ob GJk395blXHyrBNwiTHKbs0/8ZDvqv8iVxPm72En191JPXC2d6sr7PecSLl32fIkHO saLWQCzrypVNg24EfQLYNX214ACCwQad/9pd3sHL4xJxn4T1C1SsEy9SrSN80ajLG UOV1Y4D2eagP6DlL5JBaN7qh4j7xwg2z3D/YmqrzUvvzWgC2LTiHhombosIGZ2KAb rISOMI7nIdNkL9ri2MEWUosL318YL6u4POwpx4K8s8DiIP0iGGppkKIgOC/4nmKy1 qednGPj67rDQJNRl/A== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from user-Inspiron-15-5518 ([117.216.28.216]) by mail.gmx.net (mrgmx104 [212.227.17.174]) with ESMTPSA (Nemesis) id 1N6bfq-1qnwxJ0rXi-0181uD; Sun, 03 Mar 2024 03:40:35 +0100 From: James Thomas To: Daniel Semyonov Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87sf1eh53p.fsf@dsemy.com> (Daniel Semyonov's message of "Tue, 27 Feb 2024 07:02:34 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> <87sf1eh53p.fsf@dsemy.com> Date: Sun, 03 Mar 2024 08:10:32 +0530 Message-ID: <87plwcca1r.fsf@gmx.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:Ep9u5QQGPSz8CSG0aHQZwjcxFR4dGzbv5Lio1YSfOpPPIoQ5EIA ct3/UQ5lDZkbeVwCS4SAclgvgWOvavaqBcWornv5j/YiO6i4UFSH5z9vQeYWCnyJ6XJzaf+ cndyGXBbV3ZxiC9T8ivdqQ2YDg15oO/NTdfIVgSI92cj2rJVwZdYUWQzeMxg8v9tbp8ce6F jf+aotI61yFE1MtMxC7dQ== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:aJvGz3hH170=;PfDxpPK2MtnsS2f1p8TrptQE/5j ecW4moZ6uxn65DpPudneuSRK+/58YBRtdyShf/pnlNUjdiQHRPMebCGR22ERNZCMu1ZQT26Db ycBasyqZVLr/4nwOojtXLbPouV7ArtWhkttAk87L7xi4kBNkPaYQ74WRIXuAtTx0Mioba0O9P iXCUGBRhGfsxervrsHkpZ0RGz7iSSX5Kjw/s7tKA+J175+sPpKvDtLnl9bC7uqdE/QJCePFzl ukQYXs0nGhVi/tfndfuB+ILK+cYPorlyIsFyw/VGVqjmcBNOkAnd7W3gLJB4OIvFfjREcjAAS BBG+7T6I7ktvA7nuvM/UPVbretI61J/g1vW16eXomgN1JYMwSq4Q4O2YuDdD7lQKuabjQDxTx lCb2YwQJHWH+2DO+wuSQmnF6j3pomqm/d3QqwN5ygvgCb9Tcfek9huXtJyNm+PuXm270loYJ7 WXB06WalHe0eeyP26yKD1DrXQhGphR1Xy7ZnXuvZHp6w+Q+olyWbXMRcah0wuhd8/bpnaNdFa /DCSjemzUfpYAkzDfewVhap0Bqljdd5fbuqCwIcIG16DnidN4u/WN+z5WMO2aY3FxeD1GGTAW FsCSmRfm5nJzakL6GNlCK7LB3yovHtdeIvZx2PxqnLVnrQ+InjwYQt19exyaGRDqzePrvhVXa 6ePywemjg2mhgI2HwemxYI67bPkJE2xNg5D3VN8i3//kLrEP1E6gCXHnCivLOAFcn/gOBbFyc pExeqPAQrjIDwa98cAiik8h1fJ2bxGmuW3O4ZT5dHIlBTnF4wO5tkTN5AO2mFO276eyigs6/L 11Ry8fqi2crgnAqqHc4O7bReeeUdERM0DYzjzSMYW+oB0= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Daniel Semyonov wrote: > Attached. (with all commits squashed into the first) Just FYI: I've posted a patch that gets (info "(gnus) Persistent Articles") working with this: bug#69517. Regards, James From debbugs-submit-bounces@debbugs.gnu.org Mon Mar 18 12:59:17 2024 Received: (at 64202) by debbugs.gnu.org; 18 Mar 2024 16:59:17 +0000 Received: from localhost ([127.0.0.1]:33323 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rmGKQ-0005id-NI for submit@debbugs.gnu.org; Mon, 18 Mar 2024 12:59:17 -0400 Received: from dsemy.com ([46.23.89.208]:30792) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rmGKH-0005hd-Bz; Mon, 18 Mar 2024 12:59:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=/p7VccguQeLqf dYtBvHGP7IcP1uuKQLj9jnPJKbnlEw=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=JI3oQY2/yfFMkmOI37r0M4Rp5yLd4bwqZhn gVJegk0+t794MnMUZe/iJ5Y70qvEihB9GaBKoI9cWw69ZsmzUp/WZ630F6sGziS8zRfaIL W10myEBuSQ3NL4reTYY3tA6NT8GjDYnCwOjm5R6OEP+Tj+su4RjUZ6R1i2h8o4UO/ilftL rCT/U1HaU2cAgbVchKX4MNhXYf3ITJq4/iHmyD9YqaXVKfbJzHpUAgsjpvOCpQMwKlyj6x K4XPQnWH4+3jt8CStSdV6MayBDu6CqJkXRij2XRvl0CH/yWjX8O9SlMjXSGswj9RjVFwI/ UdEGSRzVchsfYG2lzT3io0AcOkg== Received: from coldharbour.local ( [147.235.219.6]) by dsemy.com (OpenSMTPD) with ESMTPSA id 8fe02efb (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 18 Mar 2024 17:58:18 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id 40f45283; Mon, 18 Mar 2024 16:58:06 +0000 (UTC) From: Daniel Semyonov To: James Thomas Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87sf1eh53p.fsf@dsemy.com> (Daniel Semyonov's message of "Tue, 27 Feb 2024 07:02:34 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <87wmtnbptm.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> <87sf1eh53p.fsf@dsemy.com> Date: Mon, 18 Mar 2024 18:58:06 +0200 Message-ID: <87ttl3wk81.fsf_-_@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 3.6 (+++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: I pushed a few more commits (an updated patch is attached) which fix some edge cases with ephemeral groups and simplify some functions. Daniel From 43f22a60a33536cfcfdec30457123d455d027159 Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds [...] Content analysis details: (3.6 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.219.6 listed in zen.spamhaus.org] 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. 0.0 T_FILL_THIS_FORM_SHORT Fill in a short form with personal information X-Debbugs-Envelope-To: 64202 Cc: Eric Abrahamsen , 66188@debbugs.gnu.org, 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 2.5 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: I pushed a few more commits (an updated patch is attached) which fix some edge cases with ephemeral groups and simplify some functions. Daniel From 43f22a60a33536cfcfdec30457123d455d027159 Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds [...] Content analysis details: (2.5 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 3.6 RCVD_IN_SBL_CSS RBL: Received via a relay in Spamhaus SBL-CSS [147.235.219.6 listed in zen.spamhaus.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines -0.0 T_SCC_BODY_TEXT_LINE No description available. -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager --=-=-= Content-Type: text/plain I pushed a few more commits (an updated patch is attached) which fix some edge cases with ephemeral groups and simplify some functions. Daniel --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch Content-Description: patch >From 43f22a60a33536cfcfdec30457123d455d027159 Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnfeed.el: New file implementing an abstract web feed back end. * lisp/gnus/nnatom.el: New file implementing a back end for Atom feeds. * doc/misc/gnus.texi (Browsing the Web, Back End Interface): * etc/NEWS (Gnus): Document nnatom and nnfeed. --- doc/misc/gnus.texi | 73 +++++ etc/NEWS | 10 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 276 ++++++++++++++++++ lisp/gnus/nnfeed.el | 683 ++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 1043 insertions(+) create mode 100644 lisp/gnus/nnatom.el create mode 100644 lisp/gnus/nnfeed.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 419a5390374..49a85576a37 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -975,6 +976,7 @@ Top * Writing New Back Ends:: Extending old back ends. * Hooking New Back Ends Into Gnus:: What has to be done on the Gnus end. * Mail-like Back Ends:: Some tips on mail back ends. +* Web Feed Back Ends:: Easily defining back ends for web feeds. Various File Formats @@ -17252,6 +17254,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17496,6 +17499,40 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@pxref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds +as Gnus servers (with a single group), as you would with any other +method, by supplying the location of the feed as the server address. +Note, however, that the server address shouldn't be prefixed with +@url{http://} or @url{https://}. + +Features of @code{nnatom} include: + +@itemize @bullet + +@item +Server data is saved per-server in the @file{atom} sub-directory of +@file{gnus-directory}. + +@item +An article part is generated for both the summary and the content for +each entry in the feed. Content of all MIME types should be displayed +correctly through Gnus (as long as they are supported and the feed +specifies a MIME type). + +@item +Article modification and publish dates are tracked, and articles are +updated if changed. + +@end itemize + @node Other Sources @section Other Sources @@ -29997,6 +30034,7 @@ Back End Interface * Writing New Back Ends:: Extending old back ends. * Hooking New Back Ends Into Gnus:: What has to be done on the Gnus end. * Mail-like Back Ends:: Some tips on mail back ends. +* Web Feed Back Ends:: Easily defining back ends for web feeds. @end menu @@ -30769,6 +30807,41 @@ Mail-like Back Ends ("some-group" (34 . 39))) @end example +@node Web Feed Back Ends +@subsection Web Feed Back Ends + +If you want to write a back end for a new type of web feed (RSS, +Atom), or some other type of feed, an ``abstract'' back end +(@code{nnfeed}) exists to enable the creation of such back ends with +minimal knowledge of Gnus. + +@code{nnfeed} defines a generic parser, which uses functions stored in +server variables to parse information from a feed (@code{nnfeed} +itself doesn't actually define any such functions though). + +The data parsed from the feed is stored in server variables (and +stored per-feed in a sub-directory of @option{gnus-directory} whose name +corresponds to the name of the back end). + +A Gnus back end interface is also defined, which uses the data parsed +from the feed. + +Therefore, a new back end only needs to inherit from @code{nnfeed}, +define (fairly) generic parsing functions for the feed type and setup +the required server variables. + +@code{nnfeed} was originally created to support Atom Syndication +Format feeds (@pxref{Atom}), but is very generic (as of writing this, +no standard web feed exists which can meaningfully use all the +features supported): it supports multiple groups contained in a single +feed, it allows for situations when the entire feed can't (or +shouldn't) be read ahead of time and it allows for very advanced +customization of the actual printing of articles from parsed data +(while providing a reasonably powerful default method). + +Further implementation details are available in the documentation +strings of the various @code{nnfeed-*} server variables and +the commentary and other comments of @file{nnfeed.el}. @node Score File Syntax @subsection Score File Syntax diff --git a/etc/NEWS b/etc/NEWS index 69e61d91b0e..ecf06015829 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -1190,6 +1190,16 @@ This keyword enables the user to install packages using 'package-vc'. ** Gnus ++++ +*** New back end 'nnfeed'. +This allows back end developers to easily create new back ends for web +feeds, as inheriting back ends of 'nnfeed'. + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + *** The 'nnweb-type' option 'gmane' has been removed. The gmane.org website is, sadly, down since a number of years with no prospect of it coming back. Therefore, it is no longer valid to set diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index dab66b60205..bc8819dc967 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1360,6 +1360,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..e8dfa12aff5 --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,276 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format . + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'subr-x)) + +(require 'nnfeed) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'nnfeed) + +(nnoo-declare nnatom nnfeed) + +(nnfeed-define-basic-backend-interface nnatom) + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "\\`https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-name-absolute-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (when-let ((data (if (libxml-available-p) + (libxml-parse-xml-region + (point-min) (point-max)) + (car (xml-parse-region + (point-min) (point-max))))) + (authors (list 'authors))) + (when (eq (car data) 'top) + (setq data (assq 'feed data))) + (dom-add-child-before data authors) + (let ((all (dom-children data))) + (while-let ((rest (cdr all)) + (child (car-safe rest)) + (type (car-safe child)) + ((not (eq type 'entry)))) + (and (or (eq type 'author) + (eq type 'contributor)) + (dom-add-child-before authors child)) + (setq all rest)) + ;; Order of entries is reversed as most Atom feeds + ;; list only the "most recent" entries, in reverse + ;; chronological order. + (setcdr all (nreverse (cdr all)))) + data)))))) +(defvoo nnatom-read-feed-function #'nnatom--read-feed + nil nnfeed-read-feed-function) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) +(defvoo nnatom-read-group-function #'nnatom--read-group + nil nnfeed-read-group-function) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) (setq data (dom-children data))) + ;; Discard any children between/after entries. + (while (and data (not (eq (car-safe (car data)) 'entry))) (pop data)) + (when-let ((article (car data)) + (auths (list 'authors)) (links (list 'links))) + (dom-add-child-before article links) + (dom-add-child-before article auths) + (dolist (child (cddddr article) `(,article . ,(cdr data))) + (pcase (car-safe child) ; Authors and links can appear + ((or 'author 'contributor) ; anywhere in the entry so we + (dom-add-child-before auths child) ; collect them all here to + (dom-add-child-before links child)) ; avoid looping over the + ((or 'link ; entry multiple times later. + (and 'content (guard (assq 'src (dom-attributes child))))) + (dom-add-child-before links child)))))) +(defvoo nnatom-read-article-function #'nnatom--read-article + nil nnfeed-read-article-function) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) +(defvoo nnatom-read-title-function #'nnatom--read-title + nil nnfeed-read-title-function) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) +(defvoo nnatom-read-description-function #'nnatom--read-description + nil nnfeed-read-description-function) + +(defun nnatom--read-article-or-group-authors (article-or-group) + "Return the authors of ARTICLE-OR-GROUP, or nil." + (when-let + ((a (mapconcat + (lambda (author) + (let* ((name (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (or (and name email (format "%s <%s>" name email)) name email))) + (dom-children (dom-child-by-tag article-or-group 'authors)) + ", ")) + ((not (string-blank-p a)))) + a)) +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-authors + nil nnfeed-read-author-function) +(defvoo nnatom-read-group-author-function + #'nnatom--read-article-or-group-authors + nil nnfeed-read-group-author-function) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) +(defvoo nnatom-read-subject-function #'nnatom--read-subject + nil nnfeed-read-subject-function) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) +(defvoo nnatom-read-id-function #'nnatom--read-id + nil nnfeed-read-id-function) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + nil nnfeed-read-publish-date-function) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-update-date-function #'nnatom--read-update + nil nnfeed-read-update-date-function) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((alt 0) (rel 0) (sel 0) (enc 0) (via 0) (aut 0)) + (mapcan + (lambda (link) + (when-let ((l (car-safe link))) + (or + (when-let (((eq l 'content)) + (src (dom-attr link 'src)) + (label (concat "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt))))) + `(((("text/plain") . ,(format "%s: %s\n" label src)) + (("text/html") . ,(format "[%s] " + src label))))) + (when-let (((or (eq l 'author) (eq l 'contributor))) + (name (dom-text (dom-child-by-tag link 'name))) + (name (if (string-blank-p name) + (concat "Author" + (and (< 1 (cl-incf aut)) + (format " %s" aut))) + name)) + (uri (dom-text (dom-child-by-tag link 'uri))) + ((not (string-blank-p uri)))) + `(((("text/plain") . ,(format "%s: %s\n" name uri)) + (("text/html") . ,(format "[%s] " + uri name))))) + (when-let (((eq l 'link)) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf rel)) + (format " %s" rel)))) + ("self" + (concat "More" + (and (< 1 (cl-incf sel)) + (format " %s" sel)))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf enc)) + (format " %s" enc)))) + ("via" + (concat "Source" + (and (< 1 (cl-incf via)) + (format " %s" via)))) + (_ (if-let + ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat + "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt)))))))) + (link (cdr (assq 'href attrs)))) + `(((("text/plain") . ,(format "%s: %s\n" label link)) + (("text/html") . ,(format "[%s] " + link label)))))))) + (dom-children (dom-child-by-tag article 'links))))) +(defvoo nnatom-read-links-function #'nnatom--read-links + nil nnfeed-read-links-function) + +(defun nnatom--read-part (part type) + (let* ((atypes '("html" "plain")) + (mtypes '(("xhtml" . "text/html") ("text" . "text/plain"))) + (xsuff (concat "[+/]xml\\(-\\(dtd\\|external-parsed-entity\\)\\)?\\'" + "\\|^text")) + (part (if (string= type "xhtml") + (with-temp-buffer + (dom-print (dom-child-by-tag part 'div) nil t) + (buffer-substring-no-properties + (point-min) (point-max))) + (dom-text part))) + (type (if (member type atypes) (concat "text/" type) type)) + (type (or (cdr (assoc type mtypes)) type))) + (unless (string-blank-p part) + `(,part (Content-Type . ,(or type (setq type "text/plain"))) + ,(and (not (string-match-p xsuff type)) + '(Content-Transfer-Encoding . "base64")))))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (cdr (assq 'type (dom-attributes summary)))) + (summary (nnatom--read-part summary stype)) + (content (dom-child-by-tag article 'content)) + (ctype (cdr (assq 'type (dom-attributes content)))) + (content (nnatom--read-part content ctype)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push summary parts)) + (push (append content '(links)) parts) + (or st (push summary parts))) + ((setq content (or summary content)) + (push (append content '(links)) parts)) + (t (push '((nil (Content-Type . "text/html") links)) parts))) + parts)) +(defvoo nnatom-read-parts-function #'nnatom--read-parts + nil nnfeed-read-parts-function) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +(provide 'nnatom) + +;;; nnatom.el ends here diff --git a/lisp/gnus/nnfeed.el b/lisp/gnus/nnfeed.el new file mode 100644 index 00000000000..3a1b22f2a2f --- /dev/null +++ b/lisp/gnus/nnfeed.el @@ -0,0 +1,683 @@ +;;; nnfeed.el --- Generic feed backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnfeed is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnfeed is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnfeed. If not, see . + +;;; Commentary: + +;; Generic Gnus backend (intended) for implementing backends for web +;; feeds (Atom, RSS). +;; +;; This backend is abstract - it doesn't implement a parser for any +;; specific web feed type, and so can't be used independently. +;; +;; Instead, it implements a generic parser, feed data storage and most +;; of the Gnus backend interface; the intended use for this backend is +;; to be a source of inheritance for backends supporting new web feed +;; types. +;; +;; To implement new backends, use `nnfeed-define-basic-backend-interface': +;; +;; ... +;; (require 'nnfeed) +;; +;; (nnoo-declare nnfoo nnfeed) +;; +;; (nnfeed-define-basic-backend-interface nnfoo) +;; ... +;; [ definitions of parsing functions, see the "Feed parser interface" +;; section for more information. ] +;; +;; (defvoo nnfoo-read-feed-function #'nnfoo--read-feed +;; nil nnfeed-read-feed-function) +;; ... +;; (gnus-declare-backend (symbol-name nnfeed-backend) 'address) +;; +;; (provide 'nnfoo) +;; +;;; Code: + +(eval-when-compile + (require 'subr-x)) + +(require 'gnus) +(require 'nnoo) + +(defgroup nnfeed nil + "Generic feed backend for Gnus." + :group 'gnus) + +(defcustom nnfeed-date-format "%F %X%p" + "Format of displayed dates (see function `format-time-string')." + :type 'string) + +(nnoo-declare nnfeed) + +(defvoo nnfeed-backend nil + "Symbol which identifies this backend.") + +(defvoo nnfeed-status-string nil + "Last status message reported by this backend.") + +;; This macro should be used to define inheriting backends. + +(defmacro nnfeed-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnfeed-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnfeed-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnfeed-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnfeed-servers) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnfeed-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnfeed-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnfeed-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnfeed-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnfeed-open-server server defs ',backend)) + (nnoo-import ,backend (nnfeed)))) + +;;;; Feed parser interface: + +;; The following set of server variables define a parser for a +;; specific web feed type. +;; An inheriting backend doesn't necessarily have to define all of +;; these functions (see the comments below for more information). +;; Apart from this set of variables there is also +;; `nnfeed-print-content-function' which can optionally be defined +;; by an inheriting backend to allow more advanced control over the +;; printing of articles. + +(defvoo nnfeed-read-feed-function #'ignore + "Function returning a Lisp object representing a feed (or part of it). + +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnfeed-read-group-function').") + +(defvoo nnfeed-read-group-function #'ignore + "Function returning a cons cell of a group and remaining data from a feed. + +The returned group can be represented by any Lisp object. +It should accept a single argument, a Lisp object representing a feed +\(as can be returned by this function or `nnfeed-read-feed-function').") + +(defvoo nnfeed-read-article-function #'ignore + "Function returning a cons cell of an article and remaining data from a group. + +The returned article can be represented by any Lisp object. +It should accept two arguments, a Lisp object representing a group +\(as can be returned by this function or `nnfeed-read-group-function'), +and a flag indicating whether the last article was not new or updated.") + +(defvoo nnfeed-read-title-function #'ignore + "Function returning the title of a group (a string). + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +;; Optional. +(defvoo nnfeed-read-description-function #'ignore + "Function returning the description of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +;; Either this function or `nnfeed-read-author-function' is required. +(defvoo nnfeed-read-group-author-function #'ignore + "Function returning the author of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function')..") + +(defvoo nnfeed-read-id-function #'ignore + "Function returning the ID of an article. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-subject-function #'ignore + "Function returning the subject of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-update-date-function' is required. +(defvoo nnfeed-read-publish-date-function #'ignore + "Function returning the publish date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-publish-date-function' is required. +(defvoo nnfeed-read-update-date-function #'ignore + "Function returning the update date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-group-author-function' is required. +(defvoo nnfeed-read-author-function #'ignore + "Function returning the author of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-headers-function #'ignore + "Function returning an alist of article-wide MIME headers. + +Each element of this alist maps a MIME header (a symbol, +i.e. `Content-Type') to its value. As a special case, `:boundary' +maps to a string which will serve as the boundary between article +parts. This must be supplied if a custom boundary is used in a +multipart content type header. The default boundary is \"-_nnfeed_-\", +and is automatically modified to match the name of the back end. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; As mentioned in their docstrings, the last two parsing functions +;; can optionally return any Lisp representation they want, provided +;; an appropriate `nnfeed-print-content-function' is defined. This +;; means they are also not _strictly_ required. + +(defvoo nnfeed-read-links-function #'ignore + "Function returning all links contained in an article. + +With the default `nnfeed-print-content-function', it should return a +list of links, where each link is an alist mapping MIME content types +to links formatted for display in a part of that type. Each content +type may also be a list of content types. +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-parts-function #'ignore + "Function returning an alist associating parts of an article to their headers. + +With the default `nnfeed-print-content-function', each part should be a +string. Otherwise, it can be any Lisp object. The \"headers\" of +each part should be a list where each element is either a cons of a +MIME header (a symbol, i.e. `Content-Type') and its value (a string), +or any other Lisp object. MIME headers will be printed, the rest will +be passed on to `nnfeed-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;;;; Feed data storage: + +;; This section defines the data types used to store feed data, and +;; functions to read and write it. +;; All variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. + +(defvoo nnfeed-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnfeed-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnfeed-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnfeed--server-address (server) + "Return SERVER's real address." + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnfeed-backend 'address) + (cddr (gnus-server-to-method + (concat + (symbol-name nnfeed-backend) ":" + server))))) + server)) + server)) + +(defun nnfeed--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnfeed-backend) + "nn") + (gnus-newsgroup-savable-name + (nnfeed--server-address server))) + gnus-directory)) + +(defun nnfeed--read-server (server) + "Read SERVER's information from storage." + (if-let ((f (nnfeed--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnfeed-servers)) + (nnheader-report nnfeed-backend "Can't read %s" server))) + +(defun nnfeed--write-server (server) + "Write SERVER's information to storage." + (if-let ((f (nnfeed--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n") + (prin1 s (current-buffer)) + (insert "\n") + t) + t) + (nnheader-report nnfeed-backend "Can't write %s" f))) + +;; The following function uses the parsing functions defined in the last +;; section to parse a feed (or just one group from it). +;; This is the only place where these parsing functions are used; the Gnus +;; backend interface extracts all required information from the parsed feed. + +(defun nnfeed--parse-feed (feed &optional group) + "Parse GROUP from FEED into a new or existing server. +If GROUP is omitted or nil, parse the entire FEED." + (let* ((feed (nnfeed--server-address feed)) + (s (or (gethash feed nnfeed-servers) (nnfeed--read-server feed) + (make-hash-table :test 'equal))) + (name group) ; (Maybe) fake name (or nil) + (group (aref (gethash group s `[,group]) 0)) ; Real name (or nil) + data) + (when (setq data (funcall nnfeed-read-feed-function feed group)) + (while-let ((cg (or (and name (cons data)) ; `data' is a single group + (funcall nnfeed-read-group-function data))) + (cg (prog1 (car cg) (setq data (cdr cg))))) + (let* ((name (funcall nnfeed-read-title-function cg)) ; Real name + (group (gethash name nnfeed-group-names name)) ; (Maybe) fake name + (info (gnus-get-info + (concat (symbol-name nnfeed-backend) "+" feed ":" group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnfeed-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnfeed-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnfeed-read-article-function cg stale)) + (article (prog1 (car article) (setq cg (cdr article))))) + (when-let ((id (funcall nnfeed-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnfeed-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnfeed-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (setq max (1+ max))))) + (publish (funcall nnfeed-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnfeed-read-author-function article) + group-author group) + ,(or (funcall nnfeed-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnfeed-read-links-function article) + ,(funcall nnfeed-read-parts-function article) + ,(funcall nnfeed-read-headers-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnfeed-servers)))) + +;;;; Gnus backend functions: + +;; The following two sections define a Gnus backend interface based on +;; the parsed data from the last section. +;; All server variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. +;; For more information about these functions see the "Back End +;; Interface" section of the Gnus manual. + +(defvoo nnfeed-group nil + "Name of the current group.") + +(defvoo nnfeed-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnfeed-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS HEADERS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers. +- HEADERS is an alist associating article-wide MIME headers to their value.") + +(defvoo nnfeed-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnfeed-group-article-min-num 1 + "Minimum article number for the current group.") + +(nnoo-define-basics nnfeed) + +(defun nnfeed--current-server-no-prefix () + "Remove the \"+\" prefix from the current server." + (string-remove-prefix (concat (symbol-name nnfeed-backend) "+") + (nnoo-current-server nnfeed-backend))) + +(defun nnfeed--group-data (group server) + "Get parsed data for GROUP from SERVER." + (when-let ((server (nnfeed--server-address server)) + (s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (gethash group s))) + +(defun nnfeed-retrieve-article (article group) + "Retrieve headers for ARTICLE from GROUP." + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string "%F %H:%M" (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnfeed-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + `[ nil ,nnfeed-group-article-ids ,nnfeed-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnfeed-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnfeed-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnfeed-backend "Group %s not found" (or group "")))) + +(deffoo nnfeed-open-server (server &optional defs backend) + (let ((backend (or backend 'nnfeed)) + (a (nnfeed--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (or (gethash a nnfeed-servers) (nnfeed--read-server server))) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash group g nnfeed-group-names))) + s)) + (setq a (nnfeed--server-file server)) + (or s (condition-case _ (make-directory (file-name-parent-directory a) t) + (:success (file-writable-p a)) + (t nil)) + (and (nnoo-close-server nnfeed-backend server) + (nnheader-report + nnfeed-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnfeed-request-close () + (when (hash-table-p nnfeed-servers) + (maphash (lambda (server _) (nnfeed--write-server server)) nnfeed-servers) + (setq nnfeed-servers (make-hash-table :test 'equal))) + (setq nnfeed-status-string nil) + t) + +;; The default content printing function, which should be suitable for +;; most inheriting backends. + +(defun nnfeed--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, with LINKS possibly added." + (let ((links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and content (format "%s\n\n" content)) + (mapconcat (lambda (link) + (cdr (assoc (cdr (assq 'Content-Type attributes)) link + (lambda (types type) + (if (stringp types) (string= types type) + (member type types)))))) + links))))) + +(defvoo nnfeed-print-content-function #'nnfeed--print-content + "Function returning a single piece of content for an article (a string). + +It should accept three arguments, a part and its attributes (as returned +by `nnfeed-read-parts-function'), and links (as returned by +`nnfeed-read-links-function').") + +(defun nnfeed--print-part (content headers mime links) + "Print part of an article using its CONTENT, HEADERS, and LINKS. +Only HEADERS of a type included in MIME are considered." + (concat + (mapconcat (lambda (header) + (when-let ((m (car-safe header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnfeed-print-content-function content headers links))) + +(deffoo nnfeed-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + (and (setq group nnfeed-group) + `[ nil ,nnfeed-group-article-ids + ,nnfeed-group-articles + ,nnfeed-group-article-max-num + ,nnfeed-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((links (aref a 5)) + (parts (aref a 6)) + (headers (aref a 7)) + (boundary (or (cdr (assq :boundary headers)) + (format "-_%s_-" nnfeed-backend))) + (multi (length> parts 1)) + (mime '( Content-Type Content-Disposition + Content-Transfer-Encoding))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnfeed-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + (if (assq 'MIME-Version headers) "" "MIME-Version: 1.0\n") + (mapconcat (lambda (header) + (unless (keywordp (car header)) + (format "%s: %s\n" (car header) (cdr header)))) + headers) + (if multi + (if (assq 'Content-Type headers) "" + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary)) + (prog1 (nnfeed--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (format "--%s\n%s\n" boundary + (nnfeed--print-part + (car part) (cdr part) mime links))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnfeed-backend "No such article"))) + +(deffoo nnfeed-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (if fast (nnfeed--group-data group server) + (setq server (nnfeed--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnfeed-group group + nnfeed-group-article-ids (aref g 1) + nnfeed-group-articles (aref g 2) + nnfeed-group-article-max-num (aref g 3) + nnfeed-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnfeed-group-article-ids) + nnfeed-group-article-min-num + nnfeed-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnfeed-backend "Group %s not found" group)))) + +(deffoo nnfeed-close-group (group &optional server) + (and (string= group nnfeed-group) + (setq nnfeed-group nil + nnfeed-group-article-ids (make-hash-table :test 'equal) + nnfeed-group-articles (make-hash-table :test 'eql) + nnfeed-group-article-max-num 0 + nnfeed-group-article-min-num 1)) + (setq server (or server (nnfeed--current-server-no-prefix))) + (nnfeed--write-server server)) + +(deffoo nnfeed-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnfeed--parse-feed + (or server (nnfeed--current-server-no-prefix)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnfeed-request-post (&optional _server) + (nnheader-report nnfeed-backend "%s is a read only backend" nnfeed-backend)) + +;;;; Optional back end functions: + +(deffoo nnfeed-retrieve-groups (_groups &optional server) + (nnfeed-request-list server) + 'active) + +(deffoo nnfeed-request-type (_group &optional _article) + 'unknown) + +(deffoo nnfeed-request-group-description (group &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (nnfeed--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnfeed-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (s (gethash (nnfeed--server-address server) nnfeed-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnfeed-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (a (nnfeed--server-address server)) + (s (or (gethash a nnfeed-servers) + (and ; Open the server to add it to `nnfeed-servers' + (save-match-data + (nnfeed-open-server + server + (cdr ; Get defs and backend. + (assoc a (cdr (assq nnfeed-backend nnoo-state-alist)) + (lambda (car key) + (and (stringp car) + (string-match + (concat + "\\`\\(\\(nn[[:alpha:]]+\\)\\+\\)?" + (regexp-quote key) "\\'") + car) + (setq server car))))) + (if (match-string 1 server) + (intern (match-string 2 server)) 'nnfeed))) + (gethash a nnfeed-servers)))) + (g (or (nnfeed--group-data group a) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnfeed-group-names) + (remhash group s) + (and (string= group nnfeed-group) + (setq nnfeed-group new-name)) + t)) + +(provide 'nnfeed) + +;;; nnfeed.el ends here -- 2.44.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Mar 24 09:37:00 2024 Received: (at 64202) by debbugs.gnu.org; 24 Mar 2024 13:37:00 +0000 Received: from localhost ([127.0.0.1]:59030 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1roO1w-0002nZ-Db for submit@debbugs.gnu.org; Sun, 24 Mar 2024 09:37:00 -0400 Received: from dsemy.com ([46.23.89.208]:10859) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1roO1r-0002mw-A7; Sun, 24 Mar 2024 09:36:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=Q6qA4KVmuzB6J zQFGVcTJK17Q55q/22TdQfK66Uy0hs=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=aRaQ2lkcj+HnFHN93bNOGjfKPxVwNj0rw1G Zw6VHM7QVcprBv+g10zM1a0dkCiCiJ+WaP3xaHJlI6+BAGYeT/b8hrFsGB3JPpSrKke6SC AH10ZOd6f4s4xsxo8INQxAzmrSdFRybJ33Ohd1QCqtz8WRYfG9J7fy3RpR9n+Jmml43d1e b3brt+1be7d0kK3dkhOIFdmU7zcZbYijOOdzNtdjtDRLdJCYDh9BNEtVBOM5YZajVL5lne K78TKf+LVc7OGVRvSd89dQz7RNmY+Z36MwviQDJgYhYhLoc4VtkgSrBI2bXpONvpZ5qmS3 jyX7km/DAfiojDYdXx0GK2vWPfQ== Received: from coldharbour.local ( [147.235.211.67]) by dsemy.com (OpenSMTPD) with ESMTPSA id 8258bd71 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sun, 24 Mar 2024 14:35:47 +0100 (CET) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id 6a3aedcb; Sun, 24 Mar 2024 13:35:18 +0000 (UTC) From: Daniel Semyonov To: James Thomas Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87ttl3wk81.fsf_-_@dsemy.com> (Daniel Semyonov's message of "Mon, 18 Mar 2024 18:58:06 +0200") References: <87v8fhmgvw.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> <87sf1eh53p.fsf@dsemy.com> <87ttl3wk81.fsf_-_@dsemy.com> Date: Sun, 24 Mar 2024 15:35:18 +0200 Message-ID: <87il1b7nxl.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: Eric Abrahamsen , 66188@debbugs.gnu.org, 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain >>>>> Daniel Semyonov writes: > I pushed a few more commits (an updated patch is attached) which fix > some edge cases with ephemeral groups and simplify some functions. And a few more which fix a typo which caused duplicate groups being created after renaming a group and relaunching Gnus, and fix the "Web Feed Back Ends" subsection in the Gnus manual by making it a subsubsection. Updated patch attached. Daniel --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch Content-Description: patch >From ca54f563b9dd03664225639c42b193a29947d391 Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnfeed.el: New file implementing an abstract web feed back end. * lisp/gnus/nnatom.el: New file implementing a back end for Atom feeds. * doc/misc/gnus.texi (Browsing the Web, Back End Interface): * etc/NEWS (Gnus): Document nnatom and nnfeed. --- doc/misc/gnus.texi | 75 +++++ etc/NEWS | 10 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 276 ++++++++++++++++++ lisp/gnus/nnfeed.el | 683 ++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 1045 insertions(+) create mode 100644 lisp/gnus/nnatom.el create mode 100644 lisp/gnus/nnfeed.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 419a5390374..7a0e6dd79a3 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -975,6 +976,7 @@ Top * Writing New Back Ends:: Extending old back ends. * Hooking New Back Ends Into Gnus:: What has to be done on the Gnus end. * Mail-like Back Ends:: Some tips on mail back ends. +* Web Feed Back Ends:: Easily defining back ends for web feeds. Various File Formats @@ -17252,6 +17254,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17496,6 +17499,40 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@pxref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds +as Gnus servers (with a single group), as you would with any other +method, by supplying the location of the feed as the server address. +Note, however, that the server address shouldn't be prefixed with +@url{http://} or @url{https://}. + +Features of @code{nnatom} include: + +@itemize @bullet + +@item +Server data is saved per-server in the @file{atom} sub-directory of +@file{gnus-directory}. + +@item +An article part is generated for both the summary and the content for +each entry in the feed. Content of all MIME types should be displayed +correctly through Gnus (as long as they are supported and the feed +specifies a MIME type). + +@item +Article modification and publish dates are tracked, and articles are +updated if changed. + +@end itemize + @node Other Sources @section Other Sources @@ -29997,6 +30034,7 @@ Back End Interface * Writing New Back Ends:: Extending old back ends. * Hooking New Back Ends Into Gnus:: What has to be done on the Gnus end. * Mail-like Back Ends:: Some tips on mail back ends. +* Web Feed Back Ends:: Easily defining back ends for web feeds. @end menu @@ -30770,6 +30808,43 @@ Mail-like Back Ends @end example +@node Web Feed Back Ends +@subsubsection Web Feed Back Ends + +If you want to write a back end for a new type of web feed (RSS, +Atom), or some other type of feed, an ``abstract'' back end +(@code{nnfeed}) exists to enable the creation of such back ends with +minimal knowledge of Gnus. + +@code{nnfeed} defines a generic parser, which uses functions stored in +server variables to parse information from a feed (@code{nnfeed} +itself doesn't actually define any such functions though). + +The data parsed from the feed is stored in server variables (and +stored per-feed in a sub-directory of @option{gnus-directory} whose name +corresponds to the name of the back end). + +A Gnus back end interface is also defined, which uses the data parsed +from the feed. + +Therefore, a new back end only needs to inherit from @code{nnfeed}, +define (fairly) generic parsing functions for the feed type and setup +the required server variables. + +@code{nnfeed} was originally created to support Atom Syndication +Format feeds (@pxref{Atom}), but is very generic (as of writing this, +no standard web feed exists which can meaningfully use all the +features supported): it supports multiple groups contained in a single +feed, it allows for situations when the entire feed can't (or +shouldn't) be read ahead of time and it allows for very advanced +customization of the actual printing of articles from parsed data +(while providing a reasonably powerful default method). + +Further implementation details are available in the documentation +strings of the various @code{nnfeed-*} server variables and +the commentary and other comments of @file{nnfeed.el}. + + @node Score File Syntax @subsection Score File Syntax diff --git a/etc/NEWS b/etc/NEWS index 19588fe8eeb..552f814ed34 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -1220,6 +1220,16 @@ This keyword enables the user to install packages using 'package-vc'. ** Gnus ++++ +*** New back end 'nnfeed'. +This allows back end developers to easily create new back ends for web +feeds, as inheriting back ends of 'nnfeed'. + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + *** The 'nnweb-type' option 'gmane' has been removed. The gmane.org website is, sadly, down since a number of years with no prospect of it coming back. Therefore, it is no longer valid to set diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index dab66b60205..bc8819dc967 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1360,6 +1360,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..e8dfa12aff5 --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,276 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format . + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'subr-x)) + +(require 'nnfeed) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'nnfeed) + +(nnoo-declare nnatom nnfeed) + +(nnfeed-define-basic-backend-interface nnatom) + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "\\`https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-name-absolute-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (when-let ((data (if (libxml-available-p) + (libxml-parse-xml-region + (point-min) (point-max)) + (car (xml-parse-region + (point-min) (point-max))))) + (authors (list 'authors))) + (when (eq (car data) 'top) + (setq data (assq 'feed data))) + (dom-add-child-before data authors) + (let ((all (dom-children data))) + (while-let ((rest (cdr all)) + (child (car-safe rest)) + (type (car-safe child)) + ((not (eq type 'entry)))) + (and (or (eq type 'author) + (eq type 'contributor)) + (dom-add-child-before authors child)) + (setq all rest)) + ;; Order of entries is reversed as most Atom feeds + ;; list only the "most recent" entries, in reverse + ;; chronological order. + (setcdr all (nreverse (cdr all)))) + data)))))) +(defvoo nnatom-read-feed-function #'nnatom--read-feed + nil nnfeed-read-feed-function) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) +(defvoo nnatom-read-group-function #'nnatom--read-group + nil nnfeed-read-group-function) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) (setq data (dom-children data))) + ;; Discard any children between/after entries. + (while (and data (not (eq (car-safe (car data)) 'entry))) (pop data)) + (when-let ((article (car data)) + (auths (list 'authors)) (links (list 'links))) + (dom-add-child-before article links) + (dom-add-child-before article auths) + (dolist (child (cddddr article) `(,article . ,(cdr data))) + (pcase (car-safe child) ; Authors and links can appear + ((or 'author 'contributor) ; anywhere in the entry so we + (dom-add-child-before auths child) ; collect them all here to + (dom-add-child-before links child)) ; avoid looping over the + ((or 'link ; entry multiple times later. + (and 'content (guard (assq 'src (dom-attributes child))))) + (dom-add-child-before links child)))))) +(defvoo nnatom-read-article-function #'nnatom--read-article + nil nnfeed-read-article-function) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) +(defvoo nnatom-read-title-function #'nnatom--read-title + nil nnfeed-read-title-function) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) +(defvoo nnatom-read-description-function #'nnatom--read-description + nil nnfeed-read-description-function) + +(defun nnatom--read-article-or-group-authors (article-or-group) + "Return the authors of ARTICLE-OR-GROUP, or nil." + (when-let + ((a (mapconcat + (lambda (author) + (let* ((name (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (or (and name email (format "%s <%s>" name email)) name email))) + (dom-children (dom-child-by-tag article-or-group 'authors)) + ", ")) + ((not (string-blank-p a)))) + a)) +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-authors + nil nnfeed-read-author-function) +(defvoo nnatom-read-group-author-function + #'nnatom--read-article-or-group-authors + nil nnfeed-read-group-author-function) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) +(defvoo nnatom-read-subject-function #'nnatom--read-subject + nil nnfeed-read-subject-function) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) +(defvoo nnatom-read-id-function #'nnatom--read-id + nil nnfeed-read-id-function) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + nil nnfeed-read-publish-date-function) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-update-date-function #'nnatom--read-update + nil nnfeed-read-update-date-function) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((alt 0) (rel 0) (sel 0) (enc 0) (via 0) (aut 0)) + (mapcan + (lambda (link) + (when-let ((l (car-safe link))) + (or + (when-let (((eq l 'content)) + (src (dom-attr link 'src)) + (label (concat "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt))))) + `(((("text/plain") . ,(format "%s: %s\n" label src)) + (("text/html") . ,(format "[%s] " + src label))))) + (when-let (((or (eq l 'author) (eq l 'contributor))) + (name (dom-text (dom-child-by-tag link 'name))) + (name (if (string-blank-p name) + (concat "Author" + (and (< 1 (cl-incf aut)) + (format " %s" aut))) + name)) + (uri (dom-text (dom-child-by-tag link 'uri))) + ((not (string-blank-p uri)))) + `(((("text/plain") . ,(format "%s: %s\n" name uri)) + (("text/html") . ,(format "[%s] " + uri name))))) + (when-let (((eq l 'link)) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf rel)) + (format " %s" rel)))) + ("self" + (concat "More" + (and (< 1 (cl-incf sel)) + (format " %s" sel)))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf enc)) + (format " %s" enc)))) + ("via" + (concat "Source" + (and (< 1 (cl-incf via)) + (format " %s" via)))) + (_ (if-let + ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat + "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt)))))))) + (link (cdr (assq 'href attrs)))) + `(((("text/plain") . ,(format "%s: %s\n" label link)) + (("text/html") . ,(format "[%s] " + link label)))))))) + (dom-children (dom-child-by-tag article 'links))))) +(defvoo nnatom-read-links-function #'nnatom--read-links + nil nnfeed-read-links-function) + +(defun nnatom--read-part (part type) + (let* ((atypes '("html" "plain")) + (mtypes '(("xhtml" . "text/html") ("text" . "text/plain"))) + (xsuff (concat "[+/]xml\\(-\\(dtd\\|external-parsed-entity\\)\\)?\\'" + "\\|^text")) + (part (if (string= type "xhtml") + (with-temp-buffer + (dom-print (dom-child-by-tag part 'div) nil t) + (buffer-substring-no-properties + (point-min) (point-max))) + (dom-text part))) + (type (if (member type atypes) (concat "text/" type) type)) + (type (or (cdr (assoc type mtypes)) type))) + (unless (string-blank-p part) + `(,part (Content-Type . ,(or type (setq type "text/plain"))) + ,(and (not (string-match-p xsuff type)) + '(Content-Transfer-Encoding . "base64")))))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (cdr (assq 'type (dom-attributes summary)))) + (summary (nnatom--read-part summary stype)) + (content (dom-child-by-tag article 'content)) + (ctype (cdr (assq 'type (dom-attributes content)))) + (content (nnatom--read-part content ctype)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push summary parts)) + (push (append content '(links)) parts) + (or st (push summary parts))) + ((setq content (or summary content)) + (push (append content '(links)) parts)) + (t (push '((nil (Content-Type . "text/html") links)) parts))) + parts)) +(defvoo nnatom-read-parts-function #'nnatom--read-parts + nil nnfeed-read-parts-function) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +(provide 'nnatom) + +;;; nnatom.el ends here diff --git a/lisp/gnus/nnfeed.el b/lisp/gnus/nnfeed.el new file mode 100644 index 00000000000..0bf599553e4 --- /dev/null +++ b/lisp/gnus/nnfeed.el @@ -0,0 +1,683 @@ +;;; nnfeed.el --- Generic feed backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnfeed is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnfeed is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnfeed. If not, see . + +;;; Commentary: + +;; Generic Gnus backend (intended) for implementing backends for web +;; feeds (Atom, RSS). +;; +;; This backend is abstract - it doesn't implement a parser for any +;; specific web feed type, and so can't be used independently. +;; +;; Instead, it implements a generic parser, feed data storage and most +;; of the Gnus backend interface; the intended use for this backend is +;; to be a source of inheritance for backends supporting new web feed +;; types. +;; +;; To implement new backends, use `nnfeed-define-basic-backend-interface': +;; +;; ... +;; (require 'nnfeed) +;; +;; (nnoo-declare nnfoo nnfeed) +;; +;; (nnfeed-define-basic-backend-interface nnfoo) +;; ... +;; [ definitions of parsing functions, see the "Feed parser interface" +;; section for more information. ] +;; +;; (defvoo nnfoo-read-feed-function #'nnfoo--read-feed +;; nil nnfeed-read-feed-function) +;; ... +;; (gnus-declare-backend (symbol-name nnfeed-backend) 'address) +;; +;; (provide 'nnfoo) +;; +;;; Code: + +(eval-when-compile + (require 'subr-x)) + +(require 'gnus) +(require 'nnoo) + +(defgroup nnfeed nil + "Generic feed backend for Gnus." + :group 'gnus) + +(defcustom nnfeed-date-format "%F %X%p" + "Format of displayed dates (see function `format-time-string')." + :type 'string) + +(nnoo-declare nnfeed) + +(defvoo nnfeed-backend nil + "Symbol which identifies this backend.") + +(defvoo nnfeed-status-string nil + "Last status message reported by this backend.") + +;; This macro should be used to define inheriting backends. + +(defmacro nnfeed-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnfeed-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnfeed-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnfeed-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnfeed-servers) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnfeed-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnfeed-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnfeed-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnfeed-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnfeed-open-server server defs ',backend)) + (nnoo-import ,backend (nnfeed)))) + +;;;; Feed parser interface: + +;; The following set of server variables define a parser for a +;; specific web feed type. +;; An inheriting backend doesn't necessarily have to define all of +;; these functions (see the comments below for more information). +;; Apart from this set of variables there is also +;; `nnfeed-print-content-function' which can optionally be defined +;; by an inheriting backend to allow more advanced control over the +;; printing of articles. + +(defvoo nnfeed-read-feed-function #'ignore + "Function returning a Lisp object representing a feed (or part of it). + +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnfeed-read-group-function').") + +(defvoo nnfeed-read-group-function #'ignore + "Function returning a cons cell of a group and remaining data from a feed. + +The returned group can be represented by any Lisp object. +It should accept a single argument, a Lisp object representing a feed +\(as can be returned by this function or `nnfeed-read-feed-function').") + +(defvoo nnfeed-read-article-function #'ignore + "Function returning a cons cell of an article and remaining data from a group. + +The returned article can be represented by any Lisp object. +It should accept two arguments, a Lisp object representing a group +\(as can be returned by this function or `nnfeed-read-group-function'), +and a flag indicating whether the last article was not new or updated.") + +(defvoo nnfeed-read-title-function #'ignore + "Function returning the title of a group (a string). + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +;; Optional. +(defvoo nnfeed-read-description-function #'ignore + "Function returning the description of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +;; Either this function or `nnfeed-read-author-function' is required. +(defvoo nnfeed-read-group-author-function #'ignore + "Function returning the author of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function')..") + +(defvoo nnfeed-read-id-function #'ignore + "Function returning the ID of an article. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-subject-function #'ignore + "Function returning the subject of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-update-date-function' is required. +(defvoo nnfeed-read-publish-date-function #'ignore + "Function returning the publish date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-publish-date-function' is required. +(defvoo nnfeed-read-update-date-function #'ignore + "Function returning the update date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-group-author-function' is required. +(defvoo nnfeed-read-author-function #'ignore + "Function returning the author of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-headers-function #'ignore + "Function returning an alist of article-wide MIME headers. + +Each element of this alist maps a MIME header (a symbol, +i.e. `Content-Type') to its value. As a special case, `:boundary' +maps to a string which will serve as the boundary between article +parts. This must be supplied if a custom boundary is used in a +multipart content type header. The default boundary is \"-_nnfeed_-\", +and is automatically modified to match the name of the back end. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; As mentioned in their docstrings, the last two parsing functions +;; can optionally return any Lisp representation they want, provided +;; an appropriate `nnfeed-print-content-function' is defined. This +;; means they are also not _strictly_ required. + +(defvoo nnfeed-read-links-function #'ignore + "Function returning all links contained in an article. + +With the default `nnfeed-print-content-function', it should return a +list of links, where each link is an alist mapping MIME content types +to links formatted for display in a part of that type. Each content +type may also be a list of content types. +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-parts-function #'ignore + "Function returning an alist associating parts of an article to their headers. + +With the default `nnfeed-print-content-function', each part should be a +string. Otherwise, it can be any Lisp object. The \"headers\" of +each part should be a list where each element is either a cons of a +MIME header (a symbol, i.e. `Content-Type') and its value (a string), +or any other Lisp object. MIME headers will be printed, the rest will +be passed on to `nnfeed-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;;;; Feed data storage: + +;; This section defines the data types used to store feed data, and +;; functions to read and write it. +;; All variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. + +(defvoo nnfeed-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnfeed-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnfeed-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnfeed--server-address (server) + "Return SERVER's real address." + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnfeed-backend 'address) + (cddr (gnus-server-to-method + (concat + (symbol-name nnfeed-backend) ":" + server))))) + server)) + server)) + +(defun nnfeed--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnfeed-backend) + "nn") + (gnus-newsgroup-savable-name + (nnfeed--server-address server))) + gnus-directory)) + +(defun nnfeed--read-server (server) + "Read SERVER's information from storage." + (if-let ((f (nnfeed--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnfeed-servers)) + (nnheader-report nnfeed-backend "Can't read %s" server))) + +(defun nnfeed--write-server (server) + "Write SERVER's information to storage." + (if-let ((f (nnfeed--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n") + (prin1 s (current-buffer)) + (insert "\n") + t) + t) + (nnheader-report nnfeed-backend "Can't write %s" f))) + +;; The following function uses the parsing functions defined in the last +;; section to parse a feed (or just one group from it). +;; This is the only place where these parsing functions are used; the Gnus +;; backend interface extracts all required information from the parsed feed. + +(defun nnfeed--parse-feed (feed &optional group) + "Parse GROUP from FEED into a new or existing server. +If GROUP is omitted or nil, parse the entire FEED." + (let* ((feed (nnfeed--server-address feed)) + (s (or (gethash feed nnfeed-servers) (nnfeed--read-server feed) + (make-hash-table :test 'equal))) + (name group) ; (Maybe) fake name (or nil) + (group (aref (gethash group s `[,group]) 0)) ; Real name (or nil) + data) + (when (setq data (funcall nnfeed-read-feed-function feed group)) + (while-let ((cg (or (and name (cons data)) ; `data' is a single group + (funcall nnfeed-read-group-function data))) + (cg (prog1 (car cg) (setq data (cdr cg))))) + (let* ((name (funcall nnfeed-read-title-function cg)) ; Real name + (group (gethash name nnfeed-group-names name)) ; (Maybe) fake name + (info (gnus-get-info + (concat (symbol-name nnfeed-backend) "+" feed ":" group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnfeed-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnfeed-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnfeed-read-article-function cg stale)) + (article (prog1 (car article) (setq cg (cdr article))))) + (when-let ((id (funcall nnfeed-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnfeed-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnfeed-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (setq max (1+ max))))) + (publish (funcall nnfeed-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnfeed-read-author-function article) + group-author group) + ,(or (funcall nnfeed-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnfeed-read-links-function article) + ,(funcall nnfeed-read-parts-function article) + ,(funcall nnfeed-read-headers-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnfeed-servers)))) + +;;;; Gnus backend functions: + +;; The following two sections define a Gnus backend interface based on +;; the parsed data from the last section. +;; All server variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. +;; For more information about these functions see the "Back End +;; Interface" section of the Gnus manual. + +(defvoo nnfeed-group nil + "Name of the current group.") + +(defvoo nnfeed-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnfeed-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS HEADERS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers. +- HEADERS is an alist associating article-wide MIME headers to their value.") + +(defvoo nnfeed-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnfeed-group-article-min-num 1 + "Minimum article number for the current group.") + +(nnoo-define-basics nnfeed) + +(defun nnfeed--current-server-no-prefix () + "Remove the \"+\" prefix from the current server." + (string-remove-prefix (concat (symbol-name nnfeed-backend) "+") + (nnoo-current-server nnfeed-backend))) + +(defun nnfeed--group-data (group server) + "Get parsed data for GROUP from SERVER." + (when-let ((server (nnfeed--server-address server)) + (s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (gethash group s))) + +(defun nnfeed-retrieve-article (article group) + "Retrieve headers for ARTICLE from GROUP." + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string "%F %H:%M" (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnfeed-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + `[ nil ,nnfeed-group-article-ids ,nnfeed-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnfeed-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnfeed-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnfeed-backend "Group %s not found" (or group "")))) + +(deffoo nnfeed-open-server (server &optional defs backend) + (let ((backend (or backend 'nnfeed)) + (a (nnfeed--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (or (gethash a nnfeed-servers) (nnfeed--read-server server))) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash g group nnfeed-group-names))) + s)) + (setq a (nnfeed--server-file server)) + (or s (condition-case _ (make-directory (file-name-parent-directory a) t) + (:success (file-writable-p a)) + (t nil)) + (and (nnoo-close-server nnfeed-backend server) + (nnheader-report + nnfeed-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnfeed-request-close () + (when (hash-table-p nnfeed-servers) + (maphash (lambda (server _) (nnfeed--write-server server)) nnfeed-servers) + (setq nnfeed-servers (make-hash-table :test 'equal))) + (setq nnfeed-status-string nil) + t) + +;; The default content printing function, which should be suitable for +;; most inheriting backends. + +(defun nnfeed--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, with LINKS possibly added." + (let ((links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and content (format "%s\n\n" content)) + (mapconcat (lambda (link) + (cdr (assoc (cdr (assq 'Content-Type attributes)) link + (lambda (types type) + (if (stringp types) (string= types type) + (member type types)))))) + links))))) + +(defvoo nnfeed-print-content-function #'nnfeed--print-content + "Function returning a single piece of content for an article (a string). + +It should accept three arguments, a part and its attributes (as returned +by `nnfeed-read-parts-function'), and links (as returned by +`nnfeed-read-links-function').") + +(defun nnfeed--print-part (content headers mime links) + "Print part of an article using its CONTENT, HEADERS, and LINKS. +Only HEADERS of a type included in MIME are considered." + (concat + (mapconcat (lambda (header) + (when-let ((m (car-safe header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnfeed-print-content-function content headers links))) + +(deffoo nnfeed-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + (and (setq group nnfeed-group) + `[ nil ,nnfeed-group-article-ids + ,nnfeed-group-articles + ,nnfeed-group-article-max-num + ,nnfeed-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((links (aref a 5)) + (parts (aref a 6)) + (headers (aref a 7)) + (boundary (or (cdr (assq :boundary headers)) + (format "-_%s_-" nnfeed-backend))) + (multi (length> parts 1)) + (mime '( Content-Type Content-Disposition + Content-Transfer-Encoding))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnfeed-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + (if (assq 'MIME-Version headers) "" "MIME-Version: 1.0\n") + (mapconcat (lambda (header) + (unless (keywordp (car header)) + (format "%s: %s\n" (car header) (cdr header)))) + headers) + (if multi + (if (assq 'Content-Type headers) "" + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary)) + (prog1 (nnfeed--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (format "--%s\n%s\n" boundary + (nnfeed--print-part + (car part) (cdr part) mime links))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnfeed-backend "No such article"))) + +(deffoo nnfeed-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (if fast (nnfeed--group-data group server) + (setq server (nnfeed--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnfeed-group group + nnfeed-group-article-ids (aref g 1) + nnfeed-group-articles (aref g 2) + nnfeed-group-article-max-num (aref g 3) + nnfeed-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnfeed-group-article-ids) + nnfeed-group-article-min-num + nnfeed-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnfeed-backend "Group %s not found" group)))) + +(deffoo nnfeed-close-group (group &optional server) + (and (string= group nnfeed-group) + (setq nnfeed-group nil + nnfeed-group-article-ids (make-hash-table :test 'equal) + nnfeed-group-articles (make-hash-table :test 'eql) + nnfeed-group-article-max-num 0 + nnfeed-group-article-min-num 1)) + (setq server (or server (nnfeed--current-server-no-prefix))) + (nnfeed--write-server server)) + +(deffoo nnfeed-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnfeed--parse-feed + (or server (nnfeed--current-server-no-prefix)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnfeed-request-post (&optional _server) + (nnheader-report nnfeed-backend "%s is a read only backend" nnfeed-backend)) + +;;;; Optional back end functions: + +(deffoo nnfeed-retrieve-groups (_groups &optional server) + (nnfeed-request-list server) + 'active) + +(deffoo nnfeed-request-type (_group &optional _article) + 'unknown) + +(deffoo nnfeed-request-group-description (group &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (nnfeed--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnfeed-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (s (gethash (nnfeed--server-address server) nnfeed-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnfeed-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (a (nnfeed--server-address server)) + (s (or (gethash a nnfeed-servers) + (and ; Open the server to add it to `nnfeed-servers' + (save-match-data + (nnfeed-open-server + server + (cdr ; Get defs and backend. + (assoc a (cdr (assq nnfeed-backend nnoo-state-alist)) + (lambda (car key) + (and (stringp car) + (string-match + (concat + "\\`\\(\\(nn[[:alpha:]]+\\)\\+\\)?" + (regexp-quote key) "\\'") + car) + (setq server car))))) + (if (match-string 1 server) + (intern (match-string 2 server)) 'nnfeed))) + (gethash a nnfeed-servers)))) + (g (or (nnfeed--group-data group a) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnfeed-group-names) + (remhash group s) + (and (string= group nnfeed-group) + (setq nnfeed-group new-name)) + t)) + +(provide 'nnfeed) + +;;; nnfeed.el ends here -- 2.44.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 06 05:01:38 2024 Received: (at 64202) by debbugs.gnu.org; 6 Apr 2024 09:01:39 +0000 Received: from localhost ([127.0.0.1]:38252 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rt1ve-0007Ch-8I for submit@debbugs.gnu.org; Sat, 06 Apr 2024 05:01:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50032) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rt1vc-0007C4-BN; Sat, 06 Apr 2024 05:01:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rt1vQ-0008LS-6y; Sat, 06 Apr 2024 05:01:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=I53jt6qtQDSZldEAefWFdhCDBac3yt+5fQrgGpsolW0=; b=UoeGaYlrr8MP dMfFinDQu+46s7/0qIr/7403w1xNrAYbrObVnQ2N04fTjOpYMLxDel0Qar7u4oYJFkuI5v1MLuSph dtXABSgEEPPuz+EmNUe5kAJGufbQTF93sHke6Q+95ScGDUFjnMKkUL7Mz4FXUEhH5xh4R6RKxTGXM LrgnVymSsXvJ0lxvhMoFQqvaJwv6CjbviVz9/4+sXpI6lnNNEGP0vMd1qthAStJHXEdD0yYPARp/B O7/UDITzZKtpjMSAWDXx4z3r4czJQnEfIAx+WX7LGaZ9X4pMplB/7xXuolWCkAEVAjJxBZR4RZJFj KFQVLyYg4vZqDMmlQGlKNQ==; Date: Sat, 06 Apr 2024 12:01:19 +0300 Message-Id: <8634ry50gw.fsf@gnu.org> From: Eli Zaretskii To: Daniel Semyonov , eric@ericabrahamsen.net In-Reply-To: <87il1b7nxl.fsf@dsemy.com> (bug-gnu-emacs@gnu.org) Subject: Re: bug#64202: bug#66188: 29.1; Include Atom feed reader References: <87v8fhmgvw.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> <87sf1eh53p.fsf@dsemy.com> <87ttl3wk81.fsf_-_@dsemy.com> <87il1b7nxl.fsf@dsemy.com> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64202 Cc: 66188@debbugs.gnu.org, 64202@debbugs.gnu.org, jimjoe@gmx.net X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ping! Eric, any comments about this? Should we install this? > Cc: Eric Abrahamsen , 66188@debbugs.gnu.org, > 64202@debbugs.gnu.org > Date: Sun, 24 Mar 2024 15:35:18 +0200 > From: Daniel Semyonov via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" > > >>>>> Daniel Semyonov writes: > > > I pushed a few more commits (an updated patch is attached) which fix > > some edge cases with ephemeral groups and simplify some functions. > > And a few more which fix a typo which caused duplicate groups being > created after renaming a group and relaunching Gnus, and fix the "Web > Feed Back Ends" subsection in the Gnus manual by making it a > subsubsection. Updated patch attached. From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 18 05:02:38 2024 Received: (at 64202) by debbugs.gnu.org; 18 Apr 2024 09:02:38 +0000 Received: from localhost ([127.0.0.1]:51179 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rxNf7-0008Gc-KI for submit@debbugs.gnu.org; Thu, 18 Apr 2024 05:02:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47492) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rxNf2-0008Ej-0z; Thu, 18 Apr 2024 05:02:31 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rxNei-0000Xk-S0; Thu, 18 Apr 2024 05:02:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=5JKOsMDxtDdyIAq/AuCQy3X+RHnP+QjZa/hs22bDxR8=; b=bLsqR6z3H8VR J6TYtM21EjoNUdZjOWzkR1STWxyAeaTb7+7VuTys7VnD2bqylON8G9e/bLxRvYmFBLu6bB2lB5RSQ b5fKIOvPdmwRXyvLBqSHdLs4nwx8HloQ2jr/ErKwx5dMgE3hVifuLU6LldSv0oykj7LndQXKqxAWg AT2wh+iMO6KGW4yXdR+mQ4L2A+1CC0fnlvuzyoOHfbs8bx//5yAYk8XL7aOoirittsefueG+hLVjI /7yyDz5kgEtbEg+nzRsrx0VW39SnIf6/gOVvldybT7uVMlgxJwkw0eCt32BoSIVgCajF9cLuHB5Gg yY0Fl52IY2p5rEUswhHA5g==; Date: Thu, 18 Apr 2024 12:01:58 +0300 Message-Id: <868r1bf3i1.fsf@gnu.org> From: Eli Zaretskii To: eric@ericabrahamsen.net In-Reply-To: <8634ry50gw.fsf@gnu.org> (message from Eli Zaretskii on Sat, 06 Apr 2024 12:01:19 +0300) Subject: Re: bug#64202: bug#66188: 29.1; Include Atom feed reader References: <87v8fhmgvw.fsf@dsemy.com> <87zfyj8ojz.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> <87sf1eh53p.fsf@dsemy.com> <87ttl3wk81.fsf_-_@dsemy.com> <87il1b7nxl.fsf@dsemy.com> <8634ry50gw.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64202 Cc: 66188@debbugs.gnu.org, 64202@debbugs.gnu.org, daniel@dsemy.com, jimjoe@gmx.net X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ping! Ping! Eric, any comments? > Cc: 66188@debbugs.gnu.org, 64202@debbugs.gnu.org, jimjoe@gmx.net > Date: Sat, 06 Apr 2024 12:01:19 +0300 > From: Eli Zaretskii > > Ping! Eric, any comments about this? Should we install this? > > > Cc: Eric Abrahamsen , 66188@debbugs.gnu.org, > > 64202@debbugs.gnu.org > > Date: Sun, 24 Mar 2024 15:35:18 +0200 > > From: Daniel Semyonov via "Bug reports for GNU Emacs, > > the Swiss army knife of text editors" > > > > >>>>> Daniel Semyonov writes: > > > > > I pushed a few more commits (an updated patch is attached) which fix > > > some edge cases with ephemeral groups and simplify some functions. > > > > And a few more which fix a typo which caused duplicate groups being > > created after renaming a group and relaunching Gnus, and fix the "Web > > Feed Back Ends" subsection in the Gnus manual by making it a > > subsubsection. Updated patch attached. > > > > From debbugs-submit-bounces@debbugs.gnu.org Sat Apr 20 20:47:09 2024 Received: (at 64202) by debbugs.gnu.org; 21 Apr 2024 00:47:09 +0000 Received: from localhost ([127.0.0.1]:39543 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ryLMJ-0001e0-VL for submit@debbugs.gnu.org; Sat, 20 Apr 2024 20:47:09 -0400 Received: from mail.ericabrahamsen.net ([52.70.2.18]:50070) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ryLMH-0001cQ-1s; Sat, 20 Apr 2024 20:47:06 -0400 Received: from localhost (71-212-21-65.tukw.qwest.net [71.212.21.65]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id A8267FA183; Sun, 21 Apr 2024 00:46:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1713660403; bh=G8nQNBi+llCet0+M/h6TuewbHFBXkunKKyldZOkS8Kg=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=FKKdz/YoYLwdCKPEMsdHHKhmgi07U2RqzBP8AbvCkm8XB1GnIabRCw2pVDMxXbwBo 3TDN9gcb1zOfOdu06duUi4q2y84tPrRIT9Bg4NkX90UrhzVnNbikr/lt+5cn6/X7aD csJQeK0bhEozVihwijLsE39fIY58XatLNMXd7Rj4= From: Eric Abrahamsen To: Eli Zaretskii Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <868r1bf3i1.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 18 Apr 2024 12:01:58 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87ttohxc56.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> <87sf1eh53p.fsf@dsemy.com> <87ttl3wk81.fsf_-_@dsemy.com> <87il1b7nxl.fsf@dsemy.com> <8634ry50gw.fsf@gnu.org> <868r1bf3i1.fsf@gnu.org> Date: Sat, 20 Apr 2024 17:46:42 -0700 Message-ID: <87mspnlez1.fsf_-_@ericabrahamsen.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64202 Cc: 66188@debbugs.gnu.org, 64202@debbugs.gnu.org, daniel@dsemy.com, jimjoe@gmx.net X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Eli Zaretskii writes: > Ping! Ping! Eric, any comments? Yes, let's do it! My only comment right now is that the manual should explicitly list the steps for creating an atom group: "B nnatom RET" and insert URL without protocol. I still have dreams of fixing the group creation code to prompt for the address, but that can wait. Daniel, would you make that addition? Do you have push access to the Emacs repo? Eric From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 21 14:43:32 2024 Received: (at 64202) by debbugs.gnu.org; 21 Apr 2024 18:43:33 +0000 Received: from localhost ([127.0.0.1]:44265 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ryc9w-00007g-Ik for submit@debbugs.gnu.org; Sun, 21 Apr 2024 14:43:32 -0400 Received: from dsemy.com ([46.23.89.208]:21349) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ryc9s-00006P-2v; Sun, 21 Apr 2024 14:43:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=cw4wu3Tc+PHe0 c2nP1NEiOzeInvu7O4db26I4ECQ/z8=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=MPbtJ0fTfnz2rbS+6mzrUF8KOaQWY41k1TI CN6gB/FGLExEe+L1h0lajxTokF9L2RAPlq6wzDLLmDZ+yti5PgUj+yCCxCwsMef2nZ2P4Z 7bdNQfZtX04JP+jCHM1N66rokmC5AnzlOY9PadlwU5+lbo0hJOTdM3dzrdFfEXHaSdez+8 czOl8S/YHj27O8XszObkoFGZonZYkU8QrNLMtWhgmRSSU9dL4yhhVQ3KX+hAbtGrKTjRPh yPTgdZK+VNCY+MBvLYWQ5UCciyO8EHqMIph3//wVLTB+iIrxecrtJbz3oUauHqiMGhK+TD 7iLKWByqV5X9OGeFHFBDaMQ37vA== Received: from coldharbour.local ( [147.235.220.9]) by dsemy.com (OpenSMTPD) with ESMTPSA id 7b2fa0de (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sun, 21 Apr 2024 20:43:01 +0200 (CEST) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id bd6d2ad8; Sun, 21 Apr 2024 18:41:57 +0000 (UTC) From: Daniel Semyonov To: Eric Abrahamsen Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87mspnlez1.fsf_-_@ericabrahamsen.net> (Eric Abrahamsen's message of "Sat, 20 Apr 2024 17:46:42 -0700") References: <87v8fhmgvw.fsf@dsemy.com> <87le83hcm8.fsf@outlook.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> <87sf1eh53p.fsf@dsemy.com> <87ttl3wk81.fsf_-_@dsemy.com> <87il1b7nxl.fsf@dsemy.com> <8634ry50gw.fsf@gnu.org> <868r1bf3i1.fsf@gnu.org> <87mspnlez1.fsf_-_@ericabrahamsen.net> Date: Sun, 21 Apr 2024 21:41:57 +0300 Message-ID: <87sezev9qi.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: Eli Zaretskii , jimjoe@gmx.net, 66188@debbugs.gnu.org, 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >>>>> Eric Abrahamsen writes: > Eli Zaretskii writes: >> Ping! Ping! Eric, any comments? > Yes, let's do it! My only comment right now is that the manual should > explicitly list the steps for creating an atom group: "B nnatom RET" = and > insert URL without protocol. I still have dreams of fixing the group > creation code to prompt for the address, but that can wait. > Daniel, would you make that addition? Does the following text (in the 'Atom' subsection of the Gnus manual) not suffice? The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or local Ato= m feeds as Gnus servers (with a single group), as you would with any other method, by supplying the location of the feed as the server address. Note, however, that the server address shouldn't be prefixed with or . > Do you have push access to the Emacs repo? I do not. Daniel From debbugs-submit-bounces@debbugs.gnu.org Sun Apr 21 23:11:40 2024 Received: (at 64202) by debbugs.gnu.org; 22 Apr 2024 03:11:40 +0000 Received: from localhost ([127.0.0.1]:45317 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ryk5j-0006Tb-Bt for submit@debbugs.gnu.org; Sun, 21 Apr 2024 23:11:40 -0400 Received: from mail.ericabrahamsen.net ([52.70.2.18]:34426) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ryk5f-0006SG-LV; Sun, 21 Apr 2024 23:11:37 -0400 Received: from localhost (71-212-21-65.tukw.qwest.net [71.212.21.65]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 31836FA059; Mon, 22 Apr 2024 03:11:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1713755474; bh=R2eErjloR/k799I5Z8tcnftYRkTq7AZvUrSFAED5UI0=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=I96q4t092UHATaoO5yvI9MN9T2NKF2DqeabLOKsC573FFESq/3+RvADgqPRDDSWTy Tqs2VqU7AlnTLklDrp46hxnfIrbIWnouoB8Qt0lFCVXWvfs6xq49M1E1l+hVwssjQa I6xqEqaUXxV06cGFy9uMZih2FoWenx+p03oBKN7A= From: Eric Abrahamsen To: Daniel Semyonov Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87sezev9qi.fsf@dsemy.com> (Daniel Semyonov's message of "Sun, 21 Apr 2024 21:41:57 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87h6io5rr5.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> <87sf1eh53p.fsf@dsemy.com> <87ttl3wk81.fsf_-_@dsemy.com> <87il1b7nxl.fsf@dsemy.com> <8634ry50gw.fsf@gnu.org> <868r1bf3i1.fsf@gnu.org> <87mspnlez1.fsf_-_@ericabrahamsen.net> <87sezev9qi.fsf@dsemy.com> Date: Sun, 21 Apr 2024 20:11:12 -0700 Message-ID: <87o7a2jdm7.fsf@ericabrahamsen.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64202 Cc: Eli Zaretskii , jimjoe@gmx.net, 66188@debbugs.gnu.org, 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On 04/21/24 21:41 PM, Daniel Semyonov wrote: >>>>>> Eric Abrahamsen writes: > > > Eli Zaretskii writes: > >> Ping! Ping! Eric, any comments? > > > Yes, let's do it! My only comment right now is that the manual shou= ld > > explicitly list the steps for creating an atom group: "B nnatom RET= " and > > insert URL without protocol. I still have dreams of fixing the group > > creation code to prompt for the address, but that can wait. > > > Daniel, would you make that addition? > > Does the following text (in the 'Atom' subsection of the Gnus manual) > not suffice? > > The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or local A= tom feeds as > Gnus servers (with a single group), as you would with any other method, > by supplying the location of the feed as the server address. Note, > however, that the server address shouldn't be prefixed with or > . I think we need to at least make it clear that you can't create these groups with "G m", which to me anyway is the "normal" way of making a group. It wouldn't hurt to be more explicit about the fact that feeds are created at the server level, not the group level. I'm not suggesting anything radical, something like: --8<---------------cut here---------------start------------->8---=20 The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or local Atom = feeds as Gnus servers, in your Gnus init files or as foreign servers via "B" in the *Group* buffer. The feed location is supplied as the server address, and each server only contains a single group. Note, however, that the server address shouldn't be prefixed with or . --8<---------------cut here---------------end--------------->8--- Or something like that. WDYT? From debbugs-submit-bounces@debbugs.gnu.org Mon Apr 22 18:46:32 2024 Received: (at 64202) by debbugs.gnu.org; 22 Apr 2024 22:46:33 +0000 Received: from localhost ([127.0.0.1]:48301 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rz2QZ-0005mi-Ke for submit@debbugs.gnu.org; Mon, 22 Apr 2024 18:46:32 -0400 Received: from dsemy.com ([46.23.89.208]:28353) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rz2QN-0005kv-Ny; Mon, 22 Apr 2024 18:46:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=QoSKBSQIUn8Vr gMetafXOJNXHkAfWBeFR/IBiHICfeo=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=l06KXr6rYfh2qX8GdJOuqLM3HdpoalHcfj7 Cqta1h7AJl0nB0FsEjnaK6JGov/8Pft24ADxdQlU6pwhAPH02dq6c11vytPLsh2QV6z3H/ c9CjVGHPjvF3TWTBGpEkOOJlmIUapfvqnSMSi31qn/DqlM5bf0SdGeEyjA01KpLASah+C9 ZgHklI2EPxfi3vg72Yz9EhFtoyidKvruv1ONVW0HTCh9YoDjeARPzEV/vRV6AsuOTw40F+ UqlxzPWsAnpHbHycsi2+RB+CAE0BNyG4ohyW6cSROBwZkvJYimGzmA+NNbHICyk2axK020 EVRVAOsaGZw9fj5awzYi/jFAZzw== Received: from coldharbour.local ( [147.235.220.9]) by dsemy.com (OpenSMTPD) with ESMTPSA id 0b5c5fb0 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 23 Apr 2024 00:45:48 +0200 (CEST) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id f7fa5824; Mon, 22 Apr 2024 22:44:44 +0000 (UTC) From: Daniel Semyonov To: Eric Abrahamsen Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <87o7a2jdm7.fsf@ericabrahamsen.net> (Eric Abrahamsen's message of "Sun, 21 Apr 2024 20:11:12 -0700") References: <87v8fhmgvw.fsf@dsemy.com> <87r0hof9ll.fsf_-_@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> <87sf1eh53p.fsf@dsemy.com> <87ttl3wk81.fsf_-_@dsemy.com> <87il1b7nxl.fsf@dsemy.com> <8634ry50gw.fsf@gnu.org> <868r1bf3i1.fsf@gnu.org> <87mspnlez1.fsf_-_@ericabrahamsen.net> <87sezev9qi.fsf@dsemy.com> <87o7a2jdm7.fsf@ericabrahamsen.net> Date: Tue, 23 Apr 2024 01:44:44 +0300 Message-ID: <871q6xuieb.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: Eli Zaretskii , jimjoe@gmx.net, 66188@debbugs.gnu.org, 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable >>>>> Eric Abrahamsen writes: > On 04/21/24 21:41 PM, Daniel Semyonov wrote: >>>>>>> Eric Abrahamsen writes: >>=20 >> > Eli Zaretskii writes: >> >> Ping! Ping! Eric, any comments? >>=20 >> > Yes, let's do it! My only comment right now is that the manual sho= uld >> > explicitly list the steps for creating an atom group: "B nnatom RE= T" and >> > insert URL without protocol. I still have dreams of fixing the gro= up >> > creation code to prompt for the address, but that can wait. >>=20 >> > Daniel, would you make that addition? >>=20 >> Does the following text (in the 'Atom' subsection of the Gnus manual) >> not suffice? >>=20 >> The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or loca= l Atom feeds as >> Gnus servers (with a single group), as you would with any other meth= od, >> by supplying the location of the feed as the server address. Note, >> however, that the server address shouldn't be prefixed with or >> . > I think we need to at least make it clear that you can't create these > groups with "G m", which to me anyway is the "normal" way of making a > group. It wouldn't hurt to be more explicit about the fact that feeds > are created at the server level, not the group level. I'm not suggest= ing > anything radical, something like: > --8<---------------cut here---------------start------------->8---=20 > The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or local= Atom feeds as Gnus > servers, in your Gnus init files or as foreign servers via "B" in the > *Group* buffer. The feed location is supplied as the server address, = and > each server only contains a single group. Note, however, that the ser= ver > address shouldn't be prefixed with or . > --8<---------------cut here---------------end--------------->8--- > Or something like that. WDYT? I gave it another thought and you're right, it should be more explicit. I've attached an updated patch but here's the new version of the paragraph (since I actually ended up being even more explicit): The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or local Ato= m feeds as Gnus servers, by adding them to =E2=80=98gnus-secondary-select-methods=E2= =80=99 or as foreign servers by pressing "B" in the =E2=80=98*Group*=E2=80=99 buffer, = for example (*note Finding the News). The address of each server is its feed's location (though the address shouldn't be prefixed with or ) and each server contains a single group which holds the feed's entries. Daniel --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Gnus-Add-back-end-for-Atom-feeds-nnatom.patch Content-Description: patch >From 58486c54878ec0496fc5006406a6a3dd4ffae033 Mon Sep 17 00:00:00 2001 From: Daniel Semyonov Date: Wed, 21 Jun 2023 10:05:04 +0300 Subject: [PATCH] Gnus: Add back end for Atom feeds (nnatom) * lisp/gnus/gnus.el (gnus-valid-select-methods): Add entry for nnatom. * lisp/gnus/nnfeed.el: New file implementing an abstract web feed back end. * lisp/gnus/nnatom.el: New file implementing a back end for Atom feeds. * doc/misc/gnus.texi (Browsing the Web, Back End Interface): * etc/NEWS (Gnus): Document nnatom and nnfeed. --- doc/misc/gnus.texi | 77 +++++ etc/NEWS | 10 + lisp/gnus/gnus.el | 1 + lisp/gnus/nnatom.el | 276 ++++++++++++++++++ lisp/gnus/nnfeed.el | 683 ++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 1047 insertions(+) create mode 100644 lisp/gnus/nnatom.el create mode 100644 lisp/gnus/nnfeed.el diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi index 419a5390374..8aa7f855aea 100644 --- a/doc/misc/gnus.texi +++ b/doc/misc/gnus.texi @@ -715,6 +715,7 @@ Top * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. Other Sources @@ -975,6 +976,7 @@ Top * Writing New Back Ends:: Extending old back ends. * Hooking New Back Ends Into Gnus:: What has to be done on the Gnus end. * Mail-like Back Ends:: Some tips on mail back ends. +* Web Feed Back Ends:: Easily defining back ends for web feeds. Various File Formats @@ -17252,6 +17254,7 @@ Browsing the Web @menu * Web Searches:: Creating groups from articles that match a string. * RSS:: Reading RDF site summary. +* Atom:: Reading Atom Syndication Format feeds. @end menu The main caveat with all these web sources is that they probably won't @@ -17496,6 +17499,42 @@ RSS @end lisp +@node Atom +@subsection Atom +@cindex nnatom +@cindex Atom + +Some web sites provide an Atom Syndication Format feed. Atom is a web +feed format similar in function to RDF Site Summary (@pxref{RSS}). + +The @code{nnatom} back end allows you to add HTTP or local Atom feeds as +Gnus servers, by adding them to @code{gnus-secondary-select-methods} or +as foreign servers by pressing "B" in the @file{*Group*} buffer, for +example (@pxref{Finding the News}). The address of each server is its +feed's location (though the address shouldn't be prefixed with or +) and each server contains a single group which holds the +feed's entries. + +Features of @code{nnatom} include: + +@itemize @bullet + +@item +Server data is saved per-server in the @file{atom} sub-directory of +@file{gnus-directory}. + +@item +An article part is generated for both the summary and the content for +each entry in the feed. Content of all MIME types should be displayed +correctly through Gnus (as long as they are supported and the feed +specifies a MIME type). + +@item +Article modification and publish dates are tracked, and articles are +updated if changed. + +@end itemize + @node Other Sources @section Other Sources @@ -29997,6 +30036,7 @@ Back End Interface * Writing New Back Ends:: Extending old back ends. * Hooking New Back Ends Into Gnus:: What has to be done on the Gnus end. * Mail-like Back Ends:: Some tips on mail back ends. +* Web Feed Back Ends:: Easily defining back ends for web feeds. @end menu @@ -30770,6 +30810,43 @@ Mail-like Back Ends @end example +@node Web Feed Back Ends +@subsubsection Web Feed Back Ends + +If you want to write a back end for a new type of web feed (RSS, +Atom), or some other type of feed, an ``abstract'' back end +(@code{nnfeed}) exists to enable the creation of such back ends with +minimal knowledge of Gnus. + +@code{nnfeed} defines a generic parser, which uses functions stored in +server variables to parse information from a feed (@code{nnfeed} +itself doesn't actually define any such functions though). + +The data parsed from the feed is stored in server variables (and +stored per-feed in a sub-directory of @option{gnus-directory} whose name +corresponds to the name of the back end). + +A Gnus back end interface is also defined, which uses the data parsed +from the feed. + +Therefore, a new back end only needs to inherit from @code{nnfeed}, +define (fairly) generic parsing functions for the feed type and setup +the required server variables. + +@code{nnfeed} was originally created to support Atom Syndication +Format feeds (@pxref{Atom}), but is very generic (as of writing this, +no standard web feed exists which can meaningfully use all the +features supported): it supports multiple groups contained in a single +feed, it allows for situations when the entire feed can't (or +shouldn't) be read ahead of time and it allows for very advanced +customization of the actual printing of articles from parsed data +(while providing a reasonably powerful default method). + +Further implementation details are available in the documentation +strings of the various @code{nnfeed-*} server variables and +the commentary and other comments of @file{nnfeed.el}. + + @node Score File Syntax @subsection Score File Syntax diff --git a/etc/NEWS b/etc/NEWS index 82c73f7416b..fea27bb8a31 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -1337,6 +1337,16 @@ when using the ':vc' keyword. ** Gnus ++++ +*** New back end 'nnfeed'. +This allows back end developers to easily create new back ends for web +feeds, as inheriting back ends of 'nnfeed'. + ++++ +*** New back end 'nnatom'. +This allow users to add Atom Syndication Format feeds to Gnus as +servers. + *** The 'nnweb-type' option 'gmane' has been removed. The gmane.org website is, sadly, down since a number of years with no prospect of it coming back. Therefore, it is no longer valid to set diff --git a/lisp/gnus/gnus.el b/lisp/gnus/gnus.el index dab66b60205..bc8819dc967 100644 --- a/lisp/gnus/gnus.el +++ b/lisp/gnus/gnus.el @@ -1360,6 +1360,7 @@ gnus-valid-select-methods ("nnimap" post-mail address prompt-address physical-address respool server-marks cloud) ("nnmaildir" mail respool address server-marks) + ("nnatom" address) ("nnnil" none)) "An alist of valid select methods. The first element of each list lists should be a string with the name diff --git a/lisp/gnus/nnatom.el b/lisp/gnus/nnatom.el new file mode 100644 index 00000000000..e8dfa12aff5 --- /dev/null +++ b/lisp/gnus/nnatom.el @@ -0,0 +1,276 @@ +;;; nnatom.el --- Atom backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnatom is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnatom is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnatom. If not, see . + +;;; Commentary: + +;; Gnus backend for HTTP or local feeds following the +;; Atom Syndication Format . + +;;; Code: + +(eval-when-compile + (require 'cl-lib) + (require 'subr-x)) + +(require 'nnfeed) +(require 'mm-url) +(require 'dom) + +(defgroup nnatom nil + "Atom backend for Gnus." + :group 'nnfeed) + +(nnoo-declare nnatom nnfeed) + +(nnfeed-define-basic-backend-interface nnatom) + +;;;; Atom feed parser: + +(defun nnatom--read-feed (feed _) + "Return a list structure representing FEED, or nil." + (if (string-match-p "\\`https?://" feed) + (nnheader-report + nnatom-backend + "Address shouldn't start with \"http://\" or \"https://\"") + (with-temp-buffer + (condition-case e + (if (file-name-absolute-p feed) + (insert-file-contents feed) + (mm-url-insert-file-contents (concat "https://" feed))) + (file-error (nnheader-report nnatom-backend (cdr e))) + (:success (when-let ((data (if (libxml-available-p) + (libxml-parse-xml-region + (point-min) (point-max)) + (car (xml-parse-region + (point-min) (point-max))))) + (authors (list 'authors))) + (when (eq (car data) 'top) + (setq data (assq 'feed data))) + (dom-add-child-before data authors) + (let ((all (dom-children data))) + (while-let ((rest (cdr all)) + (child (car-safe rest)) + (type (car-safe child)) + ((not (eq type 'entry)))) + (and (or (eq type 'author) + (eq type 'contributor)) + (dom-add-child-before authors child)) + (setq all rest)) + ;; Order of entries is reversed as most Atom feeds + ;; list only the "most recent" entries, in reverse + ;; chronological order. + (setcdr all (nreverse (cdr all)))) + data)))))) +(defvoo nnatom-read-feed-function #'nnatom--read-feed + nil nnfeed-read-feed-function) + +(defun nnatom--read-group (data) + "Return the next group and the remaining DATA in a cons cell, or nil." + `(,data)) +(defvoo nnatom-read-group-function #'nnatom--read-group + nil nnfeed-read-group-function) + +(defun nnatom--read-article (data _) + "Return the next article and the remaining DATA in a cons cell, or nil." + (when (eq (car data) 'feed) (setq data (dom-children data))) + ;; Discard any children between/after entries. + (while (and data (not (eq (car-safe (car data)) 'entry))) (pop data)) + (when-let ((article (car data)) + (auths (list 'authors)) (links (list 'links))) + (dom-add-child-before article links) + (dom-add-child-before article auths) + (dolist (child (cddddr article) `(,article . ,(cdr data))) + (pcase (car-safe child) ; Authors and links can appear + ((or 'author 'contributor) ; anywhere in the entry so we + (dom-add-child-before auths child) ; collect them all here to + (dom-add-child-before links child)) ; avoid looping over the + ((or 'link ; entry multiple times later. + (and 'content (guard (assq 'src (dom-attributes child))))) + (dom-add-child-before links child)))))) +(defvoo nnatom-read-article-function #'nnatom--read-article + nil nnfeed-read-article-function) + +(defun nnatom--read-title (group) + "Return the title of GROUP, or nil." + (dom-text (dom-child-by-tag group 'title))) +(defvoo nnatom-read-title-function #'nnatom--read-title + nil nnfeed-read-title-function) + +(defun nnatom--read-description (group) + "Return the description of GROUP, or nil." + (dom-text (dom-child-by-tag group 'subtitle))) +(defvoo nnatom-read-description-function #'nnatom--read-description + nil nnfeed-read-description-function) + +(defun nnatom--read-article-or-group-authors (article-or-group) + "Return the authors of ARTICLE-OR-GROUP, or nil." + (when-let + ((a (mapconcat + (lambda (author) + (let* ((name (dom-text (dom-child-by-tag author 'name))) + (name (unless (string-blank-p name) name)) + (email (dom-text (dom-child-by-tag author 'email))) + (email (unless (string-blank-p email) email))) + (or (and name email (format "%s <%s>" name email)) name email))) + (dom-children (dom-child-by-tag article-or-group 'authors)) + ", ")) + ((not (string-blank-p a)))) + a)) +(defvoo nnatom-read-author-function #'nnatom--read-article-or-group-authors + nil nnfeed-read-author-function) +(defvoo nnatom-read-group-author-function + #'nnatom--read-article-or-group-authors + nil nnfeed-read-group-author-function) + +(defun nnatom--read-subject (article) + "Return the subject of ARTICLE, or nil." + (dom-text (dom-child-by-tag article 'title))) +(defvoo nnatom-read-subject-function #'nnatom--read-subject + nil nnfeed-read-subject-function) + +(defun nnatom--read-id (article) + "Return the ID of ARTICLE. +If the ARTICLE doesn't contain an ID but it does contain a subject, +return the subject. Otherwise, return nil." + (or (dom-text (dom-child-by-tag article 'id)) + (nnatom--read-subject article))) +(defvoo nnatom-read-id-function #'nnatom--read-id + nil nnfeed-read-id-function) + +(defun nnatom--read-publish (article) + "Return the date and time ARTICLE was published, or nil." + (when-let (d (dom-child-by-tag article 'published)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-publish-date-function #'nnatom--read-publish + nil nnfeed-read-publish-date-function) + +(defun nnatom--read-update (article) + "Return the date and time of the last update to ARTICLE, or nil." + (when-let (d (dom-child-by-tag article 'updated)) + (date-to-time (dom-text d)))) +(defvoo nnatom-read-update-date-function #'nnatom--read-update + nil nnfeed-read-update-date-function) + +(defun nnatom--read-links (article) + "Return all links contained in ARTICLE, or nil." + (let ((alt 0) (rel 0) (sel 0) (enc 0) (via 0) (aut 0)) + (mapcan + (lambda (link) + (when-let ((l (car-safe link))) + (or + (when-let (((eq l 'content)) + (src (dom-attr link 'src)) + (label (concat "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt))))) + `(((("text/plain") . ,(format "%s: %s\n" label src)) + (("text/html") . ,(format "[%s] " + src label))))) + (when-let (((or (eq l 'author) (eq l 'contributor))) + (name (dom-text (dom-child-by-tag link 'name))) + (name (if (string-blank-p name) + (concat "Author" + (and (< 1 (cl-incf aut)) + (format " %s" aut))) + name)) + (uri (dom-text (dom-child-by-tag link 'uri))) + ((not (string-blank-p uri)))) + `(((("text/plain") . ,(format "%s: %s\n" name uri)) + (("text/html") . ,(format "[%s] " + uri name))))) + (when-let (((eq l 'link)) + (attrs (dom-attributes link)) + (label (or (cdr (assq 'title attrs)) + (pcase (cdr (assq 'rel attrs)) + ("related" + (concat "Related" + (and (< 1 (cl-incf rel)) + (format " %s" rel)))) + ("self" + (concat "More" + (and (< 1 (cl-incf sel)) + (format " %s" sel)))) + ("enclosure" + (concat "Enclosure" + (and (< 1 (cl-incf enc)) + (format " %s" enc)))) + ("via" + (concat "Source" + (and (< 1 (cl-incf via)) + (format " %s" via)))) + (_ (if-let + ((lang (cdr (assq 'hreflang link)))) + (format "Link (%s)" lang) + (concat + "Link" + (and (< 1 (cl-incf alt)) + (format " %s" alt)))))))) + (link (cdr (assq 'href attrs)))) + `(((("text/plain") . ,(format "%s: %s\n" label link)) + (("text/html") . ,(format "[%s] " + link label)))))))) + (dom-children (dom-child-by-tag article 'links))))) +(defvoo nnatom-read-links-function #'nnatom--read-links + nil nnfeed-read-links-function) + +(defun nnatom--read-part (part type) + (let* ((atypes '("html" "plain")) + (mtypes '(("xhtml" . "text/html") ("text" . "text/plain"))) + (xsuff (concat "[+/]xml\\(-\\(dtd\\|external-parsed-entity\\)\\)?\\'" + "\\|^text")) + (part (if (string= type "xhtml") + (with-temp-buffer + (dom-print (dom-child-by-tag part 'div) nil t) + (buffer-substring-no-properties + (point-min) (point-max))) + (dom-text part))) + (type (if (member type atypes) (concat "text/" type) type)) + (type (or (cdr (assoc type mtypes)) type))) + (unless (string-blank-p part) + `(,part (Content-Type . ,(or type (setq type "text/plain"))) + ,(and (not (string-match-p xsuff type)) + '(Content-Transfer-Encoding . "base64")))))) + +(defun nnatom--read-parts (article) + "Return all parts contained in ARTICLE, or an empty HTML part with links." + (let* ((summary (dom-child-by-tag article 'summary)) + (stype (cdr (assq 'type (dom-attributes summary)))) + (summary (nnatom--read-part summary stype)) + (content (dom-child-by-tag article 'content)) + (ctype (cdr (assq 'type (dom-attributes content)))) + (content (nnatom--read-part content ctype)) + (st (string= stype ctype)) + parts) + (cond ((and summary content) + (and st (push summary parts)) + (push (append content '(links)) parts) + (or st (push summary parts))) + ((setq content (or summary content)) + (push (append content '(links)) parts)) + (t (push '((nil (Content-Type . "text/html") links)) parts))) + parts)) +(defvoo nnatom-read-parts-function #'nnatom--read-parts + nil nnfeed-read-parts-function) + +(gnus-declare-backend (symbol-name nnatom-backend) 'address) + +(provide 'nnatom) + +;;; nnatom.el ends here diff --git a/lisp/gnus/nnfeed.el b/lisp/gnus/nnfeed.el new file mode 100644 index 00000000000..0bf599553e4 --- /dev/null +++ b/lisp/gnus/nnfeed.el @@ -0,0 +1,683 @@ +;;; nnfeed.el --- Generic feed backend for Gnus -*- lexical-binding: t -*- + +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Author: Daniel Semyonov + +;; This file is part of GNU Emacs. + +;; nnfeed is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; nnfeed is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with nnfeed. If not, see . + +;;; Commentary: + +;; Generic Gnus backend (intended) for implementing backends for web +;; feeds (Atom, RSS). +;; +;; This backend is abstract - it doesn't implement a parser for any +;; specific web feed type, and so can't be used independently. +;; +;; Instead, it implements a generic parser, feed data storage and most +;; of the Gnus backend interface; the intended use for this backend is +;; to be a source of inheritance for backends supporting new web feed +;; types. +;; +;; To implement new backends, use `nnfeed-define-basic-backend-interface': +;; +;; ... +;; (require 'nnfeed) +;; +;; (nnoo-declare nnfoo nnfeed) +;; +;; (nnfeed-define-basic-backend-interface nnfoo) +;; ... +;; [ definitions of parsing functions, see the "Feed parser interface" +;; section for more information. ] +;; +;; (defvoo nnfoo-read-feed-function #'nnfoo--read-feed +;; nil nnfeed-read-feed-function) +;; ... +;; (gnus-declare-backend (symbol-name nnfeed-backend) 'address) +;; +;; (provide 'nnfoo) +;; +;;; Code: + +(eval-when-compile + (require 'subr-x)) + +(require 'gnus) +(require 'nnoo) + +(defgroup nnfeed nil + "Generic feed backend for Gnus." + :group 'gnus) + +(defcustom nnfeed-date-format "%F %X%p" + "Format of displayed dates (see function `format-time-string')." + :type 'string) + +(nnoo-declare nnfeed) + +(defvoo nnfeed-backend nil + "Symbol which identifies this backend.") + +(defvoo nnfeed-status-string nil + "Last status message reported by this backend.") + +;; This macro should be used to define inheriting backends. + +(defmacro nnfeed-define-basic-backend-interface (backend) + "Define a basic set of functions and variables for BACKEND." + `(progn + (defvoo ,(nnoo-symbol backend 'backend) ',backend nil nnfeed-backend) + (defvoo ,(nnoo-symbol backend 'status-string) + nil nil nnfeed-status-string) + (defvoo ,(nnoo-symbol backend 'group) nil nil nnfeed-group) + (defvoo ,(nnoo-symbol backend 'servers) + (make-hash-table :test 'equal) nil nnfeed-servers) + (defvoo ,(nnoo-symbol backend 'group-article-ids) + (make-hash-table :test 'equal) nil nnfeed-group-article-ids) + (defvoo ,(nnoo-symbol backend 'group-articles) + (make-hash-table :test 'eql) nil nnfeed-group-articles) + (defvoo ,(nnoo-symbol backend 'group-article-max-num) 0 nil + nnfeed-group-article-max-num) + (defvoo ,(nnoo-symbol backend 'group-article-mix-num) 1 nil + nnfeed-group-article-min-num) + ,@(mapcar (lambda (fun) + `(deffoo ,(nnoo-symbol backend fun) (&optional server) + (,(nnoo-symbol 'nnoo fun) ',backend server))) + '(server-opened status-message)) + (deffoo ,(nnoo-symbol backend 'open-server) (server &optional defs) + (nnfeed-open-server server defs ',backend)) + (nnoo-import ,backend (nnfeed)))) + +;;;; Feed parser interface: + +;; The following set of server variables define a parser for a +;; specific web feed type. +;; An inheriting backend doesn't necessarily have to define all of +;; these functions (see the comments below for more information). +;; Apart from this set of variables there is also +;; `nnfeed-print-content-function' which can optionally be defined +;; by an inheriting backend to allow more advanced control over the +;; printing of articles. + +(defvoo nnfeed-read-feed-function #'ignore + "Function returning a Lisp object representing a feed (or part of it). + +It should accept two arguments, the address of a feed and the name of +a group (or nil). +If a group name is supplied, it should return a representation of only +the group (as if it was extracted from the feed with +`nnfeed-read-group-function').") + +(defvoo nnfeed-read-group-function #'ignore + "Function returning a cons cell of a group and remaining data from a feed. + +The returned group can be represented by any Lisp object. +It should accept a single argument, a Lisp object representing a feed +\(as can be returned by this function or `nnfeed-read-feed-function').") + +(defvoo nnfeed-read-article-function #'ignore + "Function returning a cons cell of an article and remaining data from a group. + +The returned article can be represented by any Lisp object. +It should accept two arguments, a Lisp object representing a group +\(as can be returned by this function or `nnfeed-read-group-function'), +and a flag indicating whether the last article was not new or updated.") + +(defvoo nnfeed-read-title-function #'ignore + "Function returning the title of a group (a string). + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +;; Optional. +(defvoo nnfeed-read-description-function #'ignore + "Function returning the description of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function').") + +;; Either this function or `nnfeed-read-author-function' is required. +(defvoo nnfeed-read-group-author-function #'ignore + "Function returning the author of a group (a string), or nil. + +It should accept a single argument, a Lisp object representing a group +\(as returned by `nnfeed-read-group-function')..") + +(defvoo nnfeed-read-id-function #'ignore + "Function returning the ID of an article. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-subject-function #'ignore + "Function returning the subject of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-update-date-function' is required. +(defvoo nnfeed-read-publish-date-function #'ignore + "Function returning the publish date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-publish-date-function' is required. +(defvoo nnfeed-read-update-date-function #'ignore + "Function returning the update date of an article (a time value), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; Either this function or `nnfeed-read-group-author-function' is required. +(defvoo nnfeed-read-author-function #'ignore + "Function returning the author of an article (a string), or nil. + +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-headers-function #'ignore + "Function returning an alist of article-wide MIME headers. + +Each element of this alist maps a MIME header (a symbol, +i.e. `Content-Type') to its value. As a special case, `:boundary' +maps to a string which will serve as the boundary between article +parts. This must be supplied if a custom boundary is used in a +multipart content type header. The default boundary is \"-_nnfeed_-\", +and is automatically modified to match the name of the back end. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;; As mentioned in their docstrings, the last two parsing functions +;; can optionally return any Lisp representation they want, provided +;; an appropriate `nnfeed-print-content-function' is defined. This +;; means they are also not _strictly_ required. + +(defvoo nnfeed-read-links-function #'ignore + "Function returning all links contained in an article. + +With the default `nnfeed-print-content-function', it should return a +list of links, where each link is an alist mapping MIME content types +to links formatted for display in a part of that type. Each content +type may also be a list of content types. +Otherwise, it could return any Lisp object. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +(defvoo nnfeed-read-parts-function #'ignore + "Function returning an alist associating parts of an article to their headers. + +With the default `nnfeed-print-content-function', each part should be a +string. Otherwise, it can be any Lisp object. The \"headers\" of +each part should be a list where each element is either a cons of a +MIME header (a symbol, i.e. `Content-Type') and its value (a string), +or any other Lisp object. MIME headers will be printed, the rest will +be passed on to `nnfeed-print-content-function', which recognizes the +following extra data by default: +- `links', if present, will cause links to be printed in the part. +It should accept a single argument, a Lisp object representing an article +\(as returned by `nnfeed-read-article-function').") + +;;;; Feed data storage: + +;; This section defines the data types used to store feed data, and +;; functions to read and write it. +;; All variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. + +(defvoo nnfeed-servers (make-hash-table :test 'equal) + "Hash table mapping known servers to their groups. + +Each value in this table should itself be a hash table mapping known +group names to their data, which should be a vector of the form +[GROUP IDS ARTICLES MAX MIN DESCRIPTION], where: +- GROUP is the \"real\" group name (the name known to the server). +- IDS is a hash table mapping article IDs to their numbers. +- ARTICLES is a hash table mapping article numbers to articles and + their attributes (see `nnfeed-group-articles'). +- MAX is the maximum article number. +- MIN is the minimum article number. +- DESCRIPTION is the group description.") + +(defvoo nnfeed-group-names (make-hash-table :test 'equal) + "Hash table mapping real group names to their custom name.") + +(defun nnfeed--server-address (server) + "Return SERVER's real address." + (if (string-suffix-p "-ephemeral" server) + (setq server (or (cadr (assq (nnoo-symbol nnfeed-backend 'address) + (cddr (gnus-server-to-method + (concat + (symbol-name nnfeed-backend) ":" + server))))) + server)) + server)) + +(defun nnfeed--server-file (server) + "Return the file containing data for SERVER." + (expand-file-name (format "%s/%s.eld" + (string-trim (symbol-name nnfeed-backend) + "nn") + (gnus-newsgroup-savable-name + (nnfeed--server-address server))) + gnus-directory)) + +(defun nnfeed--read-server (server) + "Read SERVER's information from storage." + (if-let ((f (nnfeed--server-file server)) + ((file-readable-p f))) + (with-temp-buffer + (insert-file-contents f) + (goto-char (point-min)) + (puthash server (read (current-buffer)) nnfeed-servers)) + (nnheader-report nnfeed-backend "Can't read %s" server))) + +(defun nnfeed--write-server (server) + "Write SERVER's information to storage." + (if-let ((f (nnfeed--server-file server)) + ((file-writable-p f))) + (if-let ((s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (with-temp-file f + (insert ";;;; -*- mode: lisp-data -*- DO NOT EDIT\n") + (prin1 s (current-buffer)) + (insert "\n") + t) + t) + (nnheader-report nnfeed-backend "Can't write %s" f))) + +;; The following function uses the parsing functions defined in the last +;; section to parse a feed (or just one group from it). +;; This is the only place where these parsing functions are used; the Gnus +;; backend interface extracts all required information from the parsed feed. + +(defun nnfeed--parse-feed (feed &optional group) + "Parse GROUP from FEED into a new or existing server. +If GROUP is omitted or nil, parse the entire FEED." + (let* ((feed (nnfeed--server-address feed)) + (s (or (gethash feed nnfeed-servers) (nnfeed--read-server feed) + (make-hash-table :test 'equal))) + (name group) ; (Maybe) fake name (or nil) + (group (aref (gethash group s `[,group]) 0)) ; Real name (or nil) + data) + (when (setq data (funcall nnfeed-read-feed-function feed group)) + (while-let ((cg (or (and name (cons data)) ; `data' is a single group + (funcall nnfeed-read-group-function data))) + (cg (prog1 (car cg) (setq data (cdr cg))))) + (let* ((name (funcall nnfeed-read-title-function cg)) ; Real name + (group (gethash name nnfeed-group-names name)) ; (Maybe) fake name + (info (gnus-get-info + (concat (symbol-name nnfeed-backend) "+" feed ":" group))) + (g (or (gethash group s) + `[ ,name ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""])) + (desc (funcall nnfeed-read-description-function cg)) + (ids (aref g 1)) + (articles (aref g 2)) + (max (aref g 3)) + (max (if max max + (setq max 0) ; Find max article number + (dolist ; remembered by Gnus. + ( r (cons (gnus-info-read info) + (gnus-info-marks info)) + max) + (mapc (lambda (x) + (let ((x (if (consp x) + (if (< (car x) (cdr x)) + (cdr x) (car x)) + x))) + (when (< max x) (setq max x)))) + (if (symbolp (car r)) (cdr r) r))))) + (group-author (funcall nnfeed-read-group-author-function cg)) + stale) + (and desc (aset g 5 desc)) + (while-let ((article (funcall nnfeed-read-article-function cg stale)) + (article (prog1 (car article) (setq cg (cdr article))))) + (when-let ((id (funcall nnfeed-read-id-function article)) + (id (format "<%s@%s.%s>" id name nnfeed-backend))) + (let* ((num (gethash id ids)) + (update (funcall nnfeed-read-update-date-function article)) + (prev-update (aref (gethash num articles + '[nil nil nil nil nil]) + 4))) + (if (or (null num) ; New article ID. + (and (null prev-update) update) + (and prev-update update + (time-less-p prev-update update))) + (let* ((num (or num (aset g 3 (setq max (1+ max))))) + (publish (funcall nnfeed-read-publish-date-function + article))) + (setf + (gethash id (aref g 1)) num + (gethash num (aref g 2)) + `[ ,id + ,(or (funcall nnfeed-read-author-function article) + group-author group) + ,(or (funcall nnfeed-read-subject-function article) + "no subject") + ,(or publish update '(0 0)) ; published + ,(or update publish '(0 0)) ; updated + ,(funcall nnfeed-read-links-function article) + ,(funcall nnfeed-read-parts-function article) + ,(funcall nnfeed-read-headers-function article)] + stale nil)) + (setq stale t))))) + (puthash group g s))) + (puthash feed s nnfeed-servers)))) + +;;;; Gnus backend functions: + +;; The following two sections define a Gnus backend interface based on +;; the parsed data from the last section. +;; All server variables in this section are automatically defined by +;; `nnfeed-define-basic-backend-interface'. +;; For more information about these functions see the "Back End +;; Interface" section of the Gnus manual. + +(defvoo nnfeed-group nil + "Name of the current group.") + +(defvoo nnfeed-group-article-ids (make-hash-table :test 'equal) + "Hash table mapping article IDs to their article number.") + +(defvoo nnfeed-group-articles (make-hash-table :test 'eql) + "Hash table mapping article numbers to articles and their attributes. + +Each value in this table should be a vector of the form +[ID FROM SUBJECT DATE UPDATED LINKS PARTS HEADERS], where: +- ID is the ID of the article. +- FROM is the author of the article or group. +- SUBJECT is the subject of the article. +- DATE is the date the article was published, or last updated (time value). +- UPDATE is the date the article was last updated, or published (time value). +- LINKS is a collection of links (any Lisp object). +- PARTS is an alist associating the content of each part of the + article to its headers. +- HEADERS is an alist associating article-wide MIME headers to their value.") + +(defvoo nnfeed-group-article-max-num 0 + "Maximum article number for the current group.") + +(defvoo nnfeed-group-article-min-num 1 + "Minimum article number for the current group.") + +(nnoo-define-basics nnfeed) + +(defun nnfeed--current-server-no-prefix () + "Remove the \"+\" prefix from the current server." + (string-remove-prefix (concat (symbol-name nnfeed-backend) "+") + (nnoo-current-server nnfeed-backend))) + +(defun nnfeed--group-data (group server) + "Get parsed data for GROUP from SERVER." + (when-let ((server (nnfeed--server-address server)) + (s (gethash server nnfeed-servers)) + ((hash-table-p s))) + (gethash group s))) + +(defun nnfeed-retrieve-article (article group) + "Retrieve headers for ARTICLE from GROUP." + (if-let ((a (gethash article (aref group 2)))) + (insert (format "221 %s Article retrieved. +From: %s\nSubject: %s\nDate: %s\nMessage-ID: %s\n.\n" + article + (aref a 1) + (aref a 2) + (format-time-string "%F %H:%M" (aref a 3)) + (aref a 0))) + (insert "404 Article not found.\n.\n"))) + +(deffoo nnfeed-retrieve-headers (articles &optional group server _fetch-old) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + `[ nil ,nnfeed-group-article-ids ,nnfeed-group-articles + nil nil nil]))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (or (and (stringp (car articles)) + (mapc (lambda (a) + (nnfeed-retrieve-article + (gethash a (aref g 2)) g)) + articles)) + (and (numberp (car articles)) + (range-map (lambda (a) (nnfeed-retrieve-article a g)) + articles))) + 'headers) + (nnheader-report nnfeed-backend "Group %s not found" (or group "")))) + +(deffoo nnfeed-open-server (server &optional defs backend) + (let ((backend (or backend 'nnfeed)) + (a (nnfeed--server-address server)) + s) + (nnoo-change-server backend server defs) + (when (setq s (or (gethash a nnfeed-servers) (nnfeed--read-server server))) + (maphash (lambda (group g) + (setq g (aref g 0)) + (unless (string= group g) + (puthash g group nnfeed-group-names))) + s)) + (setq a (nnfeed--server-file server)) + (or s (condition-case _ (make-directory (file-name-parent-directory a) t) + (:success (file-writable-p a)) + (t nil)) + (and (nnoo-close-server nnfeed-backend server) + (nnheader-report + nnfeed-backend "Server file %s not readable or writable" + server))))) + +(deffoo nnfeed-request-close () + (when (hash-table-p nnfeed-servers) + (maphash (lambda (server _) (nnfeed--write-server server)) nnfeed-servers) + (setq nnfeed-servers (make-hash-table :test 'equal))) + (setq nnfeed-status-string nil) + t) + +;; The default content printing function, which should be suitable for +;; most inheriting backends. + +(defun nnfeed--print-content (content attributes links) + "Return CONTENT formatted according to ATTRIBUTES, with LINKS possibly added." + (let ((links (and (memq 'links attributes) links))) + (when (or content links) + (concat + (and content (format "%s\n\n" content)) + (mapconcat (lambda (link) + (cdr (assoc (cdr (assq 'Content-Type attributes)) link + (lambda (types type) + (if (stringp types) (string= types type) + (member type types)))))) + links))))) + +(defvoo nnfeed-print-content-function #'nnfeed--print-content + "Function returning a single piece of content for an article (a string). + +It should accept three arguments, a part and its attributes (as returned +by `nnfeed-read-parts-function'), and links (as returned by +`nnfeed-read-links-function').") + +(defun nnfeed--print-part (content headers mime links) + "Print part of an article using its CONTENT, HEADERS, and LINKS. +Only HEADERS of a type included in MIME are considered." + (concat + (mapconcat (lambda (header) + (when-let ((m (car-safe header)) + ((member m mime))) + (format "%s: %s\n" m (cdr header)))) + headers) + "\n" + (funcall nnfeed-print-content-function content headers links))) + +(deffoo nnfeed-request-article (article &optional group server to-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (nnfeed--group-data group server) + (and (setq group nnfeed-group) + `[ nil ,nnfeed-group-article-ids + ,nnfeed-group-articles + ,nnfeed-group-article-max-num + ,nnfeed-group-article-min-num nil]))) + (num (or (and (stringp article) + (gethash article (aref g 1))) + (and (numberp article) article))) + ((and (<= num (aref g 3)) + (>= num (aref g 4)))) + (a (gethash num (aref g 2)))) + (with-current-buffer (or to-buffer nntp-server-buffer) + (erase-buffer) + (let* ((links (aref a 5)) + (parts (aref a 6)) + (headers (aref a 7)) + (boundary (or (cdr (assq :boundary headers)) + (format "-_%s_-" nnfeed-backend))) + (multi (length> parts 1)) + (mime '( Content-Type Content-Disposition + Content-Transfer-Encoding))) + (insert (format + "Subject: %s\nFrom: %s\nDate: %s\nMessage-ID: %s\n" + (aref a 2) (aref a 1) + (format-time-string + nnfeed-date-format (or (aref a 3) '(0 0))) + (aref a 0)) + (if (assq 'MIME-Version headers) "" "MIME-Version: 1.0\n") + (mapconcat (lambda (header) + (unless (keywordp (car header)) + (format "%s: %s\n" (car header) (cdr header)))) + headers) + (if multi + (if (assq 'Content-Type headers) "" + (format + "Content-Type: multipart/alternative; boundary=%s\n" + boundary)) + (prog1 (nnfeed--print-part + (caar parts) (cdar parts) mime links) + (setq parts nil))) + (mapconcat (lambda (part) + (format "--%s\n%s\n" boundary + (nnfeed--print-part + (car part) (cdr part) mime links))) + parts) + (if multi (format "--%s--" boundary) "\n"))) + `(,group . ,num)) + (nnheader-report nnfeed-backend "No such article"))) + +(deffoo nnfeed-request-group (group &optional server fast _info) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (if-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (or (if fast (nnfeed--group-data group server) + (setq server (nnfeed--parse-feed server group)) + (and (hash-table-p server) (gethash group server))) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) 0 1 ""]))) + (progn + (setq nnfeed-group group + nnfeed-group-article-ids (aref g 1) + nnfeed-group-articles (aref g 2) + nnfeed-group-article-max-num (aref g 3) + nnfeed-group-article-min-num (aref g 4)) + (insert (format "211 %s %s %s \"%s\"" + (hash-table-count nnfeed-group-article-ids) + nnfeed-group-article-min-num + nnfeed-group-article-max-num group)) + t) + (insert "404 group not found") + (nnheader-report nnfeed-backend "Group %s not found" group)))) + +(deffoo nnfeed-close-group (group &optional server) + (and (string= group nnfeed-group) + (setq nnfeed-group nil + nnfeed-group-article-ids (make-hash-table :test 'equal) + nnfeed-group-articles (make-hash-table :test 'eql) + nnfeed-group-article-max-num 0 + nnfeed-group-article-min-num 1)) + (setq server (or server (nnfeed--current-server-no-prefix))) + (nnfeed--write-server server)) + +(deffoo nnfeed-request-list (&optional server) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (when-let ((p (point)) + (s (nnfeed--parse-feed + (or server (nnfeed--current-server-no-prefix)))) + ((hash-table-p s))) + (maphash (lambda (group g) + (insert (format "\"%s\" %s %s y\n" + group (aref g 3) (aref g 4)))) + s) + (not (= (point) p))))) + +(deffoo nnfeed-request-post (&optional _server) + (nnheader-report nnfeed-backend "%s is a read only backend" nnfeed-backend)) + +;;;; Optional back end functions: + +(deffoo nnfeed-retrieve-groups (_groups &optional server) + (nnfeed-request-list server) + 'active) + +(deffoo nnfeed-request-type (_group &optional _article) + 'unknown) + +(deffoo nnfeed-request-group-description (group &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (g (nnfeed--group-data group server))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (insert group " " (aref g 5) "\n")))) + +(deffoo nnfeed-request-list-newsgroups (&optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (s (gethash (nnfeed--server-address server) nnfeed-servers)) + ((hash-table-p s))) + (with-current-buffer nntp-server-buffer + (erase-buffer) + (maphash (lambda (group g) + (insert group " " (aref g 5) "\n")) + s)))) + +(deffoo nnfeed-request-rename-group (group new-name &optional server) + (when-let ((server (or server (nnfeed--current-server-no-prefix))) + (a (nnfeed--server-address server)) + (s (or (gethash a nnfeed-servers) + (and ; Open the server to add it to `nnfeed-servers' + (save-match-data + (nnfeed-open-server + server + (cdr ; Get defs and backend. + (assoc a (cdr (assq nnfeed-backend nnoo-state-alist)) + (lambda (car key) + (and (stringp car) + (string-match + (concat + "\\`\\(\\(nn[[:alpha:]]+\\)\\+\\)?" + (regexp-quote key) "\\'") + car) + (setq server car))))) + (if (match-string 1 server) + (intern (match-string 2 server)) 'nnfeed))) + (gethash a nnfeed-servers)))) + (g (or (nnfeed--group-data group a) + `[ ,group ,(make-hash-table :test 'equal) + ,(make-hash-table :test 'eql) nil 1 ""]))) + (puthash new-name g s) + (puthash group new-name nnfeed-group-names) + (remhash group s) + (and (string= group nnfeed-group) + (setq nnfeed-group new-name)) + t)) + +(provide 'nnfeed) + +;;; nnfeed.el ends here -- 2.44.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 23 10:35:12 2024 Received: (at 64202) by debbugs.gnu.org; 23 Apr 2024 14:35:13 +0000 Received: from localhost ([127.0.0.1]:52577 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rzHEi-0007dS-RM for submit@debbugs.gnu.org; Tue, 23 Apr 2024 10:35:12 -0400 Received: from mail.ericabrahamsen.net ([52.70.2.18]:49300) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rzHEd-0007bM-SR; Tue, 23 Apr 2024 10:35:06 -0400 Received: from localhost (71-212-21-65.tukw.qwest.net [71.212.21.65]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 5FDBAFABD7; Tue, 23 Apr 2024 14:34:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1713882881; bh=gWlqtvNipqyp136+b4tXZIEBVr8AtxU1FxaV8ehGi3w=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=JU7v0qVDexP8W6VjpYwAxtwHHVlaro6/D3mNm7qUIK05xf5VLsixQdS2k6MWgV7Oh 4g2FAXRGV9/hYRLgih9z3IopXyDKmSvnRzMe8wPiXciJVJME+VBQrBMgeOO79LANhM 5Ob0pTK8BrA/+Gg/LrXtTJpISPEJ2+97s24wzFl8= From: Eric Abrahamsen To: Daniel Semyonov Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <871q6xuieb.fsf@dsemy.com> (Daniel Semyonov's message of "Tue, 23 Apr 2024 01:44:44 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87jznfpom1.fsf@gmx.net> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> <87sf1eh53p.fsf@dsemy.com> <87ttl3wk81.fsf_-_@dsemy.com> <87il1b7nxl.fsf@dsemy.com> <8634ry50gw.fsf@gnu.org> <868r1bf3i1.fsf@gnu.org> <87mspnlez1.fsf_-_@ericabrahamsen.net> <87sezev9qi.fsf@dsemy.com> <87o7a2jdm7.fsf@ericabrahamsen.net> <871q6xuieb.fsf@dsemy.com> Date: Tue, 23 Apr 2024 07:34:40 -0700 Message-ID: <874jbsi1vj.fsf@ericabrahamsen.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64202 Cc: Eli Zaretskii , jimjoe@gmx.net, 66188@debbugs.gnu.org, 64202@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On 04/23/24 01:44 AM, Daniel Semyonov wrote: >>>>>> Eric Abrahamsen writes: > > > On 04/21/24 21:41 PM, Daniel Semyonov wrote: > >>>>>>> Eric Abrahamsen writes: > >>=20 > >> > Eli Zaretskii writes: > >> >> Ping! Ping! Eric, any comments? > >>=20 > >> > Yes, let's do it! My only comment right now is that the manual s= hould > >> > explicitly list the steps for creating an atom group: "B nnatom = RET" and > >> > insert URL without protocol. I still have dreams of fixing the g= roup > >> > creation code to prompt for the address, but that can wait. > >>=20 > >> > Daniel, would you make that addition? > >>=20 > >> Does the following text (in the 'Atom' subsection of the Gnus manu= al) > >> not suffice? > >>=20 > >> The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or lo= cal Atom feeds as > >> Gnus servers (with a single group), as you would with any other me= thod, > >> by supplying the location of the feed as the server address. Note, > >> however, that the server address shouldn't be prefixed with or > >> . > > > I think we need to at least make it clear that you can't create the= se > > groups with "G m", which to me anyway is the "normal" way of making= a > > group. It wouldn't hurt to be more explicit about the fact that fee= ds > > are created at the server level, not the group level. I'm not sugge= sting > > anything radical, something like: > > > --8<---------------cut here---------------start------------->8---=20 > > The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or loc= al Atom feeds as Gnus > > servers, in your Gnus init files or as foreign servers via "B" in t= he > > *Group* buffer. The feed location is supplied as the server address= , and > > each server only contains a single group. Note, however, that the s= erver > > address shouldn't be prefixed with or . > > --8<---------------cut here---------------end--------------->8--- > > > Or something like that. WDYT? > > I gave it another thought and you're right, it should be more explicit. > I've attached an updated patch but here's the new version of the > paragraph (since I actually ended up being even more explicit): > > The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or local A= tom feeds as > Gnus servers, by adding them to =E2=80=98gnus-secondary-select-methods= =E2=80=99 or as > foreign servers by pressing "B" in the =E2=80=98*Group*=E2=80=99 buffer= , for example > (*note Finding the News). The address of each server is its feed's > location (though the address shouldn't be prefixed with or > ) and each server contains a single group which holds the > feed's entries. That looks great -- I think it can't hurt to provide some more hand-holding in the manual. I'll get this in this morning. Thanks! Eric From debbugs-submit-bounces@debbugs.gnu.org Tue Apr 23 18:09:20 2024 Received: (at 64202) by debbugs.gnu.org; 23 Apr 2024 22:09:20 +0000 Received: from localhost ([127.0.0.1]:54623 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rzOKC-0000Ir-H4 for submit@debbugs.gnu.org; Tue, 23 Apr 2024 18:09:20 -0400 Received: from dsemy.com ([46.23.89.208]:46422) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rzOK3-0000Fn-7l; Tue, 23 Apr 2024 18:09:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=dkim; bh=O8HKb5BLLKHnO 7npw35tis5fI1ev+XB+S2+7ndliICc=; h=date:references:in-reply-to: subject:cc:to:from; d=dsemy.com; b=BzB1mJAlIiN0FWoRi6liWbM0hzZpytlQZU+ hbxgBmU9ayVggh6PRgbIJ0AfPfejP4yRgxd+KbsYa82jkpqz/OPshRVt5YXLYjF91w7Cv5 nZARwuqNAkAk/Ia7ieme5sCbaNxmPDO9p/Mb+rmFLsDV19/bM8VcMUUKHyIixDP34KVibX AJwY8j71N08xTPFnUfytVV+KiKzJU3OdActabKyKHmSBwlMyOFhknjM3HnC4JodAxhYld+ NLZhGEu1NmbJhOP+TpibO5M8dYz0QlfapT8yanTpmKM1xeEDlZoLDZZYt9+9CJRz5JIO8l GMWjnVQoZcH/XU/p/VmJxdqOHVQ== Received: from coldharbour.local ( [2a06:c701:4864:6500:8193:ebb0:ffbc:2f3b]) by dsemy.com (OpenSMTPD) with ESMTPSA id 6f1e8eeb (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Wed, 24 Apr 2024 00:08:42 +0200 (CEST) Received: from localhost (coldharbour.local [local]) by coldharbour.local (OpenSMTPD) with ESMTPA id 591da9f1; Tue, 23 Apr 2024 22:07:41 +0000 (UTC) From: Daniel Semyonov To: Eric Abrahamsen Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <874jbsi1vj.fsf@ericabrahamsen.net> (Eric Abrahamsen's message of "Tue, 23 Apr 2024 07:34:40 -0700") References: <87v8fhmgvw.fsf@dsemy.com> <874jejb2sg.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> <87sf1eh53p.fsf@dsemy.com> <87ttl3wk81.fsf_-_@dsemy.com> <87il1b7nxl.fsf@dsemy.com> <8634ry50gw.fsf@gnu.org> <868r1bf3i1.fsf@gnu.org> <87mspnlez1.fsf_-_@ericabrahamsen.net> <87sezev9qi.fsf@dsemy.com> <87o7a2jdm7.fsf@ericabrahamsen.net> <871q6xuieb.fsf@dsemy.com> <874jbsi1vj.fsf@ericabrahamsen.net> Date: Wed, 24 Apr 2024 01:07:41 +0300 Message-ID: <877cgnka1e.fsf@dsemy.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64202 Cc: 64202@debbugs.gnu.org, Eli Zaretskii , 66188@debbugs.gnu.org, jimjoe@gmx.net X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) >>>>> Eric Abrahamsen writes: > On 04/23/24 01:44 AM, Daniel Semyonov wrote: >>>>>>> Eric Abrahamsen writes: >>=20 >> > On 04/21/24 21:41 PM, Daniel Semyonov wrote: >> >>>>>>> Eric Abrahamsen writes: >> >>=20 >> >> > Eli Zaretskii writes: >> >> >> Ping! Ping! Eric, any comments? >> >>=20 >> >> > Yes, let's do it! My only comment right now is that the manual = should >> >> > explicitly list the steps for creating an atom group: "B nnatom= RET" and >> >> > insert URL without protocol. I still have dreams of fixing the = group >> >> > creation code to prompt for the address, but that can wait. >> >>=20 >> >> > Daniel, would you make that addition? >> >>=20 >> >> Does the following text (in the 'Atom' subsection of the Gnus man= ual) >> >> not suffice? >> >>=20 >> >> The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or l= ocal Atom feeds as >> >> Gnus servers (with a single group), as you would with any other m= ethod, >> >> by supplying the location of the feed as the server address. Not= e, >> >> however, that the server address shouldn't be prefixed with or >> >> . >>=20 >> > I think we need to at least make it clear that you can't create th= ese >> > groups with "G m", which to me anyway is the "normal" way of makin= g a >> > group. It wouldn't hurt to be more explicit about the fact that fe= eds >> > are created at the server level, not the group level. I'm not sugg= esting >> > anything radical, something like: >>=20 >> > --8<---------------cut here---------------start------------->8---= =20 >> > The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or lo= cal Atom feeds as Gnus >> > servers, in your Gnus init files or as foreign servers via "B" in = the >> > *Group* buffer. The feed location is supplied as the server addres= s, and >> > each server only contains a single group. Note, however, that the = server >> > address shouldn't be prefixed with or . >> > --8<---------------cut here---------------end--------------->8--- >>=20 >> > Or something like that. WDYT? >>=20 >> I gave it another thought and you're right, it should be more explic= it. >> I've attached an updated patch but here's the new version of the >> paragraph (since I actually ended up being even more explicit): >>=20 >> The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or loca= l Atom feeds as >> Gnus servers, by adding them to =E2=80=98gnus-secondary-select-metho= ds=E2=80=99 or as >> foreign servers by pressing "B" in the =E2=80=98*Group*=E2=80=99 buf= fer, for example >> (*note Finding the News). The address of each server is its feed's >> location (though the address shouldn't be prefixed with or >> ) and each server contains a single group which holds the >> feed's entries. > That looks great -- I think it can't hurt to provide some more > hand-holding in the manual. I'll get this in this morning. Great, I just did a git pull and I see it; thank you very much! Daniel (BTW I guess this report should be closed; I'm not really sure what the procedure is since this technically isn't my report, but instead was merged with mine at some point) From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 24 10:19:58 2024 Received: (at 64202-done) by debbugs.gnu.org; 24 Apr 2024 14:20:00 +0000 Received: from localhost ([127.0.0.1]:58690 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rzdTS-0006Xt-AD for submit@debbugs.gnu.org; Wed, 24 Apr 2024 10:19:58 -0400 Received: from mail.ericabrahamsen.net ([52.70.2.18]:56728) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rzdTI-0006Vk-2v; Wed, 24 Apr 2024 10:19:46 -0400 Received: from localhost (71-212-21-65.tukw.qwest.net [71.212.21.65]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 63102FA059; Wed, 24 Apr 2024 14:19:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1713968355; bh=rojycCApCil+wyjlC9JexbXPrUG3LqYoNHYbK0CKagw=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=U6w5g1F5bJhy1XtrwAjJCf80piMzbvbe+O7yKqM0XLFfwbXVG7sqzMPEU9caXE1GG pkZ7jJLriAypkeFlrmohfItgbYGH1/ro7EPt3IbEDmsrULG8TmsaoxNy3u4hYf3zkX qFdz0m79pzu4bvft07+YqzIXQdhGupOwVhglgsWw= From: Eric Abrahamsen To: Daniel Semyonov Subject: Re: bug#66188: 29.1; Include Atom feed reader In-Reply-To: <877cgnka1e.fsf@dsemy.com> (Daniel Semyonov's message of "Wed, 24 Apr 2024 01:07:41 +0300") References: <87v8fhmgvw.fsf@dsemy.com> <87fry2pa1n.fsf@gmx.net> <87ttmhwxby.fsf@dsemy.com> <87wmrd9zcy.fsf@outlook.com> <875xyx5ksi.fsf@dsemy.com> <87sf219ntf.fsf@gmx.net> <87msrtkztg.fsf@outlook.com> <87ttm0mnku.fsf@dsemy.com> <87o7c7b0yf.fsf@dsemy.com> <87le79yl11.fsf@gmx.net> <87sf1eh53p.fsf@dsemy.com> <87ttl3wk81.fsf_-_@dsemy.com> <87il1b7nxl.fsf@dsemy.com> <8634ry50gw.fsf@gnu.org> <868r1bf3i1.fsf@gnu.org> <87mspnlez1.fsf_-_@ericabrahamsen.net> <87sezev9qi.fsf@dsemy.com> <87o7a2jdm7.fsf@ericabrahamsen.net> <871q6xuieb.fsf@dsemy.com> <874jbsi1vj.fsf@ericabrahamsen.net> <877cgnka1e.fsf@dsemy.com> Date: Wed, 24 Apr 2024 07:19:14 -0700 Message-ID: <87plue4zdp.fsf@ericabrahamsen.net> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64202-done Cc: Eli Zaretskii , 64202-done@debbugs.gnu.org, 66188-done@debbugs.gnu.org, jimjoe@gmx.net X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On 04/24/24 01:07 AM, Daniel Semyonov wrote: >>>>>> Eric Abrahamsen writes: > > > On 04/23/24 01:44 AM, Daniel Semyonov wrote: > >>>>>>> Eric Abrahamsen writes: > >>=20 > >> > On 04/21/24 21:41 PM, Daniel Semyonov wrote: > >> >>>>>>> Eric Abrahamsen writes: > >> >>=20 > >> >> > Eli Zaretskii writes: > >> >> >> Ping! Ping! Eric, any comments? > >> >>=20 > >> >> > Yes, let's do it! My only comment right now is that the manua= l should > >> >> > explicitly list the steps for creating an atom group: "B nnat= om RET" and > >> >> > insert URL without protocol. I still have dreams of fixing th= e group > >> >> > creation code to prompt for the address, but that can wait. > >> >>=20 > >> >> > Daniel, would you make that addition? > >> >>=20 > >> >> Does the following text (in the 'Atom' subsection of the Gnus m= anual) > >> >> not suffice? > >> >>=20 > >> >> The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or= local Atom feeds as > >> >> Gnus servers (with a single group), as you would with any other= method, > >> >> by supplying the location of the feed as the server address. N= ote, > >> >> however, that the server address shouldn't be prefixed with or > >> >> . > >>=20 > >> > I think we need to at least make it clear that you can't create = these > >> > groups with "G m", which to me anyway is the "normal" way of mak= ing a > >> > group. It wouldn't hurt to be more explicit about the fact that = feeds > >> > are created at the server level, not the group level. I'm not su= ggesting > >> > anything radical, something like: > >>=20 > >> > --8<---------------cut here---------------start------------->8--= -=20 > >> > The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or = local Atom feeds as Gnus > >> > servers, in your Gnus init files or as foreign servers via "B" i= n the > >> > *Group* buffer. The feed location is supplied as the server addr= ess, and > >> > each server only contains a single group. Note, however, that th= e server > >> > address shouldn't be prefixed with or . > >> > --8<---------------cut here---------------end--------------->8--- > >>=20 > >> > Or something like that. WDYT? > >>=20 > >> I gave it another thought and you're right, it should be more expl= icit. > >> I've attached an updated patch but here's the new version of the > >> paragraph (since I actually ended up being even more explicit): > >>=20 > >> The =E2=80=98nnatom=E2=80=99 back end allows you to add HTTP or lo= cal Atom feeds as > >> Gnus servers, by adding them to =E2=80=98gnus-secondary-select-met= hods=E2=80=99 or as > >> foreign servers by pressing "B" in the =E2=80=98*Group*=E2=80=99 b= uffer, for example > >> (*note Finding the News). The address of each server is its feed's > >> location (though the address shouldn't be prefixed with = or > >> ) and each server contains a single group which holds the > >> feed's entries. > > > That looks great -- I think it can't hurt to provide some more > > hand-holding in the manual. I'll get this in this morning. > > Great, I just did a git pull and I see it; thank you very much! > > Daniel > > (BTW I guess this report should be closed; I'm not really sure what the > procedure is since this technically isn't my report, but instead was > merged with mine at some point) Yes, we can close these, if they've been merged then they're the "same" report. And bug reports aren't precious, if we need another, we can open another! Speaking of getting more bug reports, do you want to post on gnus.general and let people know this new backend is available? It would be good to get some usage. Thanks, Eric From unknown Sun Jun 22 07:45:17 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Thu, 23 May 2024 11:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator