From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 25 04:19:38 2022 Received: (at submit) by debbugs.gnu.org; 25 Feb 2022 09:19:38 +0000 Received: from localhost ([127.0.0.1]:51270 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nNWlF-0005PB-LS for submit@debbugs.gnu.org; Fri, 25 Feb 2022 04:19:38 -0500 Received: from lists.gnu.org ([209.51.188.17]:33064) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nNWXC-00050h-Pn for submit@debbugs.gnu.org; Fri, 25 Feb 2022 04:05:07 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59570) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nNWXC-00012j-1T for bug-gnu-emacs@gnu.org; Fri, 25 Feb 2022 04:05:06 -0500 Received: from mailout06.t-online.de ([194.25.134.19]:59334) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nNWX9-00043g-5D for bug-gnu-emacs@gnu.org; Fri, 25 Feb 2022 04:05:05 -0500 Received: from fwd89.dcpf.telekom.de (fwd89.aul.t-online.de [10.223.144.115]) by mailout06.t-online.de (Postfix) with SMTP id 95FB6208E1 for ; Fri, 25 Feb 2022 10:04:53 +0100 (CET) Received: from mail.tetzco.de ([188.192.172.49]) by fwd89.t-online.de with (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384 encrypted) esmtp id 1nNWWz-3QF0Ph0; Fri, 25 Feb 2022 10:04:53 +0100 Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id D219A6C00B7 for ; Fri, 25 Feb 2022 10:04:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1645779888; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=PwDgaHjyL8rA/3fBFfSZbaZYA0yGb7cVYkFEX3BdMa4=; b=j+Qm8m1edEiKhn1n/ay7M5dIMFHshuIwGvM/2fxAhyMjQ5CM8xcthdGduNtmQv+TshT0Y3 RR1xFqRuW4FbiKd7vA1DMScovItq3rPANK2d4ySvSu87zvzXyDxfZMT2vuizE+voekVHFw KrkvxbNXIWN/XkpScqKTgTU0E16Kv/kU/LeuK/GJ9bktCcfxb4TBsV52cmNgr3TDzqHWVy NjaYVAWw8NxdEdm433kv5fx8lLi7AMGLYGn/uIhyHRzOQe5Odf2bEblBaWADg20DP2OaWb 37CEs9j845ckkUVJhVmJWgJxqHX1TUQs9d464mUAyOrPbEvMaHY4EYAIY91Hwd+CtHACTf HJpQNG7ArecXvJ8fJl/S+hLV3bIGz3Z/nB0eA62B3iLZL8jucTnlUbvq8q/FwK1EsRgqTt 1v++50Zg7pJKdOTjW91K45U70Kg4jXmaGt1sLnudL56ioHUl30W2qqAk2/htrjKhLXnOoF Owcnrj/8SCykY+P3Zxy/nSTl8Dkbv8Hc+DOtJCO8UCpIUcLGFDE0yNsRPWl2yiuI91RsGh krbPh8PIwu8ZbhsK0W8Nng/X1bLTJqUzl+Ko+XCcmQGRoLg5H3uzILqHEequ/rIBCkm7eb EMjWAKqeR9JXhOI2Y0UXT3OCMkL6HVDn8Ed6/+r5r7OZsudlLnSDI= From: "Kai Tetzlaff" To: bug-gnu-emacs@gnu.org Subject: 29.0.50; [PATCH] `sieve-manage-getscript' fails if script contains multibyte characters Date: Fri, 25 Feb 2022 10:04:47 +0100 Message-ID: <87wnhj5nbk.fsf@tetzco.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Queue-Id: D219A6C00B7 X-Spamd-Result: default: False [-6.69 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-3.00)[-1.000]; GENERIC_REPUTATION(-0.69)[-0.68619248898889]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; MIME_UNKNOWN(0.10)[application/emacs-lisp]; MIME_TRACE(0.00)[0:+,1:+,2:~,3:+,4:+,5:+]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Server: rakaposhi X-TOI-EXPURGATEID: 150726::1645779893-0000F5B3-056C70B4/0/0 CLEAN NORMAL X-TOI-MSGID: 8bbb26c9-1b3f-4aef-bf78-94f83fce10fb Received-SPF: none client-ip=194.25.134.19; envelope-from=kai.tetzlaff@t-online.de; helo=mailout06.t-online.de X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, TONLINE_FAKE_DKIM=1, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Fri, 25 Feb 2022 04:19:36 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --=-=-= Content-Type: text/plain The sieve-manage package uses the managesieve protocol (s. https://datatracker.ietf.org/doc/html/draft-ietf-sieve-managesieve) to communicate with a sieve server process. When the sieve-manage client retrieves a script from the server it uses the `sieve-manage-getscript' function to send command `GETSCRIPT ""` to the server and tries to parse the response. If the downloaded sieve script contains multibyte characters the attempt to parse the response results in an infloop (in `sieve-manage-parse-okno'). To reproduce, save the following code --=-=-= Content-Type: application/emacs-lisp; charset=utf-8 Content-Disposition: inline; filename=sieve-manage-getscript-minimal-example.el Content-Transfer-Encoding: quoted-printable Content-Description: minimal example (require 'sieve-manage) (require 'cl) ; for flet below (let* ((script-name "test.sieve") ;; variables `sieve-manage-server' and `sieve-manage-port' are ;; used in `sieve-manage-make-process-buffer' (sieve-manage-server) (sieve-manage-port "sieve") (sieve-buffer (sieve-manage-make-process-buffer)) (output-buffer (generate-new-buffer script-name))) (with-current-buffer sieve-buffer (goto-char (point-min)) ;; simulate managesieve response-getscript with a single multibyte ;; character: `=C3=A4` (insert "{32}\r\nif body :matches \"=C3=A4\" { stop; }\n\r\nOK \"Getscr= ipt completed.\"\r\n")) ;; use flet to mock some functions in call chain of sieve-manage-getscript (flet ((sieve-manage-send (_) nil) (accept-process-output (&optional _ _ _ _) nil) (get-buffer-process (_) nil)) ;; watch `sieve-manage-getscript' infloop (sieve-manage-getscript script-name output-buffer sieve-buffer) (kill-buffer sieve-buffer))) ;; Local Variables: ;; coding: utf-8-unix ;; End: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable to a file and run: emacs -Q -l * Detailed analysis: The example code sets up a response buffer for a successful managesieve `response-getscript` defined as: response-getscript =3D (sieve-script CRLF response-ok) Here's the buffer content: ``` 1: {32} 2: if body :matches "=C3=A4" { stop; } 3: 4: OK "Getscript completed." ``` It comprises: 1. lines 1-2 (`sieve-script`): encoded as a managesieve `literal-s2c` string which: a. starts with a length in the form '{}' (i.e. 32) b. followed by the string data (i.e. the actual script: 'if body :matches "=C3=A4" { stop;}') using UTF-8 encoding 2. line 3 (`CRLF`) 3. line 4 (`response-ok`): 'OK' SP "Getscript completed." (the latter is an optional `quoted` string which can be shown to the user) The sieve-manage code is parsing the length into an integer and uses it to skip over `sieve-script` to get to the start of line 3 ( - empty) which is then also skipped to get to line 4 in order to parse the result ('OK'). Now the problem: Since sieve-manage explicitly enables multibyte support in the response buffer (by calling '(mm-enable-multibyte)' in `sieve-manage-make-process-buffer`) and uses `goto-char' for the purpose of skipping/jumping over `sieve-script`, each multibyte character in `sieve-script` causes the jump to go 1 (2, 3) character(s) too far. In the example above there's only a single 2 byte character (`=C3=A4`), so instead of skipping to the beginning of line 3, we land in the middle of : . This causes the following attempt to parse the result code (i.e. the 'OK "Getscript completed."' line) to infloop in `sieve-manage-parse-okno'. * An attempt of a fix: As far as I can tell, the attached patch fixes the issue for the GETSCRIPT command. --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=sieve-manage-getscript-multibyte-fix.patch Content-Description: Fix for multibyte issue in `sieve-manage-getscript' diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 50342b9105..8020e6fdca 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -449,10 +449,19 @@ sieve-manage-deletescript (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "GETSCRIPT \"%s\"" name)) + (set-buffer-multibyte nil) (let ((script (sieve-manage-parse-string))) + (set-buffer-multibyte t) (sieve-manage-parse-crlf) (with-current-buffer output-buffer - (insert script)) + (insert (decode-coding-string + script + ;; not sure if using `buffer-file-coding-system' is + ;; the right approach, it might be better to hardcode + ;; it to utf-8-* (managesieve requires UTF-8 + ;; encoding) but in that case, which variant of + ;; utf-8-unix/dos/... is to be used? + buffer-file-coding-system t))) (sieve-manage-parse-okno)))) (defun sieve-manage-setactive (name &optional buffer) --=-=-= Content-Type: text/plain * Additional remarks: There might be more problems. E.g. `sieve-manage-putscript' contains the following comment: ;; Here we assume that the coding-system will ;; replace each char with a single byte. ;; This is always the case if `content' is ;; a unibyte string. which seems to indicate that it might also have an issue with multibyte content (even though I have not experienced any uploading issues). I will try do some more testing to check that. In general, it is also not clear to me why the response (or process) buffer needs to be multibyte enabled at all as it should only be used for the line/byte oriented protocol data. But the commit message of 8e16fb987df9b which introduced the multibyte handling states: commit 8e16fb987df9b80b8328e9dbf80351a5f9d85bbb Author: Albert Krewinkel Date: 2013-06-11 07:32:25 +0000 ... * Enable Multibyte for SieveManage buffers: The parser won't properly handle umlauts and line endings unless multibyte is turned on in the process buffer. ... so this was obviously done on purpose. I contacted Albert about this but he couldn't remember the details (it's been nearly 10 years). In GNU Emacs 29.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.31, cairo version 1.16.0) of 2022-02-18 built on moka Repository revision: 51e51ce2df46fc0c6e17a97e74b00366bb9c09d8 Repository branch: master System Description: Debian GNU/Linux bookworm/sid Configured using: 'configure --with-pgtk --with-native-compilation' Configured features: ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS XIM GTK3 ZLIB Important settings: value of $LANG: en_US.UTF-8 locale-coding-system: utf-8-unix --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 25 07:20:08 2022 Received: (at 54154) by debbugs.gnu.org; 25 Feb 2022 12:20:09 +0000 Received: from localhost ([127.0.0.1]:51488 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nNZZw-0002L8-Dy for submit@debbugs.gnu.org; Fri, 25 Feb 2022 07:20:08 -0500 Received: from quimby.gnus.org ([95.216.78.240]:57796) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nNZZq-0002KS-F6 for 54154@debbugs.gnu.org; Fri, 25 Feb 2022 07:20:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date: References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=sFFuLKAnldBhuLGOgN5GT5SW0WnftzEVR5vYyOAZqWo=; b=nJ0/bdPIvPdjh4MD0S6x5NTJ/5 LX9uEGzvXz1Kn1E6GJ8M28Jhme9OcIZf9ihQSFI4FHAvJvT0KIviOTaG4RTqq7/tBSbCnSEhg0Aml g5/4zPPVOh0zo+wqoBKrHk2E5I+DfmT4VsZl7c1PtKQwxVLBOs4FGKJeCWe2q/ONfarI=; Received: from [84.212.220.105] (helo=giant) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nNZZf-0001QT-Rb; Fri, 25 Feb 2022 13:19:56 +0100 From: Lars Ingebrigtsen To: "Kai Tetzlaff" Subject: Re: bug#54154: 29.0.50; [PATCH] `sieve-manage-getscript' fails if script contains multibyte characters References: <87wnhj5nbk.fsf@tetzco.de> X-Now-Playing: Patrik Torsson's =?utf-8?Q?=5FKolv=C3=A4teserenader=5F=3A?= =?utf-8?Q?_=22Avm=C3=B6nstring=22?= Date: Fri, 25 Feb 2022 13:19:51 +0100 In-Reply-To: <87wnhj5nbk.fsf@tetzco.de> (Kai Tetzlaff's message of "Fri, 25 Feb 2022 10:04:47 +0100") Message-ID: <878rtzxhnc.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: "Kai Tetzlaff" writes: > (with-current-buffer (or buffer (current-buffer)) > (sieve-manage-send (format "GETSCRIPT \"%s\"" name)) > + (set-buffer-multibyte nil) > (let ((script (sieve-manage-parse-string))) > + (set-buffer- [...] Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54154 Cc: 54154@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) "Kai Tetzlaff" writes: > (with-current-buffer (or buffer (current-buffer)) > (sieve-manage-send (format "GETSCRIPT \"%s\"" name)) > + (set-buffer-multibyte nil) > (let ((script (sieve-manage-parse-string))) > + (set-buffer-multibyte t) Changing multibyteness in a buffer like this is (virtually) never the right thing to do -- it usually leads to obscure breakages. > In general, it is also not clear to me why the response (or process) > buffer needs to be multibyte enabled at all as it should only be used > for the line/byte oriented protocol data. But the commit message of > 8e16fb987df9b which introduced the multibyte handling states: > > commit 8e16fb987df9b80b8328e9dbf80351a5f9d85bbb > Author: Albert Krewinkel > Date: 2013-06-11 07:32:25 +0000 > ... > * Enable Multibyte for SieveManage buffers: The parser won't properly > handle umlauts and line endings unless multibyte is turned on in the > process buffer. > ... > > so this was obviously done on purpose. I contacted Albert about this but > he couldn't remember the details (it's been nearly 10 years). I don't see why this buffer should be multibyte, either. The communication with the server is done using bytes, not characters. When we need to have characters, we should decode the data and put it in a multibyte buffer. So can you try to back out that commit and see whether it fixes the problem instead? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 25 08:11:07 2022 Received: (at 54154) by debbugs.gnu.org; 25 Feb 2022 13:11:07 +0000 Received: from localhost ([127.0.0.1]:51547 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nNaNH-00060P-JA for submit@debbugs.gnu.org; Fri, 25 Feb 2022 08:11:07 -0500 Received: from smtp.domeneshop.no ([194.63.252.55]:35145) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nNaNE-0005zm-0O for 54154@debbugs.gnu.org; Fri, 25 Feb 2022 08:11:06 -0500 Received: from [84.212.220.105] (port=34652 helo=giant) by smtp.domeneshop.no with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nNaN7-0008W1-HY; Fri, 25 Feb 2022 14:10:57 +0100 From: Lars Ingebrigtsen To: "Kai Tetzlaff" Subject: Re: bug#54154: 29.0.50; [PATCH] `sieve-manage-getscript' fails if script contains multibyte characters In-Reply-To: <878rtzxhnc.fsf@gnus.org> (Lars Ingebrigtsen's message of "Fri, 25 Feb 2022 13:19:51 +0100") References: <87wnhj5nbk.fsf@tetzco.de> <878rtzxhnc.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAABGdBTUEAALGPC/xhBQAAACBj SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAGFBMVEX27dH07NTb0rSr pI7m2rjGuaOHiHr///+41XY3AAAAAWJLR0QHFmGI6wAAAAd0SU1FB+YCGQs2C0mDvlYAAAG6SURB VDjLbZJBlpswDECl1N2bDN0TYbqeGZesjWv2FEIvwJvc/wgjAU4MQS/hgb/1ZcsGyF4C5QHP122A /C7zq96Dk7WfaRZG8GZN+e9IVX5g/nkAdKt/XQYUg95m6J/Dx4BRHiFgpn/fncUDFd7GvH6tgVnb 6B/dY5lnyhvZF4B+q6lGBA0Yfecmz+SrsgVqBFkYnrO8WVWY8eRMA8g4nKkiMt5wcR5myg+2shmX vrJYQ8MZMIdgXuai8oEMXSCGhuNAAOdAHZAQHCwA5R8FKhgXwWMqGwKHiiBOdqKR2LkdhJWsGRjl oLjAmiK6UJFM4U8XvAC1cXnHCkUUixTJkgRTotrk+RmoF6BmoBQLFDyYrwq1qP53YNK988IZmIFv LOx36YNp7dXaXYm5SGknazvZ4BaU9nqfbAe7DqswXO/3qUt2v/IwTFc7OWX2RXxZT8OX4xaB21Xv rTcwZ7htEV84M59Jf9uC0POoe70m3BHDR+ALaN+fdDlcIhp7gr9/TmSedTxVIweDgVvpisd14dlM qpHAvqclTlTNGRWDOl1wywX6ik0MbNouGut6JBJVfUsyTr14GHj6BnxrY2eh47n+AAAAJXRFWHRk YXRlOmNyZWF0ZQAyMDIyLTAyLTI1VDExOjU0OjExKzAwOjAwquXbswAAACV0RVh0ZGF0ZTptb2Rp ZnkAMjAyMi0wMi0yNVQxMTo1NDoxMSswMDowMNu4Yw8AAAAASUVORK5CYII= X-Now-Playing: Patrik Torsson's =?utf-8?Q?=5FKolv=C3=A4teserenader=5F=3A?= "Nattglimmer" Date: Fri, 25 Feb 2022 14:10:56 +0100 Message-ID: <875yp3w0pr.fsf@gnus.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 1.2 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: The mail bounced with: kai.tetzlaff@t-online.de host mx03.t-online.de [194.25.134.73] SMTP error from remote mail server after initial connection: 554 IP=95.216.78.240 - A problem occurred. (Ask your postmaster for help or [...] Content analysis details: (1.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 RCVD_IN_MSPIKE_H5 RBL: Excellent reputation (+5) [194.63.252.55 listed in wl.mailspike.net] 0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: 54154 Cc: 54154@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.2 (/) The mail bounced with: kai.tetzlaff@t-online.de host mx03.t-online.de [194.25.134.73] SMTP error from remote mail server after initial connection: 554 IP=95.216.78.240 - A problem occurred. (Ask your postmaster for help or to contact tosa@rx.t-online.de to clarify.) Trying to send via a different SMTP server... Lars Ingebrigtsen writes: > "Kai Tetzlaff" writes: > >> (with-current-buffer (or buffer (current-buffer)) >> (sieve-manage-send (format "GETSCRIPT \"%s\"" name)) >> + (set-buffer-multibyte nil) >> (let ((script (sieve-manage-parse-string))) >> + (set-buffer-multibyte t) > > Changing multibyteness in a buffer like this is (virtually) never the > right thing to do -- it usually leads to obscure breakages. > >> In general, it is also not clear to me why the response (or process) >> buffer needs to be multibyte enabled at all as it should only be used >> for the line/byte oriented protocol data. But the commit message of >> 8e16fb987df9b which introduced the multibyte handling states: >> >> commit 8e16fb987df9b80b8328e9dbf80351a5f9d85bbb >> Author: Albert Krewinkel >> Date: 2013-06-11 07:32:25 +0000 >> ... >> * Enable Multibyte for SieveManage buffers: The parser won't properly >> handle umlauts and line endings unless multibyte is turned on in the >> process buffer. >> ... >> >> so this was obviously done on purpose. I contacted Albert about this but >> he couldn't remember the details (it's been nearly 10 years). > > I don't see why this buffer should be multibyte, either. The > communication with the server is done using bytes, not characters. When > we need to have characters, we should decode the data and put it in a > multibyte buffer. > > So can you try to back out that commit and see whether it fixes the > problem instead? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 25 12:26:55 2022 Received: (at 54154) by debbugs.gnu.org; 25 Feb 2022 17:26:55 +0000 Received: from localhost ([127.0.0.1]:53630 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nNeMn-0007U7-HO for submit@debbugs.gnu.org; Fri, 25 Feb 2022 12:26:54 -0500 Received: from mout-p-102.mailbox.org ([80.241.56.152]:45712) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nNd1g-0002k1-46 for 54154@debbugs.gnu.org; Fri, 25 Feb 2022 11:01:01 -0500 Received: from smtp102.mailbox.org (smtp102.mailbox.org [80.241.60.233]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-102.mailbox.org (Postfix) with ESMTPS id 4K4vdK5N1Mz9sV4; Fri, 25 Feb 2022 17:00:53 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzlaff.eu; s=20210624; t=1645804839; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=MFS0aNH9Y2gV4HS794o0V2/dUOOHjpqxcDPZmgIli+c=; b=UyzA2i4c3QWsqHKtduWsiarraWWw2b/lz34MmSCzkVAEC0Mj/myG3syER59QKd+9n+1W1z zaDmAXBhWKZcUQOxb3ClUlxOWz5DnETJOskDCf+aFe+BpvO0XC5KvbcIjMfvXwi0DohOdA apMLYTSLdDQEr/KGjO4Ie26ZkoGHSjxBahr8AF59v2KHZ16rxPve233tMg/W5xI/fhSrvG W7DwFJDSieBNMu7IOI0sGah898vdyHDTB7yujjYedMFffIidA9LTt3z8v4cLPK9m25M/eb O7jtek7/+UnEmYGW8RrSQdpxh/sBHTPVxUSrTsX1OFo1MHzf0WwDnNjFkpQjVQGIuSyPOw bRWkBnbLn9hqlUW6pP7JIrWvzUHJPNyYx1yTUiveo8+Lur/33AOEVELXpfwVawooUop6N8 yQ1wkBvN9unxqKvb/mw5wQRuzSbbMiOcGkwEsFBen67wM5nzwCxqy9nHrwGf7Nu2JahOPd IJTgYqVngXXJPWq+gf6Byiwd+XPt76H+Jj62o+UDeQIhh32m9DcWZ+zJ9FMxiYj8U5Y5aL oTRKGPnXgyGWNaYl9DQNfp9yRedhUNbhB0jKtjq2rH6MKwi0DNbljq/LrzMKdCWySFBkqd yASDDNSmRaZ+TbemkwXBj3UO1LNKa5zd73G7QuHgZ1OuV8pgZbeB0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzlaff.eu; s=MBO0001; t=1645804852; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=MFS0aNH9Y2gV4HS794o0V2/dUOOHjpqxcDPZmgIli+c=; b=il7c/x5sqvcl0hxJsj90tFWhv2knOcAx5399E7Nn4ze/n+4ikwJ/eOp8sXsQgRQX5K+Ee3 fVl/QV5G3eLn79CkwQVwr2QkOy7pOcQFJn5Z1bfBKnwMQn9xrSpYCuSf2r8ScBCfIVSL2p PLJ7x/Hre9MS2aRMO1edjh/NgN0T+n0aCQFOwV0x36UAGkEKruQ+FHNdHK2QiHm/bkR1bj OUm6yExUnBwPTcpk8euW+reoLBuVIfCivlyV5Is+bUaLkC3k2+slAX8A1JjmXnsI2NWFXj kosDAz7oV6nqgLG2HdCIhQCLsUqg0JVf04YAPvg8hDV5lkGHHLGiHS0m9ONZCQ== From: Kai Tetzlaff To: Lars Ingebrigtsen Subject: Re: bug#54154: 29.0.50; [PATCH] `sieve-manage-getscript' fails if script contains multibyte characters References: <87wnhj5nbk.fsf@tetzco.de> <878rtzxhnc.fsf@gnus.org> <875yp3w0pr.fsf@gnus.org> Date: Fri, 25 Feb 2022 17:00:36 +0100 In-Reply-To: <875yp3w0pr.fsf@gnus.org> (Lars Ingebrigtsen's message of "Fri, 25 Feb 2022 14:10:56 +0100") Message-ID: <87ilt33pi3.fsf@tetzco.de> MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Queue-Id: 1D20B6C00B7 X-Spamd-Result: default: False [-6.79 / 30.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM(-3.00)[-1.000]; GENERIC_REPUTATION(-0.69)[-0.68619817412468]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; DKIM_SIGNED(0.00)[tetzlaff.eu:s=20210624]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2] X-Rspamd-Server: rakaposhi X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 54154 X-Mailman-Approved-At: Fri, 25 Feb 2022 12:26:50 -0500 Cc: 54154@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) Lars Ingebrigtsen writes: > The mail bounced with: > > kai.tetzlaff@t-online.de > host mx03.t-online.de [194.25.134.73] > SMTP error from remote mail server after initial connection: > 554 IP=95.216.78.240 - A problem occurred. (Ask your postmaster for help or to contact tosa@rx.t-online.de to clarify.) Sorry, not sure what is happening there. Using a different From: address, now (hopefully - using the t-online address was accidental anyway). >>> (with-current-buffer (or buffer (current-buffer)) >>> (sieve-manage-send (format "GETSCRIPT \"%s\"" name)) >>> + (set-buffer-multibyte nil) >>> (let ((script (sieve-manage-parse-string))) >>> + (set-buffer-multibyte t) >> >> Changing multibyteness in a buffer like this is (virtually) never the >> right thing to do -- it usually leads to obscure breakages. >> >>> In general, it is also not clear to me why the response (or process) >>> buffer needs to be multibyte enabled at all as it should only be used >>> for the line/byte oriented protocol data. But the commit message of >>> 8e16fb987df9b which introduced the multibyte handling states: >>> >>> commit 8e16fb987df9b80b8328e9dbf80351a5f9d85bbb >>> Author: Albert Krewinkel >>> Date: 2013-06-11 07:32:25 +0000 >>> ... >>> * Enable Multibyte for SieveManage buffers: The parser won't properly >>> handle umlauts and line endings unless multibyte is turned on in the >>> process buffer. >>> ... >>> >>> so this was obviously done on purpose. I contacted Albert about this but >>> he couldn't remember the details (it's been nearly 10 years). >> >> I don't see why this buffer should be multibyte, either. The >> communication with the server is done using bytes, not characters. When >> we need to have characters, we should decode the data and put it in a >> multibyte buffer. >> >> So can you try to back out that commit and see whether it fixes the >> problem instead? Most of the referenced commit was about changes related to STARTTLS handling. Here's the full commit message: lisp/gnus/sievel-manage.el: fully support STARTTLS, fix bit rot * Make sieve-manage-open work with STARTTLS: shorten stream managing functions by using open-protocol-stream to do most of the work. Has the nice benefit of enabling STARTTLS. * Remove unneeded functions and options: the following functions and options are neither in the API, nor called by any other function, so they are deleted: - sieve-manage-network-p - sieve-manage-network-open - sieve-manage-starttls-p - sieve-manage-starttls-open - sieve-manage-forward - sieve-manage-streams - sieve-manage-stream-alist The options could not be applied in a meaningful way anymore; they didn't happen to have much effect before. * Cosmetic changes and code clean-up * Enable Multibyte for SieveManage buffers: The parser won't properly handle umlauts and line endings unless multibyte is turned on in the process buffer. * Wait for capabilities after STARTTLS: following RFC5804, the server sends new capabilities after successfully establishing a TLS connection with the client. The client should update the cached list of capabilities, but we just ignore the answer for now. So just reverting it won't work. I will try to undo the parts relevant to this issue. For clarification: The original code before Alberts change was using this macro (which seemingly contains an error in the doc string): (defmacro sieve-manage-disable-multibyte () "Enable multibyte in the current buffer." (unless (featurep 'xemacs) '(set-buffer-multibyte nil))) to *disable* multibyte handling in the response/protocol buffer. If using `set-buffer-multibyte' is not the right thing, what should be used instead? From debbugs-submit-bounces@debbugs.gnu.org Sat Feb 26 10:07:30 2022 Received: (at 54154) by debbugs.gnu.org; 26 Feb 2022 15:07:30 +0000 Received: from localhost ([127.0.0.1]:56086 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nNyfS-000383-0U for submit@debbugs.gnu.org; Sat, 26 Feb 2022 10:07:30 -0500 Received: from quimby.gnus.org ([95.216.78.240]:42110) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nNyfP-00037r-Tt for 54154@debbugs.gnu.org; Sat, 26 Feb 2022 10:07:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date: References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=9BTDYko715Xwl68dwKfdSQ3HpveO7vvWdgCShAGr7ko=; b=I+/9NrNxdOkXjtiU0nfy9tXtQH bBnAewQ466P3xJl+hyqyH8KCJbjSkxJebzv0XYDwS1n/VWhKGFDrlBFnTacXyYJLsMCr6lshOXln2 hvqO7KlRQp+CvewdqRo8l9HRlJElOUxFigV19z96ZPC4YmCaoZAaXSxbiheqHcJ+oBtc=; Received: from [84.212.220.105] (helo=giant) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nNyfH-0006LE-9Z; Sat, 26 Feb 2022 16:07:22 +0100 From: Lars Ingebrigtsen To: Kai Tetzlaff Subject: Re: bug#54154: 29.0.50; [PATCH] `sieve-manage-getscript' fails if script contains multibyte characters References: <87wnhj5nbk.fsf@tetzco.de> <878rtzxhnc.fsf@gnus.org> <875yp3w0pr.fsf@gnus.org> <87ilt33pi3.fsf@tetzco.de> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAABGdBTUEAALGPC/xhBQAAACBj SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAElBMVEX8+vnY08+tpaGE e3ZANzH////1ngwYAAAAAWJLR0QF+G/pxwAAAAd0SU1FB+YCGg8DJFXfRosAAAGwSURBVDjLtZPb gYQgDEVBGwCcAgSmADUUMEPSf02bhIfu/i8fo+SY5N7AGPMvy3v3Z7/rI3qbHsSHDB95aiz4CWJI EhtV7MzZ/+xnytHArLD8zng0Ha+1VwoJrugeKSSbkAPp2qNZegp9jdmIjgZOQjdANW+O1LcCxELH yDheEuKfApL+jrGDYvVbAsLCGpel5dB3+/DXeCVZzkS3sgYBhCCSQNYVLTip7AQwKk0T4Wk2bYlO AQ1Qk7NmJU1pX3I9aXERRGNTFnUSvmcLqZy+DVv9TrD5grEb1BFNgECHvcF3Dp1tHsE1gM+UcK4u u5lB9XE7DAwg/o77BK0A9SG1cNYyC/oOnv3t3oCfgNy4PWtzMidV9bYaAecvABA1zib35ghKH2Qf zFYDsLAJgLqujKqFIwpyDf2OvgkST3+c3QtTB4VO729QsHtceBPtBCwNukVWHsNutG3zIaqsddmn GJoPkcHnc/8XfBildqvzCdF5D5Bihg6QL/9KkOUVxGwazcvHvLDU19By3WNfWKXxKfU53KBcF3e3 fmv7+zwwI3e+nGgseGeom1ILD8OHUugHUE2KyKSnXokAAAAldEVYdGRhdGU6Y3JlYXRlADIwMjIt MDItMjZUMTU6MDM6MzYrMDA6MDBXnYsfAAAAJXRFWHRkYXRlOm1vZGlmeQAyMDIyLTAyLTI2VDE1 OjAzOjM2KzAwOjAwJsAzowAAAABJRU5ErkJggg== X-Now-Playing: Laraaji's _Moon Piano_: "Through This Moment" Date: Sat, 26 Feb 2022 16:07:14 +0100 In-Reply-To: <87ilt33pi3.fsf@tetzco.de> (Kai Tetzlaff's message of "Fri, 25 Feb 2022 17:00:36 +0100") Message-ID: <877d9hvf8d.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Kai Tetzlaff writes: > So just reverting it won't work. I will try to undo the parts relevant > to this issue. Sounds good. Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54154 Cc: 54154@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Kai Tetzlaff writes: > So just reverting it won't work. I will try to undo the parts relevant > to this issue. Sounds good. > For clarification: The original code before Alberts change was using > this macro (which seemingly contains an error in the doc string): > > (defmacro sieve-manage-disable-multibyte () > "Enable multibyte in the current buffer." > (unless (featurep 'xemacs) > '(set-buffer-multibyte nil))) > > to *disable* multibyte handling in the response/protocol buffer. If > using `set-buffer-multibyte' is not the right thing, what should be used > instead? Using (set-buffer-multibyte nil) is the right thing to do to make a buffer unibyte -- but usually only when the buffer is empty. There's been some discussion about making `set-buffer-multibyte' signal an error if used in a non-empty buffer, because in 99.7% of the cases where people do that, it's the wrong thing to do. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no From debbugs-submit-bounces@debbugs.gnu.org Mon Feb 28 07:35:44 2022 Received: (at 54154) by debbugs.gnu.org; 28 Feb 2022 12:35:44 +0000 Received: from localhost ([127.0.0.1]:60398 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nOfFf-0004Hh-Mf for submit@debbugs.gnu.org; Mon, 28 Feb 2022 07:35:44 -0500 Received: from mailout11.t-online.de ([194.25.134.85]:34128) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nOfFd-0004HW-IG for 54154@debbugs.gnu.org; Mon, 28 Feb 2022 07:35:42 -0500 Received: from fwd86.dcpf.telekom.de (fwd86.aul.t-online.de [10.223.144.112]) by mailout11.t-online.de (Postfix) with SMTP id 15CE418472; Mon, 28 Feb 2022 13:35:40 +0100 (CET) Received: from mail.tetzco.de ([188.192.172.49]) by fwd86.t-online.de with (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384 encrypted) esmtp id 1nOfFa-4XC9mD0; Mon, 28 Feb 2022 13:35:39 +0100 Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id DDD1B6C00B7; Mon, 28 Feb 2022 13:35:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1646051732; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QNXwyAVrDrx67iwZo0vfxJsfMtRmNd7AVvVnttje4iE=; b=tlo/IEk/TZw1GTGhi3yLO5paWt1cvQY59N19sj4jVP6zd2eekApzlzN45AgdcAVqrVC8Dr NJBvewMWBiIQn8mlHp3VaAoUYJwN4YPIR7k9M+xs7KjgXPojzPhqRIl31565/6qHitzrJK kYq/Y4GNtkl+xFbnrxv9mboQW3c84gzj5IvF3LguURjyyDv9KfJA9+xAXKcZyyr3zu43yc POo3vYPaUYryL+wh1brBFlk+X7ZgIF9NJvwzceH/pMP8i4nmbDMdNxAw28OqMtBkTvVhhy PWHoyCOanEI/RQ2hlVJwwOlaPNvcmjJVkHEP/cJWnvzKsRQXjC5MI+9M6lzII2A+2tfAef HbIn6hbfvZvT+t3cXj/MNoISO/BfRR8BU5gRp/YJy4XY8zffAObxo3HG2mTUGrSZwpXaP4 dL62px9XPBy0XCK/xPNxNlcLOt8096871hJaUT5QzYkeoeak5ZRFCmb7zBLv8A2eb4HcSZ IDxruMpizsB5X5qBbrGCqmgWnZf+y2SYZfVf2zFtLr9CbDwAkuhjkW6yYOIRrBfXoQNaG8 g4hIPA/IYpMxoA6aDsU0vI2Gu0sEQPLAGBxLqCJL9FmWC/j3p+vFzhleASZbVvw/s8bPgE xOMejvhobb/7MDSU5XixmDQxSIkXEREZYITM49t5ZXr5rW1oq13+4= From: "Kai Tetzlaff" To: Lars Ingebrigtsen Subject: Re: bug#54154: 29.0.50; [PATCH] `sieve-manage-getscript' fails if script contains multibyte characters In-Reply-To: <877d9hvf8d.fsf@gnus.org> (Lars Ingebrigtsen's message of "Sat, 26 Feb 2022 16:07:14 +0100") References: <87wnhj5nbk.fsf@tetzco.de> <878rtzxhnc.fsf@gnus.org> <875yp3w0pr.fsf@gnus.org> <87ilt33pi3.fsf@tetzco.de> <877d9hvf8d.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Date: Mon, 28 Feb 2022 13:35:30 +0100 Message-ID: <87o82rw4ml.fsf@tetzco.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Queue-Id: DDD1B6C00B7 X-Spamd-Result: default: False [-6.79 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-3.00)[-1.000]; GENERIC_REPUTATION(-0.69)[-0.68704150986515]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+]; RCPT_COUNT_TWO(0.00)[2]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Server: rakaposhi X-TOI-EXPURGATEID: 150726::1646051739-000116DC-48D18D04/0/0 CLEAN NORMAL X-TOI-MSGID: c3085302-4321-4847-b496-06633083348e X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 54154 Cc: 54154@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) --=-=-= Content-Type: text/plain Oh - the previous version of the first patch in my last email still contained a bug (I forgot to re-run `git format-patch` before sending the mail). Lars Ingebrigtsen writes: > Kai Tetzlaff writes: > >> So just reverting it won't work. I will try to undo the parts relevant >> to this issue. > > Sounds good. Ok, I'm attaching two patches which fix all issues I noticed. What I ended up with is quite a bit more than the initial attempt. Since these changes are non-trivial, I will need to do the copyright assignment. About a week ago I actually sent an email to assign@gnu.org to get the process started. But I haven't received a reply. Could you please send me the necessary papers? I'm in Germany, so my understanding is that it should be possible to do this electronically? The first (and major) set of fixes are in sieve-manage.el for the issues with multibyte characters in sieve scripts (sieve-manage-getscript/putscript). This also adds supports for multibyte characters in script names (sieve-manage-listscripts/getscript/putscript/havespace/deletescript/setactive). There is now also some handling of getscript errors reported by the server and improved logging. --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-Fix-mostly-multibyte-issues-in-sieve-manage.el-Bug-5.patch Content-Description: fix multibyte issues >From 3a4ecad9f680d130fba9e792b87824e1f5e6a6eb Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:08:07 +0100 Subject: [PATCH 1/2] Fix (mostly multibyte) issues in sieve-manage.el (Bug#54154) The managesieve protocol (s. RFC5804) requires support for (a sightly restricted variant of) UTF-8 in script content and script names. This commit fixes/improves the handling of multibyte characters. In addition, `sieve-manage-getscript' now properly handles NO responses from the server instead of inflooping. There are also some logging improvements. * lisp/net/sieve-manage.el (sieve-manage--append-to-log): (sieve-manage--message): (sieve-manage--error): (sieve-manage-encode): (sieve-manage-decode): (sieve-manage-no-p): New functions. (sieve-manage-make-process-buffer): Switch process buffer to unibyte. (sieve-manage-open-server): Add `:coding 'raw-text-unix` to `open-network-stream' call. Use unix EOLs in order to keep matching CRLF (aka "\r\n") intact. (sieve-manage-send): Make sure that UTF-8 multibyte characters are properly encoded before sending data to the server. (sieve-manage-getscript): (sieve-manage-putscript): Use the changes above to fix down/uploading scripts containing UTF-8 multibyte characters. (sieve-manage-listscripts): (sieve-manage-havespace) (sieve-manage-getscript) (sieve-manage-putscript): (sieve-manage-deletescript): (sieve-manage-setactive): Use the changes above to fix handling of script names which contain UTF-8 multibyte characters. (sieve-manage-parse-string): (sieve-manage-getscript): Add handling of server responses with type NO. Abort `sieve-manage-getscript' and show error message in message area. (sieve-manage-erase): (sieve-manage-drop-next-answer): (sieve-manage-parse-crlf): Return erased/dropped data (instead of nil). (sieve-sasl-auth): (sieve-manage-getscript): (sieve-manage-erase): (sieve-manage-open-server): (sieve-manage-open): (sieve-manage-send): Improve logging. --- lisp/net/sieve-manage.el | 126 +++++++++++++++++++++++++++------------ 1 file changed, 87 insertions(+), 39 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 50342b9105..4a36f94431 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -167,7 +167,53 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions -(autoload 'mm-enable-multibyte "mm-util") +(defun sieve-manage--append-to-log (&rest args) + "Append ARGS to sieve-manage log buffer. + +ARGS can be a string or a list of strings. +The buffer to use for logging is specifified via +`sieve-manage-log'. If it is nil, logging is disabled." + (when sieve-manage-log + (with-current-buffer (or (get-buffer sieve-manage-log) + (with-current-buffer + (get-buffer-create sieve-manage-log) + (set-buffer-multibyte nil) + (buffer-disable-undo) + (current-buffer))) + (goto-char (point-max)) + (apply #'insert args)))) + +(defun sieve-manage--message (format-string &rest args) + "Wrapper around `message' which also logs to sieve manage log. + +See `sieve-manage--append-to-log'." + (let ((ret (apply #'message + (concat "sieve-manage: " format-string) + args))) + (sieve-manage--append-to-log ret "\n") + ret)) + +(defun sieve-manage--error (format-string &rest args) + "Wrapper around `error' which also logs to sieve manage log. + +See `sieve-manage--append-to-log'." + (let ((msg (apply #'format + (concat "sieve-manage/ERROR: " format-string) + args))) + (sieve-manage--append-to-log msg "\n") + (error msg))) + +(defun sieve-manage-encode (utf8-string) + "Convert UTF8-STRING to managesieve protocol octets." + (encode-coding-string utf8-string 'raw-text t)) + +(defun sieve-manage-decode (octets &optional buffer) + "Convert managesieve protocol OCTETS to utf-8 string. + +If optional BUFFER is non-nil, insert decoded string into BUFFER." + (when octets + ;; eol type unix is required to preserve "\r\n" + (decode-coding-string octets 'utf-8-unix t buffer))) (defun sieve-manage-make-process-buffer () (with-current-buffer @@ -175,22 +221,19 @@ sieve-manage-make-process-buffer sieve-manage-server sieve-manage-port)) (mapc #'make-local-variable sieve-manage-local-variables) - (mm-enable-multibyte) + (set-buffer-multibyte nil) + (setq-local after-change-functions nil) (buffer-disable-undo) (current-buffer))) (defun sieve-manage-erase (&optional p buffer) - (let ((buffer (or buffer (current-buffer)))) - (and sieve-manage-log - (with-current-buffer (get-buffer-create sieve-manage-log) - (mm-enable-multibyte) - (buffer-disable-undo) - (goto-char (point-max)) - (insert-buffer-substring buffer (with-current-buffer buffer - (point-min)) - (or p (with-current-buffer buffer - (point-max))))))) - (delete-region (point-min) (or p (point-max)))) + (with-current-buffer (or buffer (current-buffer)) + (let* ((start (point-min)) + (end (or p (point-max))) + (logdata (buffer-substring-no-properties start end))) + (sieve-manage--append-to-log logdata) + (delete-region start end) + logdata))) (defun sieve-manage-open-server (server port &optional stream buffer) "Open network connection to SERVER on PORT. @@ -202,6 +245,8 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream + ;; eol type unix is required to preserve "\r\n" + :coding 'raw-text-unix :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" @@ -224,7 +269,7 @@ sieve-manage-open-server ;; Authenticators (defun sieve-sasl-auth (buffer mech) "Login to server using the SASL MECH method." - (message "sieve: Authenticating using %s..." mech) + (sieve-manage--message "Authenticating using %s..." mech) (with-current-buffer buffer (let* ((auth-info (auth-source-search :host sieve-manage-server :port "sieve" @@ -275,11 +320,15 @@ sieve-sasl-auth (if (and (setq step (sasl-next-step client step)) (setq data (sasl-step-data step))) ;; We got data for server but it's finished - (error "Server not ready for SASL data: %s" data) + (sieve-manage--error + "Server not ready for SASL data: %s" data) ;; The authentication process is finished. + (sieve-manage--message "Logged in as %s using %s" + user-name mech) (throw 'done t))) (unless (stringp rsp) - (error "Server aborted SASL authentication: %s" (caddr rsp))) + (sieve-manage--error + "Server aborted SASL authentication: %s" (caddr rsp))) (sasl-step-set-data step (base64-decode-string rsp)) (setq step (sasl-next-step client step)) (sieve-manage-send @@ -288,8 +337,7 @@ sieve-sasl-auth (base64-encode-string (sasl-step-data step) 'no-line-break) "\"") - "")))) - (message "sieve: Login using %s...done" mech)))) + ""))))))) (defun sieve-manage-cram-md5-p (buffer) (sieve-manage-capability "SASL" "CRAM-MD5" buffer)) @@ -353,7 +401,7 @@ sieve-manage-open sieve-manage-default-stream) sieve-manage-auth (or auth sieve-manage-auth)) - (message "sieve: Connecting to %s..." sieve-manage-server) + (sieve-manage--message "Connecting to %s..." sieve-manage-server) (sieve-manage-open-server sieve-manage-server sieve-manage-port sieve-manage-stream @@ -368,7 +416,8 @@ sieve-manage-open (setq sieve-manage-auth auth) (cl-return))) (unless sieve-manage-auth - (error "Couldn't figure out authenticator for server"))) + (sieve-manage--error + "Couldn't figure out authenticator for server"))) (sieve-manage-erase) (current-buffer)))) @@ -433,11 +482,7 @@ sieve-manage-havespace (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name - ;; Here we assume that the coding-system will - ;; replace each char with a single byte. - ;; This is always the case if `content' is - ;; a unibyte string. - (length content) + (length (sieve-manage-encode content)) sieve-manage-client-eol content)) (sieve-manage-parse-okno))) @@ -449,11 +494,10 @@ sieve-manage-deletescript (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "GETSCRIPT \"%s\"" name)) - (let ((script (sieve-manage-parse-string))) - (sieve-manage-parse-crlf) - (with-current-buffer output-buffer - (insert script)) - (sieve-manage-parse-okno)))) + (sieve-manage-decode (sieve-manage-parse-string) + output-buffer) + (sieve-manage-parse-crlf) + (sieve-manage-parse-okno))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -478,6 +522,9 @@ sieve-manage-drop-next-answer (defun sieve-manage-ok-p (rsp) (string= (downcase (or (car-safe rsp) "")) "ok")) +(defun sieve-manage-no-p (rsp) + (string= (downcase (or (car-safe rsp) "")) "no")) + (defun sieve-manage-is-okno () (when (looking-at (concat "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" @@ -528,7 +575,11 @@ sieve-manage-parse-string (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-string))) + (unless (setq rsp (sieve-manage-is-string)) + (when (sieve-manage-no-p (sieve-manage-is-okno)) + ;; simple `error' is enough since `sieve-manage-erase' + ;; already adds the server response to the log + (error (sieve-manage-erase))))) (sieve-manage-erase (point)) rsp)) @@ -540,7 +591,8 @@ sieve-manage-parse-listscripts (let (tmp rsp data) (while (null rsp) (while (null (or (setq rsp (sieve-manage-is-okno)) - (setq tmp (sieve-manage-is-string)))) + (setq tmp (sieve-manage-decode + (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min))) (when tmp @@ -559,13 +611,9 @@ sieve-manage-parse-listscripts rsp))) (defun sieve-manage-send (cmdstr) - (setq cmdstr (concat cmdstr sieve-manage-client-eol)) - (and sieve-manage-log - (with-current-buffer (get-buffer-create sieve-manage-log) - (mm-enable-multibyte) - (buffer-disable-undo) - (goto-char (point-max)) - (insert cmdstr))) + (setq cmdstr (sieve-manage-encode + (concat cmdstr sieve-manage-client-eol))) + (sieve-manage--append-to-log cmdstr) (process-send-string sieve-manage-process cmdstr)) (provide 'sieve-manage) -- 2.34.1 --=-=-= Content-Type: text/plain Both, the (internal) process/protocol buffer and the log buffer are now unibyte. The conversion to multibyte UTF-8 is only done for user visible (UI) buffers. To properly handle the protocol line termination (CRLF), I added `:coding 'raw-text-unix` (with explicit unix EOL convention) to the `open-network-stream' call (also in the new `manage-sieve-encode' function. This was needed to allow keep the various (looking-at "...\r\n" ...) calls working. This is something which still feels a bit weird, but I haven't found another way to do it. I did some tests with (setq-default buffer-file-coding-system 'utf-8-unix/'utf-8-dos) which did not show any issues. I would also add some ERT tests, probably in a separate commit? In addition, I found that `sieve-manage-quit' in sieve.el had the tendency to kill unrelated buffers in case of errors during earlier steps. For this, I created a sepate patch: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0002-Improve-robustnes-of-sieve-manage-quit-in-case-of-er.patch Content-Description: avoid killing unrelated buffers >From 83ab45907c7b528ae4db98f33415e05e679c312e Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:33:56 +0100 Subject: [PATCH 2/2] Improve robustnes of `sieve-manage-quit' in case of errors * lisp/net/sieve.el (sieve-manage-quit): Avoid killing buffers it's not supposed to touch. --- lisp/net/sieve.el | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index 630ea04070..5680526389 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -154,7 +154,8 @@ sieve-manage-quit (interactive) (sieve-manage-close sieve-manage-buffer) (kill-buffer sieve-manage-buffer) - (kill-buffer (current-buffer))) + (when-let ((buffer (get-buffer sieve-buffer))) + (kill-buffer buffer))) (defun sieve-bury-buffer () "Bury the Manage Sieve buffer without closing the connection." -- 2.34.1 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Feb 28 08:07:07 2022 Received: (at 54154) by debbugs.gnu.org; 28 Feb 2022 13:07:07 +0000 Received: from localhost ([127.0.0.1]:60477 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nOfk3-0005Cz-2f for submit@debbugs.gnu.org; Mon, 28 Feb 2022 08:07:07 -0500 Received: from quimby.gnus.org ([95.216.78.240]:36796) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nOfk1-0005CQ-Op for 54154@debbugs.gnu.org; Mon, 28 Feb 2022 08:07:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date: References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=sUYL8Ltm6kcIncs9acQLiIs5Xs41wlxFJhjF1giMczo=; b=NdBj0fyqkq52LGpAwpBOUFeLSK ztL+lDlBGrmcvBQr8DM1nrAYZ2VwVNVCnD+PfJkc7SY6duhH485EdmVJN5TIRJ7CWmw4TRyKjWj23 dIRa9O6K4Dd0z+XfKQ1iqoBdu0G+JcOuuUMwQtOROtUAkzpnQMzjz2G/PsaGl2S1Z3sQ=; Received: from [84.212.220.105] (helo=giant) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nOfjt-00084n-CW; Mon, 28 Feb 2022 14:06:59 +0100 From: Lars Ingebrigtsen To: "Kai Tetzlaff" Subject: Re: bug#54154: 29.0.50; [PATCH] `sieve-manage-getscript' fails if script contains multibyte characters References: <87wnhj5nbk.fsf@tetzco.de> <878rtzxhnc.fsf@gnus.org> <875yp3w0pr.fsf@gnus.org> <87ilt33pi3.fsf@tetzco.de> <877d9hvf8d.fsf@gnus.org> <87o82rw4ml.fsf@tetzco.de> Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAABGdBTUEAALGPC/xhBQAAACBj SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAD1BMVEWdypd5tIRkoHEY JSX////pchdQAAAAAWJLR0QEj2jZUQAAAAd0SU1FB+YCHAwdMyVgHRYAAAGxSURBVDjLdZNhouMg CITFEwC9gOAFXsP977YDatLX3bXNj/g5DCBpJNJIWiNuTZjaXtSUiJS6sxiz5Q4vrIJtISEWlgHd ESVQuV+f5aLqBZjXjsE1FQZkfMxXqHy0pXn7e8EDv/8BhKUhYiOL4lWUZ6gsEek2rjq2wlyhUN0K 6rzMvTyQsaBAIyb0SAqYQrEAKmE5AJtproIw4r4KKoB3PM4wnxFhv8wdP6gil/xSkELxKjBOE/eC Yha4Tq+QxlbEDUpheRrpGiyuJPd9VFb5jyEOwF8KeLPofADalHUU4A/FThf90OAUPEALoH1h5T4O oG0uIfEDyc/dkjJHgeExAN6rwBxQ5ey1vIa0BVaovj2aDu5xgzVwWYejpGzwew+cYBjW7PZhrycr WOMsrTtXnx+gmo6s3KZks95y98prhFzmyMu69vi40BaZX3W7V0nCEfdco8e+LHwY955i23xuFA/I 488+gGT0HCC83bEKoA7MYsfh+QmQJnWrO415fYSy9blVzX7FA6jVJ1tBtuA6CnSz+3yibI+GMZTu 8bWQFSGrf4Icqn4CPZVjPBBK57fiDzF0aGQ0w9opAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIyLTAy LTI4VDEyOjI5OjUwKzAwOjAw4J1uWQAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMi0wMi0yOFQxMjoy OTo1MCswMDowMJHA1uUAAAAASUVORK5CYII= X-Now-Playing: Blaine L. Reininger's _Songs From The Rain Palace_: "Where Are You" Date: Mon, 28 Feb 2022 14:06:56 +0100 In-Reply-To: <87o82rw4ml.fsf@tetzco.de> (Kai Tetzlaff's message of "Mon, 28 Feb 2022 13:35:30 +0100") Message-ID: <87y21vm973.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: "Kai Tetzlaff" writes: > Ok, I'm attaching two patches which fix all issues I noticed. The patches look good to me, but I haven't tried them myself, because I don't use sieve-manage. If somebody who does could try the patches, that would be helpful. Anybody? Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54154 Cc: 54154@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) "Kai Tetzlaff" writes: > Ok, I'm attaching two patches which fix all issues I noticed. The patches look good to me, but I haven't tried them myself, because I don't use sieve-manage. If somebody who does could try the patches, that would be helpful. Anybody? > What I ended up with is quite a bit more than the initial attempt. Since > these changes are non-trivial, I will need to do the copyright > assignment. About a week ago I actually sent an email to assign@gnu.org > to get the process started. But I haven't received a reply. Could you > please send me the necessary papers? I'm in Germany, so my understanding > is that it should be possible to do this electronically? It sometimes takes a while to get the process going -- if you don't get a response from the copyright clerk within a couple more days, send me an email and I'll get in touch with them to see what's up. From debbugs-submit-bounces@debbugs.gnu.org Mon Feb 28 08:08:31 2022 Received: (at 54154) by debbugs.gnu.org; 28 Feb 2022 13:08:32 +0000 Received: from localhost ([127.0.0.1]:60486 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nOflP-0005G1-Ky for submit@debbugs.gnu.org; Mon, 28 Feb 2022 08:08:31 -0500 Received: from quimby.gnus.org ([95.216.78.240]:36842) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nOflA-0005FS-Il for 54154@debbugs.gnu.org; Mon, 28 Feb 2022 08:08:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:MIME-Version:Message-ID:Date:References: In-Reply-To:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=qT7RZeukXNAue7KzIfwIV1x8GtEiSFpipo6FBUtVqhg=; b=JdRJ7XxcnELrj4maFcNmFQ9bft Nf8ABBQi/0pF1NHDi2SSLlT7B1dXdkelWJlmflkFsZyvNTbL8MsPNsiwvODGKjJOENxavnyb8FsGD AxdbeUe94eo171imDTcaQx1Eiq9tsXG0GsZNDM1gcoe0DARzhwUeCofBEGSY4kzedyBc=; Received: from [84.212.220.105] (helo=giant) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nOfl2-00085Y-Fx; Mon, 28 Feb 2022 14:08:10 +0100 From: Lars Ingebrigtsen To: emacs@tetzco.de Subject: Re: bug#54154: 29.0.50; [PATCH] `sieve-manage-getscript' fails if script contains multibyte characters In-Reply-To: <87o82rw4ml.fsf@tetzco.de> (Kai Tetzlaff's message of "Mon, 28 Feb 2022 13:35:30 +0100") References: <87wnhj5nbk.fsf@tetzco.de> <878rtzxhnc.fsf@gnus.org> <875yp3w0pr.fsf@gnus.org> <87ilt33pi3.fsf@tetzco.de> <877d9hvf8d.fsf@gnus.org> <87o82rw4ml.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAABGdBTUEAALGPC/xhBQAAACBj SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAAD1BMVEWdypd5tIRkoHEY JSX////pchdQAAAAAWJLR0QEj2jZUQAAAAd0SU1FB+YCHAwdMyVgHRYAAAGxSURBVDjLdZNhouMg CITFEwC9gOAFXsP977YDatLX3bXNj/g5DCBpJNJIWiNuTZjaXtSUiJS6sxiz5Q4vrIJtISEWlgHd ESVQuV+f5aLqBZjXjsE1FQZkfMxXqHy0pXn7e8EDv/8BhKUhYiOL4lWUZ6gsEek2rjq2wlyhUN0K 6rzMvTyQsaBAIyb0SAqYQrEAKmE5AJtproIw4r4KKoB3PM4wnxFhv8wdP6gil/xSkELxKjBOE/eC Yha4Tq+QxlbEDUpheRrpGiyuJPd9VFb5jyEOwF8KeLPofADalHUU4A/FThf90OAUPEALoH1h5T4O oG0uIfEDyc/dkjJHgeExAN6rwBxQ5ey1vIa0BVaovj2aDu5xgzVwWYejpGzwew+cYBjW7PZhrycr WOMsrTtXnx+gmo6s3KZks95y98prhFzmyMu69vi40BaZX3W7V0nCEfdco8e+LHwY955i23xuFA/I 488+gGT0HCC83bEKoA7MYsfh+QmQJnWrO415fYSy9blVzX7FA6jVJ1tBtuA6CnSz+3yibI+GMZTu 8bWQFSGrf4Icqn4CPZVjPBBK57fiDzF0aGQ0w9opAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIyLTAy LTI4VDEyOjI5OjUwKzAwOjAw4J1uWQAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMi0wMi0yOFQxMjoy OTo1MCswMDowMJHA1uUAAAAASUVORK5CYII= X-Now-Playing: Blaine L. Reininger's _Songs From The Rain Palace_: "Where Are You" Date: Mon, 28 Feb 2022 14:08:06 +0100 Message-ID: <87v8wzm955.fsf@gnus.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: (Re-sending because the previous mail went to to-online.de, which rejected it.) "Kai Tetzlaff" writes: > Ok, I'm attaching two patches which fix all issues I noticed. Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: 54154@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) (Re-sending because the previous mail went to to-online.de, which rejected it.) "Kai Tetzlaff" writes: > Ok, I'm attaching two patches which fix all issues I noticed. The patches look good to me, but I haven't tried them myself, because I don't use sieve-manage. If somebody who does could try the patches, that would be helpful. Anybody? > What I ended up with is quite a bit more than the initial attempt. Since > these changes are non-trivial, I will need to do the copyright > assignment. About a week ago I actually sent an email to assign@gnu.org > to get the process started. But I haven't received a reply. Could you > please send me the necessary papers? I'm in Germany, so my understanding > is that it should be possible to do this electronically? It sometimes takes a while to get the process going -- if you don't get a response from the copyright clerk within a couple more days, send me an email and I'll get in touch with them to see what's up. From debbugs-submit-bounces@debbugs.gnu.org Mon Feb 28 11:19:29 2022 Received: (at 54154) by debbugs.gnu.org; 28 Feb 2022 16:19:29 +0000 Received: from localhost ([127.0.0.1]:34597 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nOik8-0004JL-R1 for submit@debbugs.gnu.org; Mon, 28 Feb 2022 11:19:29 -0500 Received: from mail1.tetzco.de ([140.238.222.122]:42703 helo=mx2.tetzco.de) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nOfgX-000579-BD for 54154@debbugs.gnu.org; Mon, 28 Feb 2022 08:03:30 -0500 Received: from mail.tetzco.de (unknown [IPv6:2a02:810d:1380:7900::9]) by mx2.tetzco.de (Postfix) with ESMTPS id 525C3BD32C; Mon, 28 Feb 2022 14:03:22 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id B34DC6C00B7; Mon, 28 Feb 2022 14:03:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1646053400; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QNXwyAVrDrx67iwZo0vfxJsfMtRmNd7AVvVnttje4iE=; b=oK3AgXzytBd28neVh01DU5LQQXYSEY0hH+AOCcl4+Ce+G1557SjWbDst+rYqRvrSQlz1l0 sBoLCUCquDFZW1SNXqmPZEBfqBtLGxTuz1hbm6yc83tDTjteWGCn8kVQoXG81TbHiY0Dv/ LrcGDSKINBLzPaumYdkefYlfNf43LEGqNsvSBTdTCCiWYmlZjcgy6L1G4A3+zwjQb7Ygru 3WMbHrRK+B5JwniazikFz24ZiWdtnQZxOOX9OfQpZqNVXaf7Fa7P1Q9M4zcJVwiALqSmOA ns0e7x1BM1YtSu/BwGFDg2ZNWlPw3VJxQWxnzNSgTriX8IDWyxtjO4S7h9Zhyudqg68p3F 6WSIAMU67QERB1BfJi98qLiu5y80M+JR9731n89f3W/HTfTedRloFHW75T6jF6b2xo2muw 00DtXTujjoaGJPox8D5c6TLMSUC/A/rk+/I8f1V+trDez79a9z6J9xauOgWH7VFdFz2Jv8 9qfJVu9YgKZrrRS81KMzHgaM0hHIyUg7FJuVyLhZToh9GwXRf6Iv3VYnEvXUi6lN/eMvbT +E76fUAevya+4kvLE+11V7VIQ7xMY/lr1fk6Ud6GYfF9Q5e3JT3AKjycU0lPSuILjhiQ9M ZbMVubRS+UbEmCs2Oh3jApqIjMv9AsmbDUJOUHYdL0IzuAWPGJM2o= From: Kai Tetzlaff To: Lars Ingebrigtsen Subject: Re: bug#54154: 29.0.50; [PATCH] `sieve-manage-getscript' fails if script contains multibyte characters In-Reply-To: <877d9hvf8d.fsf@gnus.org> (Lars Ingebrigtsen's message of "Sat, 26 Feb 2022 16:07:14 +0100") References: <87wnhj5nbk.fsf@tetzco.de> <878rtzxhnc.fsf@gnus.org> <875yp3w0pr.fsf@gnus.org> <87ilt33pi3.fsf@tetzco.de> <877d9hvf8d.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Date: Mon, 28 Feb 2022 14:03:19 +0100 Message-ID: <87mtibw3c8.fsf@tetzco.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Queue-Id: B34DC6C00B7 X-Spamd-Result: default: False [-6.79 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-3.00)[-1.000]; GENERIC_REPUTATION(-0.69)[-0.6870502201696]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+]; RCPT_COUNT_TWO(0.00)[2]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Server: rakaposhi X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 54154 X-Mailman-Approved-At: Mon, 28 Feb 2022 11:19:23 -0500 Cc: 54154@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Oh - the previous version of the first patch in my last email still contained a bug (I forgot to re-run `git format-patch` before sending the mail). Lars Ingebrigtsen writes: > Kai Tetzlaff writes: > >> So just reverting it won't work. I will try to undo the parts relevant >> to this issue. > > Sounds good. Ok, I'm attaching two patches which fix all issues I noticed. What I ended up with is quite a bit more than the initial attempt. Since these changes are non-trivial, I will need to do the copyright assignment. About a week ago I actually sent an email to assign@gnu.org to get the process started. But I haven't received a reply. Could you please send me the necessary papers? I'm in Germany, so my understanding is that it should be possible to do this electronically? The first (and major) set of fixes are in sieve-manage.el for the issues with multibyte characters in sieve scripts (sieve-manage-getscript/putscript). This also adds supports for multibyte characters in script names (sieve-manage-listscripts/getscript/putscript/havespace/deletescript/setactive). There is now also some handling of getscript errors reported by the server and improved logging. --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-Fix-mostly-multibyte-issues-in-sieve-manage.el-Bug-5.patch Content-Description: fix multibyte issues >From 3a4ecad9f680d130fba9e792b87824e1f5e6a6eb Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:08:07 +0100 Subject: [PATCH 1/2] Fix (mostly multibyte) issues in sieve-manage.el (Bug#54154) The managesieve protocol (s. RFC5804) requires support for (a sightly restricted variant of) UTF-8 in script content and script names. This commit fixes/improves the handling of multibyte characters. In addition, `sieve-manage-getscript' now properly handles NO responses from the server instead of inflooping. There are also some logging improvements. * lisp/net/sieve-manage.el (sieve-manage--append-to-log): (sieve-manage--message): (sieve-manage--error): (sieve-manage-encode): (sieve-manage-decode): (sieve-manage-no-p): New functions. (sieve-manage-make-process-buffer): Switch process buffer to unibyte. (sieve-manage-open-server): Add `:coding 'raw-text-unix` to `open-network-stream' call. Use unix EOLs in order to keep matching CRLF (aka "\r\n") intact. (sieve-manage-send): Make sure that UTF-8 multibyte characters are properly encoded before sending data to the server. (sieve-manage-getscript): (sieve-manage-putscript): Use the changes above to fix down/uploading scripts containing UTF-8 multibyte characters. (sieve-manage-listscripts): (sieve-manage-havespace) (sieve-manage-getscript) (sieve-manage-putscript): (sieve-manage-deletescript): (sieve-manage-setactive): Use the changes above to fix handling of script names which contain UTF-8 multibyte characters. (sieve-manage-parse-string): (sieve-manage-getscript): Add handling of server responses with type NO. Abort `sieve-manage-getscript' and show error message in message area. (sieve-manage-erase): (sieve-manage-drop-next-answer): (sieve-manage-parse-crlf): Return erased/dropped data (instead of nil). (sieve-sasl-auth): (sieve-manage-getscript): (sieve-manage-erase): (sieve-manage-open-server): (sieve-manage-open): (sieve-manage-send): Improve logging. --- lisp/net/sieve-manage.el | 126 +++++++++++++++++++++++++++------------ 1 file changed, 87 insertions(+), 39 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 50342b9105..4a36f94431 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -167,7 +167,53 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions -(autoload 'mm-enable-multibyte "mm-util") +(defun sieve-manage--append-to-log (&rest args) + "Append ARGS to sieve-manage log buffer. + +ARGS can be a string or a list of strings. +The buffer to use for logging is specifified via +`sieve-manage-log'. If it is nil, logging is disabled." + (when sieve-manage-log + (with-current-buffer (or (get-buffer sieve-manage-log) + (with-current-buffer + (get-buffer-create sieve-manage-log) + (set-buffer-multibyte nil) + (buffer-disable-undo) + (current-buffer))) + (goto-char (point-max)) + (apply #'insert args)))) + +(defun sieve-manage--message (format-string &rest args) + "Wrapper around `message' which also logs to sieve manage log. + +See `sieve-manage--append-to-log'." + (let ((ret (apply #'message + (concat "sieve-manage: " format-string) + args))) + (sieve-manage--append-to-log ret "\n") + ret)) + +(defun sieve-manage--error (format-string &rest args) + "Wrapper around `error' which also logs to sieve manage log. + +See `sieve-manage--append-to-log'." + (let ((msg (apply #'format + (concat "sieve-manage/ERROR: " format-string) + args))) + (sieve-manage--append-to-log msg "\n") + (error msg))) + +(defun sieve-manage-encode (utf8-string) + "Convert UTF8-STRING to managesieve protocol octets." + (encode-coding-string utf8-string 'raw-text t)) + +(defun sieve-manage-decode (octets &optional buffer) + "Convert managesieve protocol OCTETS to utf-8 string. + +If optional BUFFER is non-nil, insert decoded string into BUFFER." + (when octets + ;; eol type unix is required to preserve "\r\n" + (decode-coding-string octets 'utf-8-unix t buffer))) (defun sieve-manage-make-process-buffer () (with-current-buffer @@ -175,22 +221,19 @@ sieve-manage-make-process-buffer sieve-manage-server sieve-manage-port)) (mapc #'make-local-variable sieve-manage-local-variables) - (mm-enable-multibyte) + (set-buffer-multibyte nil) + (setq-local after-change-functions nil) (buffer-disable-undo) (current-buffer))) (defun sieve-manage-erase (&optional p buffer) - (let ((buffer (or buffer (current-buffer)))) - (and sieve-manage-log - (with-current-buffer (get-buffer-create sieve-manage-log) - (mm-enable-multibyte) - (buffer-disable-undo) - (goto-char (point-max)) - (insert-buffer-substring buffer (with-current-buffer buffer - (point-min)) - (or p (with-current-buffer buffer - (point-max))))))) - (delete-region (point-min) (or p (point-max)))) + (with-current-buffer (or buffer (current-buffer)) + (let* ((start (point-min)) + (end (or p (point-max))) + (logdata (buffer-substring-no-properties start end))) + (sieve-manage--append-to-log logdata) + (delete-region start end) + logdata))) (defun sieve-manage-open-server (server port &optional stream buffer) "Open network connection to SERVER on PORT. @@ -202,6 +245,8 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream + ;; eol type unix is required to preserve "\r\n" + :coding 'raw-text-unix :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" @@ -224,7 +269,7 @@ sieve-manage-open-server ;; Authenticators (defun sieve-sasl-auth (buffer mech) "Login to server using the SASL MECH method." - (message "sieve: Authenticating using %s..." mech) + (sieve-manage--message "Authenticating using %s..." mech) (with-current-buffer buffer (let* ((auth-info (auth-source-search :host sieve-manage-server :port "sieve" @@ -275,11 +320,15 @@ sieve-sasl-auth (if (and (setq step (sasl-next-step client step)) (setq data (sasl-step-data step))) ;; We got data for server but it's finished - (error "Server not ready for SASL data: %s" data) + (sieve-manage--error + "Server not ready for SASL data: %s" data) ;; The authentication process is finished. + (sieve-manage--message "Logged in as %s using %s" + user-name mech) (throw 'done t))) (unless (stringp rsp) - (error "Server aborted SASL authentication: %s" (caddr rsp))) + (sieve-manage--error + "Server aborted SASL authentication: %s" (caddr rsp))) (sasl-step-set-data step (base64-decode-string rsp)) (setq step (sasl-next-step client step)) (sieve-manage-send @@ -288,8 +337,7 @@ sieve-sasl-auth (base64-encode-string (sasl-step-data step) 'no-line-break) "\"") - "")))) - (message "sieve: Login using %s...done" mech)))) + ""))))))) (defun sieve-manage-cram-md5-p (buffer) (sieve-manage-capability "SASL" "CRAM-MD5" buffer)) @@ -353,7 +401,7 @@ sieve-manage-open sieve-manage-default-stream) sieve-manage-auth (or auth sieve-manage-auth)) - (message "sieve: Connecting to %s..." sieve-manage-server) + (sieve-manage--message "Connecting to %s..." sieve-manage-server) (sieve-manage-open-server sieve-manage-server sieve-manage-port sieve-manage-stream @@ -368,7 +416,8 @@ sieve-manage-open (setq sieve-manage-auth auth) (cl-return))) (unless sieve-manage-auth - (error "Couldn't figure out authenticator for server"))) + (sieve-manage--error + "Couldn't figure out authenticator for server"))) (sieve-manage-erase) (current-buffer)))) @@ -433,11 +482,7 @@ sieve-manage-havespace (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name - ;; Here we assume that the coding-system will - ;; replace each char with a single byte. - ;; This is always the case if `content' is - ;; a unibyte string. - (length content) + (length (sieve-manage-encode content)) sieve-manage-client-eol content)) (sieve-manage-parse-okno))) @@ -449,11 +494,10 @@ sieve-manage-deletescript (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "GETSCRIPT \"%s\"" name)) - (let ((script (sieve-manage-parse-string))) - (sieve-manage-parse-crlf) - (with-current-buffer output-buffer - (insert script)) - (sieve-manage-parse-okno)))) + (sieve-manage-decode (sieve-manage-parse-string) + output-buffer) + (sieve-manage-parse-crlf) + (sieve-manage-parse-okno))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -478,6 +522,9 @@ sieve-manage-drop-next-answer (defun sieve-manage-ok-p (rsp) (string= (downcase (or (car-safe rsp) "")) "ok")) +(defun sieve-manage-no-p (rsp) + (string= (downcase (or (car-safe rsp) "")) "no")) + (defun sieve-manage-is-okno () (when (looking-at (concat "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" @@ -528,7 +575,11 @@ sieve-manage-parse-string (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-string))) + (unless (setq rsp (sieve-manage-is-string)) + (when (sieve-manage-no-p (sieve-manage-is-okno)) + ;; simple `error' is enough since `sieve-manage-erase' + ;; already adds the server response to the log + (error (sieve-manage-erase))))) (sieve-manage-erase (point)) rsp)) @@ -540,7 +591,8 @@ sieve-manage-parse-listscripts (let (tmp rsp data) (while (null rsp) (while (null (or (setq rsp (sieve-manage-is-okno)) - (setq tmp (sieve-manage-is-string)))) + (setq tmp (sieve-manage-decode + (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min))) (when tmp @@ -559,13 +611,9 @@ sieve-manage-parse-listscripts rsp))) (defun sieve-manage-send (cmdstr) - (setq cmdstr (concat cmdstr sieve-manage-client-eol)) - (and sieve-manage-log - (with-current-buffer (get-buffer-create sieve-manage-log) - (mm-enable-multibyte) - (buffer-disable-undo) - (goto-char (point-max)) - (insert cmdstr))) + (setq cmdstr (sieve-manage-encode + (concat cmdstr sieve-manage-client-eol))) + (sieve-manage--append-to-log cmdstr) (process-send-string sieve-manage-process cmdstr)) (provide 'sieve-manage) -- 2.34.1 --=-=-= Content-Type: text/plain Both, the (internal) process/protocol buffer and the log buffer are now unibyte. The conversion to multibyte UTF-8 is only done for user visible (UI) buffers. To properly handle the protocol line termination (CRLF), I added `:coding 'raw-text-unix` (with explicit unix EOL convention) to the `open-network-stream' call (also in the new `manage-sieve-encode' function. This was needed to allow keep the various (looking-at "...\r\n" ...) calls working. This is something which still feels a bit weird, but I haven't found another way to do it. I did some tests with (setq-default buffer-file-coding-system 'utf-8-unix/'utf-8-dos) which did not show any issues. I would also add some ERT tests, probably in a separate commit? In addition, I found that `sieve-manage-quit' in sieve.el had the tendency to kill unrelated buffers in case of errors during earlier steps. For this, I created a sepate patch: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0002-Improve-robustnes-of-sieve-manage-quit-in-case-of-er.patch Content-Description: avoid killing unrelated buffers >From 83ab45907c7b528ae4db98f33415e05e679c312e Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:33:56 +0100 Subject: [PATCH 2/2] Improve robustnes of `sieve-manage-quit' in case of errors * lisp/net/sieve.el (sieve-manage-quit): Avoid killing buffers it's not supposed to touch. --- lisp/net/sieve.el | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index 630ea04070..5680526389 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -154,7 +154,8 @@ sieve-manage-quit (interactive) (sieve-manage-close sieve-manage-buffer) (kill-buffer sieve-manage-buffer) - (kill-buffer (current-buffer))) + (when-let ((buffer (get-buffer sieve-buffer))) + (kill-buffer buffer))) (defun sieve-bury-buffer () "Bury the Manage Sieve buffer without closing the connection." -- 2.34.1 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Feb 28 11:19:36 2022 Received: (at 54154) by debbugs.gnu.org; 28 Feb 2022 16:19:36 +0000 Received: from localhost ([127.0.0.1]:34599 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nOikJ-0004Ji-HB for submit@debbugs.gnu.org; Mon, 28 Feb 2022 11:19:36 -0500 Received: from mout-p-101.mailbox.org ([80.241.56.151]:47242) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nOf8K-0001p4-GO for 54154@debbugs.gnu.org; Mon, 28 Feb 2022 07:28:13 -0500 Received: from smtp1.mailbox.org (smtp1.mailbox.org [IPv6:2001:67c:2050:105:465:1:1:0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-101.mailbox.org (Postfix) with ESMTPS id 4K6fmL0Mshz9sHY; Mon, 28 Feb 2022 13:28:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzlaff.eu; s=20210624; t=1646051266; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=T/k60dYoBh92lO3qi7hK/VTqHEWeb4DOzBpbBz2lGb8=; b=ZZSttbTA47u4BdoEOZOxoAxX0hEBVMtQLAVPAhzYT5TcdX4P/O8+4bJ18waILIVvLg2mNB oiFjVxzd2nA/3++bI4HFhUHmuckpVHJYgUSan7bODqwpvTyx0PUjT/lQLZnqE+xv9xiCDQ h9AredSj4eJ8i0tw5AVvp7gHXw5inbMB0CTmdKR2yql55uBptdYH0KkLukH/3LPl+3UAY6 jV2wsG/hxyRtrAWDnU2wtP6Pp0tojb/+YAWsBwcmEpJqz3QejDAFeAhJXN2zE4fPplO69B Dhu0nhkpyKaEKRk2rP2NFOcI+Nb0QxUXEsi85YqpjX0j1EtFGopVjpl51ZO77D/jdR+wVk mjOUl9fXqp5VfCmurl3am3XBglyA4mTdlgQNxKfdo4fxKTDsZtkmS1sxPUM5fGXh2fwTb2 TbJFH+8aSR0oWnlLmWpQLGjTgPKywJfgXh/VKwJ9X41xqBk2Y1R34acy4MCESGwjfbiOSA 0ViRVgnx3iwX486UoJe4tE0qp1OLb9tzIsh42KcCjf3KfvmFXjkWkn0nI9ofVzrLNhV0Oj fN9tPfEF2GmEuVrPNlrncGaC64Up8pVlRIPpSzjwmwdzesZKfKku545Bgaex05haSxOW3j hrZb46hel5/2f/ruYDRH/NmzHoZgs9tdyRZNAmLdaLCEJ37EfpHGs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzlaff.eu; s=MBO0001; t=1646051280; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=T/k60dYoBh92lO3qi7hK/VTqHEWeb4DOzBpbBz2lGb8=; b=Fw75l3w+nysQ+bPRUqtd7WC0UJ6LJYP5z7aN22/3iGT4iTc5Cg4ZXbRNMzWI/TmClVOhhK SgPpP3t9IC/6I9/XYETxVgwm93yMRQvwIPHUYgNJ5lSgB8S9f15ae+IwxXharA4f7l2YEJ 2rMJOWqSb+7DN0LbNpPBy47NkyLSU3pL2MhIZMziNAlpMBFCq5NLgM6cvbppNSD/8iYWW8 9UMcZ6pX4FRqGuzoxqrtAcsm5nhEr71PlUrBxjKaICCnPgZg7zXPuuxjUq+XvPx5Y+xwds LnpHOuqTyzWkyW3FDo4CawZdszpE3LPQpz+756gCHyG6ImfGgJK1gotTWCTIlA== From: Kai Tetzlaff To: Lars Ingebrigtsen Subject: Re: bug#54154: 29.0.50; [PATCH] `sieve-manage-getscript' fails if script contains multibyte characters References: <87wnhj5nbk.fsf@tetzco.de> <878rtzxhnc.fsf@gnus.org> <875yp3w0pr.fsf@gnus.org> <87ilt33pi3.fsf@tetzco.de> <877d9hvf8d.fsf@gnus.org> Date: Mon, 28 Feb 2022 13:27:42 +0100 In-Reply-To: <877d9hvf8d.fsf@gnus.org> (Lars Ingebrigtsen's message of "Sat, 26 Feb 2022 16:07:14 +0100") Message-ID: <87sfs3w4zl.fsf@tetzco.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Queue-Id: 40FDB6C00B7 X-Spamd-Result: default: False [-6.79 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-3.00)[-1.000]; GENERIC_REPUTATION(-0.69)[-0.68541051511092]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; DKIM_SIGNED(0.00)[tetzlaff.eu:s=20210624]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Server: rakaposhi X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 54154 X-Mailman-Approved-At: Mon, 28 Feb 2022 11:19:34 -0500 Cc: 54154@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --=-=-= Content-Type: text/plain Lars Ingebrigtsen writes: > Kai Tetzlaff writes: > >> So just reverting it won't work. I will try to undo the parts relevant >> to this issue. > > Sounds good. Ok, I'm attaching two patches which fix all issues I noticed. What I ended up with is quite a bit more than the initial attempt. Since these changes are non-trivial, I will need to do the copyright assignment. About a week ago I actually sent an email to assign@gnu.org to get the process started. But I haven't received a reply. Could you please send me the necessary papers? I'm in Germany, so my understanding is that it should be possible to do this electronically? The first (and major) set of fixes are in sieve-manage.el for the issues with multibyte characters in sieve scripts (sieve-manage-getscript/putscript). This also adds supports for multibyte characters in script names (sieve-manage-listscripts/getscript/putscript/havespace/deletescript/setactive). There is now also some handling of getscript errors reported by the server and improved logging. --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-Fix-mostly-multibyte-issues-in-sieve-manage.el-Bug-5.patch Content-Description: fix multibyte issues >From fd18929ce2004f7448ab997bc86e206afdbd8673 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:08:07 +0100 Subject: [PATCH 1/2] Fix (mostly multibyte) issues in sieve-manage.el (Bug#54154) The managesieve protocol (s. RFC5804) requires support for (a sightly restricted variant of) UTF-8 in script content and script names. This commit fixes/improves the handling of multibyte characters. In addition, `sieve-manage-getscript' now properly handles NO responses from the server instead of inflooping. There are also some logging improvements. * lisp/net/sieve-manage.el (sieve-manage--append-to-log): (sieve-manage--message): (sieve-manage--error): (sieve-manage-encode): (sieve-manage-decode): (sieve-manage-no-p): New functions. (sieve-manage-make-process-buffer): Switch process buffer to unibyte. (sieve-manage-open-server): Add `:coding 'raw-text-unix` to `open-network-stream' call. Use unix EOLs in order to keep matching CRLF (aka "\r\n") intact. (sieve-manage-send): Make sure that UTF-8 multibyte characters are properly encoded before sending data to the server. (sieve-manage-getscript): (sieve-manage-putscript): Use the changes above to fix down/uploading scripts containing UTF-8 multibyte characters. (sieve-manage-listscripts): (sieve-manage-havespace) (sieve-manage-getscript) (sieve-manage-putscript): (sieve-manage-deletescript): (sieve-manage-setactive): Use the changes above to fix handling of script names which contain UTF-8 multibyte characters. (sieve-manage-parse-string): (sieve-manage-getscript): Add handling of server responses with type NO. Abort `sieve-manage-getscript' and show error message in message area. (sieve-manage-erase): (sieve-manage-drop-next-answer): (sieve-manage-parse-crlf): Return erased/dropped data (instead of nil). (sieve-sasl-auth): (sieve-manage-getscript): (sieve-manage-erase): (sieve-manage-open-server): (sieve-manage-open): (sieve-manage-send): Improve logging. --- lisp/net/sieve-manage.el | 125 +++++++++++++++++++++++++++------------ 1 file changed, 86 insertions(+), 39 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 50342b9105..a57d81efcd 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -167,7 +167,52 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions -(autoload 'mm-enable-multibyte "mm-util") +(defun sieve-manage--append-to-log (&rest args) + "Append ARGS to sieve-manage log buffer. + +ARGS can be a string or a list of strings. +The buffer to use for logging is specifified via +`sieve-manage-log'. If it is nil, logging is disabled." + (when sieve-manage-log + (with-current-buffer (or (get-buffer sieve-manage-log) + (with-current-buffer + (get-buffer-create sieve-manage-log) + (set-buffer-multibyte nil) + (buffer-disable-undo))) + (goto-char (point-max)) + (apply #'insert args)))) + +(defun sieve-manage--message (format-string &rest args) + "Wrapper around `message' which also logs to sieve manage log. + +See `sieve-manage--append-to-log'." + (let ((ret (apply #'message + (concat "sieve-manage: " format-string) + args))) + (sieve-manage--append-to-log ret "\n") + ret)) + +(defun sieve-manage--error (format-string &rest args) + "Wrapper around `error' which also logs to sieve manage log. + +See `sieve-manage--append-to-log'." + (let ((msg (apply #'format + (concat "sieve-manage/ERROR: " format-string) + args))) + (sieve-manage--append-to-log msg "\n") + (error msg))) + +(defun sieve-manage-encode (utf8-string) + "Convert UTF8-STRING to managesieve protocol octets." + (encode-coding-string utf8-string 'raw-text t)) + +(defun sieve-manage-decode (octets &optional buffer) + "Convert managesieve protocol OCTETS to utf-8 string. + +If optional BUFFER is non-nil, insert decoded string into BUFFER." + (when octets + ;; eol type unix is required to preserve "\r\n" + (decode-coding-string octets 'utf-8-unix t buffer))) (defun sieve-manage-make-process-buffer () (with-current-buffer @@ -175,22 +220,19 @@ sieve-manage-make-process-buffer sieve-manage-server sieve-manage-port)) (mapc #'make-local-variable sieve-manage-local-variables) - (mm-enable-multibyte) + (set-buffer-multibyte nil) + (setq-local after-change-functions nil) (buffer-disable-undo) (current-buffer))) (defun sieve-manage-erase (&optional p buffer) - (let ((buffer (or buffer (current-buffer)))) - (and sieve-manage-log - (with-current-buffer (get-buffer-create sieve-manage-log) - (mm-enable-multibyte) - (buffer-disable-undo) - (goto-char (point-max)) - (insert-buffer-substring buffer (with-current-buffer buffer - (point-min)) - (or p (with-current-buffer buffer - (point-max))))))) - (delete-region (point-min) (or p (point-max)))) + (with-current-buffer (or buffer (current-buffer)) + (let* ((start (point-min)) + (end (or p (point-max))) + (logdata (buffer-substring-no-properties start end))) + (sieve-manage--append-to-log logdata) + (delete-region start end) + logdata))) (defun sieve-manage-open-server (server port &optional stream buffer) "Open network connection to SERVER on PORT. @@ -202,6 +244,8 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream + ;; eol type unix is required to preserve "\r\n" + :coding 'raw-text-unix :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" @@ -224,7 +268,7 @@ sieve-manage-open-server ;; Authenticators (defun sieve-sasl-auth (buffer mech) "Login to server using the SASL MECH method." - (message "sieve: Authenticating using %s..." mech) + (sieve-manage--message "Authenticating using %s..." mech) (with-current-buffer buffer (let* ((auth-info (auth-source-search :host sieve-manage-server :port "sieve" @@ -275,11 +319,15 @@ sieve-sasl-auth (if (and (setq step (sasl-next-step client step)) (setq data (sasl-step-data step))) ;; We got data for server but it's finished - (error "Server not ready for SASL data: %s" data) + (sieve-manage--error + "Server not ready for SASL data: %s" data) ;; The authentication process is finished. + (sieve-manage--message "Logged in as %s using %s" + user-name mech) (throw 'done t))) (unless (stringp rsp) - (error "Server aborted SASL authentication: %s" (caddr rsp))) + (sieve-manage--error + "Server aborted SASL authentication: %s" (caddr rsp))) (sasl-step-set-data step (base64-decode-string rsp)) (setq step (sasl-next-step client step)) (sieve-manage-send @@ -288,8 +336,7 @@ sieve-sasl-auth (base64-encode-string (sasl-step-data step) 'no-line-break) "\"") - "")))) - (message "sieve: Login using %s...done" mech)))) + ""))))))) (defun sieve-manage-cram-md5-p (buffer) (sieve-manage-capability "SASL" "CRAM-MD5" buffer)) @@ -353,7 +400,7 @@ sieve-manage-open sieve-manage-default-stream) sieve-manage-auth (or auth sieve-manage-auth)) - (message "sieve: Connecting to %s..." sieve-manage-server) + (sieve-manage--message "Connecting to %s..." sieve-manage-server) (sieve-manage-open-server sieve-manage-server sieve-manage-port sieve-manage-stream @@ -368,7 +415,8 @@ sieve-manage-open (setq sieve-manage-auth auth) (cl-return))) (unless sieve-manage-auth - (error "Couldn't figure out authenticator for server"))) + (sieve-manage--error + "Couldn't figure out authenticator for server"))) (sieve-manage-erase) (current-buffer)))) @@ -433,11 +481,7 @@ sieve-manage-havespace (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name - ;; Here we assume that the coding-system will - ;; replace each char with a single byte. - ;; This is always the case if `content' is - ;; a unibyte string. - (length content) + (length (sieve-manage-encode content)) sieve-manage-client-eol content)) (sieve-manage-parse-okno))) @@ -449,11 +493,10 @@ sieve-manage-deletescript (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "GETSCRIPT \"%s\"" name)) - (let ((script (sieve-manage-parse-string))) - (sieve-manage-parse-crlf) - (with-current-buffer output-buffer - (insert script)) - (sieve-manage-parse-okno)))) + (sieve-manage-decode (sieve-manage-parse-string) + output-buffer) + (sieve-manage-parse-crlf) + (sieve-manage-parse-okno))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -478,6 +521,9 @@ sieve-manage-drop-next-answer (defun sieve-manage-ok-p (rsp) (string= (downcase (or (car-safe rsp) "")) "ok")) +(defun sieve-manage-no-p (rsp) + (string= (downcase (or (car-safe rsp) "")) "no")) + (defun sieve-manage-is-okno () (when (looking-at (concat "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" @@ -528,7 +574,11 @@ sieve-manage-parse-string (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-string))) + (unless (setq rsp (sieve-manage-is-string)) + (when (sieve-manage-no-p (sieve-manage-is-okno)) + ;; simple `error' is enough since `sieve-manage-erase' + ;; already adds the server response to the log + (error (sieve-manage-erase))))) (sieve-manage-erase (point)) rsp)) @@ -540,7 +590,8 @@ sieve-manage-parse-listscripts (let (tmp rsp data) (while (null rsp) (while (null (or (setq rsp (sieve-manage-is-okno)) - (setq tmp (sieve-manage-is-string)))) + (setq tmp (sieve-manage-decode + (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min))) (when tmp @@ -559,13 +610,9 @@ sieve-manage-parse-listscripts rsp))) (defun sieve-manage-send (cmdstr) - (setq cmdstr (concat cmdstr sieve-manage-client-eol)) - (and sieve-manage-log - (with-current-buffer (get-buffer-create sieve-manage-log) - (mm-enable-multibyte) - (buffer-disable-undo) - (goto-char (point-max)) - (insert cmdstr))) + (setq cmdstr (sieve-manage-encode + (concat cmdstr sieve-manage-client-eol))) + (sieve-manage--append-to-log cmdstr) (process-send-string sieve-manage-process cmdstr)) (provide 'sieve-manage) -- 2.34.1 --=-=-= Content-Type: text/plain Both, the (internal) process/protocol buffer and the log buffer are now unibyte. The conversion to multibyte UTF-8 is only done for user visible (UI) buffers. To properly handle the protocol line termination (CRLF), I added `:coding 'raw-text-unix` (with explicit unix EOL convention) to the `open-network-stream' call (also in the new `manage-sieve-encode' function. This was needed to allow keep the various (looking-at "...\r\n" ...) calls working. This is something which still feels a bit weird, but I haven't found another way to do it. I did some tests with (setq-default buffer-file-coding-system 'utf-8-unix/'utf-8-dos) which did not show any issues. I would also add some ERT tests, probably in a separate commit? In addition, I found that `sieve-manage-quit' in sieve.el had the tendency to kill unrelated buffers in case of errors during earlier steps. For this, I created a sepate patch: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0002-Improve-robustnes-of-sieve-manage-quit-in-case-of-er.patch Content-Description: avoid killing unrelated buffers >From 559ce20b4c9b75f67bef3a1e23b7501bdeaa98d2 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:33:56 +0100 Subject: [PATCH 2/2] Improve robustnes of `sieve-manage-quit' in case of errors * lisp/net/sieve.el (sieve-manage-quit): Avoid killing buffers it's not supposed to touch. --- lisp/net/sieve.el | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index 630ea04070..5680526389 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -154,7 +154,8 @@ sieve-manage-quit (interactive) (sieve-manage-close sieve-manage-buffer) (kill-buffer sieve-manage-buffer) - (kill-buffer (current-buffer))) + (when-let ((buffer (get-buffer sieve-buffer))) + (kill-buffer buffer))) (defun sieve-bury-buffer () "Bury the Manage Sieve buffer without closing the connection." -- 2.34.1 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 06 07:34:24 2022 Received: (at 54154) by debbugs.gnu.org; 6 Sep 2022 11:34:24 +0000 Received: from localhost ([127.0.0.1]:49888 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oVWqW-0002zc-8l for submit@debbugs.gnu.org; Tue, 06 Sep 2022 07:34:24 -0400 Received: from quimby.gnus.org ([95.216.78.240]:56434) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oVWqU-0002zO-Jy for 54154@debbugs.gnu.org; Tue, 06 Sep 2022 07:34:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Content-Type:MIME-Version:Message-ID:Date:References: In-Reply-To:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=yed/b2KpN4WCxnXSsEdTizL5yDtnB7Il0BZD7BezlIE=; b=Ifrtg6MXku9w32TulTioKzAnX3 32mGltr3b7VEormcDeq+88DkAROxCT5w+dtJHOw1lwGBgPkFNyPsY0yN1qZywtKb3IZ098z2GX/dn gByKgbBSKZ6Z3gPTrLC9A1NoK8t6Iyy6YW49aogK05Mq/oWS8CK9Qhs146ZNPomAss/8=; Received: from [84.212.220.105] (helo=joga) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oVWqM-00048b-63; Tue, 06 Sep 2022 13:34:16 +0200 From: Lars Ingebrigtsen To: Kai Tetzlaff Subject: Re: bug#54154: 29.0.50; [PATCH] `sieve-manage-getscript' fails if script contains multibyte characters In-Reply-To: <87sfs3w4zl.fsf@tetzco.de> (Kai Tetzlaff's message of "Mon, 28 Feb 2022 13:27:42 +0100") References: <87wnhj5nbk.fsf@tetzco.de> <878rtzxhnc.fsf@gnus.org> <875yp3w0pr.fsf@gnus.org> <87ilt33pi3.fsf@tetzco.de> <877d9hvf8d.fsf@gnus.org> <87sfs3w4zl.fsf@tetzco.de> X-Now-Playing: Yes's _Tormato_: "Don't Kill The Whale" Date: Tue, 06 Sep 2022 13:34:13 +0200 Message-ID: <87h71kspmy.fsf@gnus.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: Kai Tetzlaff writes: > Ok, I'm attaching two patches which fix all issues I noticed. Sorry; looks like I forgot about this. Now finally pushed to Emacs 29. Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54154 Cc: 54154@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Kai Tetzlaff writes: > Ok, I'm attaching two patches which fix all issues I noticed. Sorry; looks like I forgot about this. Now finally pushed to Emacs 29. From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 06 07:34:29 2022 Received: (at control) by debbugs.gnu.org; 6 Sep 2022 11:34:30 +0000 Received: from localhost ([127.0.0.1]:49891 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oVWqb-0002zx-Gu for submit@debbugs.gnu.org; Tue, 06 Sep 2022 07:34:29 -0400 Received: from quimby.gnus.org ([95.216.78.240]:56450) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oVWqZ-0002zU-EG for control@debbugs.gnu.org; Tue, 06 Sep 2022 07:34:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org; s=20200322; h=Subject:From:To:Message-Id:Date:Sender:Reply-To:Cc: MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=HgVVDzUuMmF5tbBXltHbzPJKYr6WLu/+FEqWA8FQmbE=; b=qQsIBX/CyXQOFNkTbiPHL6VbMj Enui2TjwtjamrIq2rHNgQrlt7uc9A+rWiUEw5GeDAY+Q9p19QYV4lE1+KF/2Jpfjs1SdnDZ81CKhY k87dYLOg5JI/K5K4Z8RqckU7bi6z240PyEuOzasZdI8CHt+nvzlKcVxcSaIspdC+Q5qM=; Received: from [84.212.220.105] (helo=joga) by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oVWqR-00048i-M7 for control@debbugs.gnu.org; Tue, 06 Sep 2022 13:34:21 +0200 Date: Tue, 06 Sep 2022 13:34:19 +0200 Message-Id: <87fsh4spms.fsf@gnus.org> To: control@debbugs.gnu.org From: Lars Ingebrigtsen Subject: control message for bug #54154 X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see @@CONTACT_ADDRESS@@ for details. Content preview: close 54154 29.1 quit Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) close 54154 29.1 quit From unknown Thu Aug 21 12:12:19 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Wed, 05 Oct 2022 11:24:07 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator From debbugs-submit-bounces@debbugs.gnu.org Wed Jan 18 10:08:03 2023 Received: (at control) by debbugs.gnu.org; 18 Jan 2023 15:08:03 +0000 Received: from localhost ([127.0.0.1]:41558 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIA2k-0005kz-H3 for submit@debbugs.gnu.org; Wed, 18 Jan 2023 10:08:03 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:46865) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pI7jt-0001YD-Ov for control@debbugs.gnu.org; Wed, 18 Jan 2023 07:40:26 -0500 Received: from mail.tetzco.de (ipb21a9472.dynamic.kabel-deutschland.de [178.26.148.114]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id A51DDBD078 for ; Wed, 18 Jan 2023 13:40:16 +0100 (CET) Received: from [IPV6:2a02:810d:1380:7900:e5ed:cb07:dbae:7249] (unknown [IPv6:2a02:810d:1380:7900:e5ed:cb07:dbae:7249]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id 089506C00B2 for ; Wed, 18 Jan 2023 13:59:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674046799; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZfO4lLJuFcCQabs74O0CrrtFcn9oc2tDZeiyAYWZ94k=; b=ZaMB+Zv4L45oV9lx9DozA/4il4zDIgVSmKn3h0aMbXkHaE16WombiZvJx5TX636GVZ1fYa 8sq4++K5b5sZllSL1h3YHUCNYnue0TIKOpia5u3bCZES8Mfag8zgqvCPZ0iBQCG77mgnRa FVlhdh51ED/I3tvRzS3y4cASeHKY9hPHbo+WuCvs0saYVWKJNziGJdWd6qkGqjrh1K5mCf ZxLENKTn4zQVjuNUOZyDc+YdPL1/whuPms/zvxeqMu3xlQHy6dkhAyUifND2ehTH4QNS4c AnR5SSDicK8apvD7Qr1I0RbNLSkJ0WmI8FOWMl/7sI0q2+LNXJ9JCTdTqRTskqoS2erBUU 1eECshS1SLVH8WVei4PbXjlx9A+Y8WezP4AqWZJGLMAAZRUYzPI+HHg267zFylOZdMFDn5 PFg4DrHt3HbXXn6yy/mzL4GNni3YgtjvpIGwle3rXAKjjLmvkw0Mfd9cYOcMouBjtWVZkq GDlTfhN2wIwkTa2yOYPKj9eQBYFiAkGKF0RSjaXd/DVpUItf1vagCchtwS9d/aDZLyFjOy 1cdwlT+JomQUZoxoxAGyihpwY/72wmGsthcLrd7iXbBKmwP6Fq3kd6CDhTt82+5mXWRY6w vwWfapc+b0IfxTYdYqh/ekMmc7e+rbTKd3K+9cFJJdwSAzeTW1Uc8= Message-ID: <1f0d3f5c-2d2d-4170-4603-0f0e3da28dc9@tetzco.de> Date: Wed, 18 Jan 2023 13:40:03 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.6.1 To: control@debbugs.gnu.org Content-Language: en-US From: Kai Tetzlaff Subject: unarchive 54154 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: 089506C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [2.88 / 30.00]; NEURAL_SPAM(2.98)[0.994]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ZERO(0.00)[0]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; TAGGED_FROM(0.00)[bug]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: control X-Mailman-Approved-At: Wed, 18 Jan 2023 10:08:00 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) unarchive 54154 From debbugs-submit-bounces@debbugs.gnu.org Wed Jan 18 13:34:00 2023 Received: (at 54154) by debbugs.gnu.org; 18 Jan 2023 18:34:00 +0000 Received: from localhost ([127.0.0.1]:41809 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIDG3-0000KW-Ti for submit@debbugs.gnu.org; Wed, 18 Jan 2023 13:34:00 -0500 Received: from mail.bsd4all.net ([94.130.200.20]:32816) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIDB0-0006HO-Iw for 54154@debbugs.gnu.org; Wed, 18 Jan 2023 13:28:48 -0500 Date: Wed, 18 Jan 2023 19:28:40 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1674066520; bh=zn9wHdAswIoE2lKbjpYMOA505YhlJDBqSjq4qraTqU0=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type; b=Rz8dcGnPG7GCdpgS/NBGu33z/x0ZJRUF4HFm5DzLq+FCLPCy0pZbKj2Otk8f3FppC HjVfQMdRPaSPXaIn5hWiuc4Hfjgnyca95OD6J5aItvVBMfhw6Fdaxv79dgo4ub1oj4 qmxfahpczJC78Ss3lVaqAbn0qcI0aY7vWShooiFb09QUbe9M/t6/YIRFNl85daWNAg wQWIlt+RM1485BofZgKlZ/8kMglWlqymLQZcYhehWVyCAs7tldB8Gb8Y6UQKWuquTQ z5bii7Mh9CoSwYNg6TnQSPLZbdHQXarfpDW2SmtMrJ2hvUCd0CokchSOYEYeDRROH8 LXWrqbLustl0A== From: "Herbert J. Skuhra" To: Kai Tetzlaff Subject: Re: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="IBPvcxdEInI5LRK/" Content-Disposition: inline In-Reply-To: <87bkmwi0ut.fsf@tetzco.de> <87fsc8i2c5.fsf@tetzco.de> <87bkmw8b02.fsf@tetzco.de> X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 X-Mailman-Approved-At: Wed, 18 Jan 2023 13:33:58 -0500 Cc: larsi , 54154@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --IBPvcxdEInI5LRK/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Jan 18, 2023 at 10:09:33AM +0100, Kai Tetzlaff wrote: > Hello Herbert, > > it seems that I'm responsible for this issue. Unfortunately, I cannot > reproduce it with my imap/sieve server setup. However, if you're willing > to provide some additional info, we should hopefully be able to find the > bug. Hello, this is strange because I can reproduce it easily on different systems: - master on FreeBSD 13.1-STABLE - emacs-29 and master on macOS 12.6.2 - master on WLS2/Windows11 (Ubuntu) > "Herbert J. Skuhra" writes: > > I think commit ae963e80a79f5a9184daabfc8197f211a39b136d is causing the > > following issue: > > > > 1. build master or emacs-29 > > 2. run emacs -Q > > 3. M-x sieve-manage and enter imap server. > In my case, after entering the server address, I do get prompted for a > username followed by a password prompt. > > > The following error is displayed: > > sieve-manage: Connecting to ... > > sieve-manage--message: Wrong type argument: stringp, t > > 4. Repeat step 3 and this time sieve-manage will connect and prompt for username/password > As I wrote above, I already get these prompts after 3. So somehow, my > setup is different from the one you're using. What imap server are you using? > Is the connection to the server using SSL/TLS (in my case it is)? I use imap.mailbox.org and the connection is encrypted (using STARTTLS). But tcpdump doesn't capture any packets when I run sieve-manage for the first time. > Could you re-run the steps above with the following additional steps > before 3.: > > 2a) M-x find-library sieve-manage > 2b) M-x eval-buffer > 2c) M-x find-library sieve > 2d) M-x eval-buffer > 2e) M-x toggle-debug-on-error > > to get a full backtrace and send it to me? Backtrace attached. On Wed, Jan 18, 2023 at 10:09:33AM +0100, Kai Tetzlaff wrote: > Hello Herbert, > > a small update: Please also send the content of the '*sieve-manage-log*' > buffer. There is no *sieve-manage-log* buffer after running sieve-manage for the first time. > (I also added a - hopefully working - email address for Lars.) Sorry, copy&paste error. :-( Thanks. -- Herbert --IBPvcxdEInI5LRK/ Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="sieve-manage.txt" Debugger entered--Lisp error: (wrong-type-argument stringp t) set-buffer(t) (save-current-buffer (set-buffer (or (get-buffer sieve-manage-log) (save-current-buffer (set-buffer (get-buffer-create sieve-manage-log)) (set-buffer-multibyte nil) (buffer-disable-undo)))) (goto-char (point-max)) (apply #'insert args)) (progn (save-current-buffer (set-buffer (or (get-buffer sieve-manage-log) (save-current-buffer (set-buffer (get-buffer-create sieve-manage-log)) (set-buffer-multibyte nil) (buffer-disable-undo)))) (goto-char (point-max)) (apply #'insert args))) (if sieve-manage-log (progn (save-current-buffer (set-buffer (or (get-buffer sieve-manage-log) (save-current-buffer (set-buffer (get-buffer-create sieve-manage-log)) (set-buffer-multibyte nil) (buffer-disable-undo)))) (goto-char (point-max)) (apply #'insert args)))) sieve-manage--append-to-log("sieve-manage: Connecting to imap.mailbox.org..." "\n") (let ((ret (apply #'message (concat "sieve-manage: " format-string) args))) (sieve-manage--append-to-log ret "\n") ret) sieve-manage--message("Connecting to %s..." "imap.mailbox.org") (save-current-buffer (set-buffer (or buffer (sieve-manage-make-process-buffer))) (progn (setq sieve-manage-server (or server sieve-manage-server)) (setq sieve-manage-stream (or stream sieve-manage-stream sieve-manage-default-stream)) (setq sieve-manage-auth (or auth sieve-manage-auth))) (sieve-manage--message "Connecting to %s..." sieve-manage-server) (sieve-manage-open-server sieve-manage-server sieve-manage-port sieve-manage-stream (current-buffer)) (if (sieve-manage-opened (current-buffer)) (progn (if (and (null sieve-manage-auth) (not (eq sieve-manage-state 'auth))) (progn (catch '--cl-block-nil-- (let (...) (while tail ...))) (if sieve-manage-auth nil (sieve-manage--error "Couldn't figure out authenticator for server")))) (sieve-manage-erase) (current-buffer)))) sieve-manage-open("imap.mailbox.org" nil) (set (make-local-variable 'sieve-manage-buffer) (sieve-manage-open server port)) (or (set (make-local-variable 'sieve-manage-buffer) (sieve-manage-open server port)) (error "Error opening server %s" server)) (set-buffer (or (set (make-local-variable 'sieve-manage-buffer) (sieve-manage-open server port)) (error "Error opening server %s" server))) (save-current-buffer (set-buffer (or (set (make-local-variable 'sieve-manage-buffer) (sieve-manage-open server port)) (error "Error opening server %s" server))) (sieve-manage-authenticate)) sieve-open-server("imap.mailbox.org" nil) (if (sieve-open-server server port) (sieve-refresh-scriptlist) (message "Could not open server %s" server)) sieve-manage("imap.mailbox.org") funcall-interactively(sieve-manage "imap.mailbox.org") call-interactively(sieve-manage record nil) command-execute(sieve-manage record) execute-extended-command(nil "sieve-manage" "sieve-man") funcall-interactively(execute-extended-command nil "sieve-manage" "sieve-man") call-interactively(execute-extended-command nil nil) command-execute(execute-extended-command) --IBPvcxdEInI5LRK/-- From debbugs-submit-bounces@debbugs.gnu.org Wed Jan 18 14:17:55 2023 Received: (at 54154) by debbugs.gnu.org; 18 Jan 2023 19:17:55 +0000 Received: from localhost ([127.0.0.1]:41874 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIDwZ-0001fJ-0J for submit@debbugs.gnu.org; Wed, 18 Jan 2023 14:17:55 -0500 Received: from eggs.gnu.org ([209.51.188.92]:52488) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIDwW-0001f0-D1 for 54154@debbugs.gnu.org; Wed, 18 Jan 2023 14:17:53 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIDwP-000403-Ke; Wed, 18 Jan 2023 14:17:45 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=Rq2j+O+4WDEGfvl3NG0p/dC/P0kLcw3ia0e7voJbALE=; b=a1unuxJ3vsqP XwYMoKcmHLkj+rx4hJHl48WCHfGAuIFQbLluzQYViV8I5v+KEMXZ3G4cqrmxVCbku00MXN7e8pBO8 DGM+Bxm5xyg0qqn2fLvYz+V3I8Dp+W3XGBT5k1Z4N/E9jeANT4hPweXii5qf7tF+3p452our0HIjr jy9cqaBJZbL2jd9oCzLeBajfP53iO8J1b7oDwQJxx6f1X1obGk3RSWcH6HbWocCXNaxaZGLSrs61U iMBJ5QYG+v5xKgOXUuTz83OB02sHBglQ5t7qJ2DUXFEUxNDQHsBF9m8Fe1rg/CbabGKNa3OnqPbOH aS/RMAdMsHs4xCOEBSZ+8w==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIDwP-0006WP-19; Wed, 18 Jan 2023 14:17:45 -0500 Date: Wed, 18 Jan 2023 21:17:59 +0200 Message-Id: <835yd31wk8.fsf@gnu.org> From: Eli Zaretskii To: "Herbert J. Skuhra" In-Reply-To: (herbert@gojira.at) Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) References: <87wnhj5nbk.fsf@tetzco.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54154 Cc: larsi@gnus.org, 54154@debbugs.gnu.org, emacs+bug@tetzco.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Cc: larsi , 54154@debbugs.gnu.org > Date: Wed, 18 Jan 2023 19:28:40 +0100 > From: "Herbert J. Skuhra" > > On Wed, Jan 18, 2023 at 10:09:33AM +0100, Kai Tetzlaff wrote: > > Hello Herbert, > > > > it seems that I'm responsible for this issue. Unfortunately, I cannot > > reproduce it with my imap/sieve server setup. However, if you're willing > > to provide some additional info, we should hopefully be able to find the > > bug. > > Hello, > > this is strange because I can reproduce it easily on different systems: > > - master on FreeBSD 13.1-STABLE > - emacs-29 and master on macOS 12.6.2 > - master on WLS2/Windows11 (Ubuntu) Is this problem still relevant? I thought that Lars closed the bug report back in September? > > Could you re-run the steps above with the following additional steps > > before 3.: > > > > 2a) M-x find-library sieve-manage > > 2b) M-x eval-buffer > > 2c) M-x find-library sieve > > 2d) M-x eval-buffer > > 2e) M-x toggle-debug-on-error > > > > to get a full backtrace and send it to me? > > Backtrace attached. Thanks. The error is here: (defun sieve-manage--append-to-log (&rest args) "Append ARGS to sieve-manage log buffer. ARGS can be a string or a list of strings. The buffer to use for logging is specifified via `sieve-manage-log'. If it is nil, logging is disabled." (when sieve-manage-log (with-current-buffer (or (get-buffer sieve-manage-log) (with-current-buffer <<<<<<<<<<<<<<<<<<<<<< (get-buffer-create sieve-manage-log) (set-buffer-multibyte nil) (buffer-disable-undo))) And I admit that I don't understand this code. What is it trying to do? Shouldn't it be just (when sieve-manage-log (with-current-buffer (get-buffer-create sieve-manage-log) (set-buffer-multibyte nil) (buffer-disable-undo))) Kai, am I missing something? Herbert, if you make the change above, does the problem go away? From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 19 02:45:16 2023 Received: (at 54154) by debbugs.gnu.org; 19 Jan 2023 07:45:16 +0000 Received: from localhost ([127.0.0.1]:42447 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIPbn-0007Gq-MW for submit@debbugs.gnu.org; Thu, 19 Jan 2023 02:45:16 -0500 Received: from eggs.gnu.org ([209.51.188.92]:51694) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIPbm-0007Ga-45 for 54154@debbugs.gnu.org; Thu, 19 Jan 2023 02:45:14 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIPbe-0006os-GF; Thu, 19 Jan 2023 02:45:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=zy3EHALh5+TCCf0ru/gL2OxVoYtLdyY0rI06x6vXfUg=; b=kw9+oPuZ5cf6 40CyLdiJoyyQJUYGMhi6qR7DgQ2Dhj5m+V/HRsOCVc+t3pHCO+IfMWMP+NDm+UsCoyJ25O1wAUThm iBXni4L2JVf8WbZm7yGBVfeZdDPAS1F78W9BQny/K9IIWOzCMPWndL5jMzAiP2fPdOYDyarGJS4LF QUidF0NroDDv6QhwCr9vJIS/YLq5Y6W36+2yHYvjZTekS8V3b5sws1K6+EiikPBMST7hx9ds26S+4 kgAphKvOF87AaA/Xeb3v77Mbf9qwnZojRJrkj6gkKOA86aQkdT4PrD1YJRPcUJXVCM8ZlfvxsU5J6 RaNACmBLiukqSIzRxi7M2Q==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIPbd-0001Mb-Fv; Thu, 19 Jan 2023 02:45:05 -0500 Date: Thu, 19 Jan 2023 09:45:21 +0200 Message-Id: <83tu0nynla.fsf@gnu.org> From: Eli Zaretskii To: Kai Tetzlaff In-Reply-To: <874jsngod2.fsf@tetzco.de> (message from Kai Tetzlaff on Thu, 19 Jan 2023 05:06:01 +0100) Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Kai Tetzlaff > Cc: Eli Zaretskii , larsi@gnus.org, 54154@debbugs.gnu.org > Date: Thu, 19 Jan 2023 05:06:01 +0100 > > >> Kai, am I missing something? > > The additional '(or ...' was meant to only run > > (set-buffer-multibyte nil) > (buffer-disable-undo) > > once, when creating the log buffer (not everytime something gets > appended to the log). What is missing in my code is an additional > `current-buffer'. Here's the complete fixed function: > > (defun sieve-manage--append-to-log (&rest args) > "Append ARGS to sieve-manage log buffer. > > ARGS can be a string or a list of strings. > The buffer to use for logging is specifified via > `sieve-manage-log'. If it is nil, logging is disabled." > (when sieve-manage-log > (with-current-buffer (or (get-buffer sieve-manage-log) > (with-current-buffer > (get-buffer-create sieve-manage-log) > (set-buffer-multibyte nil) > (buffer-disable-undo) > (current-buffer))) > (goto-char (point-max)) > (apply #'insert args)))) Thanks. The duplicate use of with-current-buffer is sub-optimal, IMO. What about the simpler code below: (when sieve-manage-log (let* ((existing-log-buffer (get-buffer sieve-manage-log)) (log-buffer (or existing-log-buffer (get-buffer-create sieve-manage-log)))) (with-current-buffer log-buffer (unless existing-log-buffer ;; Do this only once, when creating the log buffer. (set-buffer-multibyte nil) (buffer-disable-undo)) (goto-char (point-max)) (apply #'insert args))))) > ;; Internal utility functions > +(defun sieve-manage--set-internal-buffer-properties (buffer) > + "Set BUFFER properties for internally used buffers. > + > +Used for process and log buffers, this function makes sure that > +those buffers keep received and sent data intact by: > +- setting the coding system to 'sieve-manage--coding-system', > +- setting `after-change-functions' to nil to avoid those > + functions messing with buffer content. > +Also disables undo (to save a bit of memory and improve > +performance). > + > +Returns BUFFER." > + (with-current-buffer buffer > + (set-buffer-file-coding-system sieve-manage--coding-system) > + (setq-local after-change-functions nil) > + (buffer-disable-undo) > + (current-buffer))) > + > (defun sieve-manage--append-to-log (&rest args) > "Append ARGS to sieve-manage log buffer. > > @@ -175,10 +202,8 @@ sieve-manage--append-to-log > `sieve-manage-log'. If it is nil, logging is disabled." > (when sieve-manage-log > (with-current-buffer (or (get-buffer sieve-manage-log) > - (with-current-buffer > - (get-buffer-create sieve-manage-log) > - (set-buffer-multibyte nil) > - (buffer-disable-undo))) > + (sieve-manage--set-internal-buffer-properties > + (get-buffer-create sieve-manage-log))) > (goto-char (point-max)) > (apply #'insert args)))) This still uses a less-than-elegant implementation that calls with-current-buffer twice. > (defun sieve-manage-encode (utf8-string) > "Convert UTF8-STRING to managesieve protocol octets." > - (encode-coding-string utf8-string 'raw-text t)) > + (encode-coding-string utf8-string sieve-manage--coding-system t)) Why is the argument called utf8-string? If it's indeed a string encoded in UTF-8, why do you need to encode it again with raw-text-unix? it should be a no-op in that case. So please tell more about the underlying issue. > @@ -244,8 +267,7 @@ sieve-manage-open-server > (open-network-stream > "SIEVE" buffer server port > :type stream > - ;; eol type unix is required to preserve "\r\n" > - :coding 'raw-text-unix > + :coding `(binary . ,sieve-manage--coding-system) Same question about encoding with raw-text-unix here: using it means some other code will need to encode the text with a real encoding, which in this case is UTF-8 (AFAIU the managesieve protocol RFC). So why not use utf-8-unix here? Should the addition of BYE support be mentioned in NEWS? On balance, I think the additional patches should go to master, indeed. But let's resolve the issues mentioned above first. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 19 03:21:01 2023 Received: (at 54154) by debbugs.gnu.org; 19 Jan 2023 08:21:02 +0000 Received: from localhost ([127.0.0.1]:42460 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIQAI-0008A0-LY for submit@debbugs.gnu.org; Thu, 19 Jan 2023 03:21:01 -0500 Received: from mail.bsd4all.net ([94.130.200.20]:51497) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIHla-0006ai-FK for 54154@debbugs.gnu.org; Wed, 18 Jan 2023 18:22:51 -0500 Date: Thu, 19 Jan 2023 00:22:44 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1674084164; bh=5PSpFVBWxTEUkGESCf78zLVQK/rsrM2eUb0WoMZdjFY=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type; b=rhj8QwbDGcJLymmTsR8I88Lh/KnOnY8tD44lFJ3erw2byBA0gs2npUFocKSr+6yv1 gHXSjNVDG99koFre8qOkUVXfvk68WN7Y+FSi//rIEj3wiCKq0Aes4qcTOmi2/cP+1L aoabGpva0YfrtNF/lJ9NJApX761O9nCegH2ZmVH87WHYQwhaSxl9r6o2E/cSq03kpx +mS/Gbs2ehCPkivTapdEhFXllLM5DPRFENRKpkUxDkwKaDETe80V8vhoDPcf3No+Ub giy9cZGS31fegXbAIDStQfTWy0sd0FnYEjyU6YE5VMzuup8BaTIOtzoGIZEcpEZ6PJ WyrsOcA+4Obpw== From: "Herbert J. Skuhra" To: Eli Zaretskii Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) Message-ID: References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <835yd31wk8.fsf@gnu.org> X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 X-Mailman-Approved-At: Thu, 19 Jan 2023 03:20:53 -0500 Cc: larsi@gnus.org, 54154@debbugs.gnu.org, emacs+bug@tetzco.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Wed, Jan 18, 2023 at 09:17:59PM +0200, Eli Zaretskii wrote: > > Cc: larsi , 54154@debbugs.gnu.org > > Date: Wed, 18 Jan 2023 19:28:40 +0100 > > From: "Herbert J. Skuhra" > > > > On Wed, Jan 18, 2023 at 10:09:33AM +0100, Kai Tetzlaff wrote: > > > Hello Herbert, > > > > > > it seems that I'm responsible for this issue. Unfortunately, I cannot > > > reproduce it with my imap/sieve server setup. However, if you're willing > > > to provide some additional info, we should hopefully be able to find the > > > bug. > > > > Hello, > > > > this is strange because I can reproduce it easily on different systems: > > > > - master on FreeBSD 13.1-STABLE > > - emacs-29 and master on macOS 12.6.2 > > - master on WLS2/Windows11 (Ubuntu) > > Is this problem still relevant? I thought that Lars closed the bug > report back in September? > > > > Could you re-run the steps above with the following additional steps > > > before 3.: > > > > > > 2a) M-x find-library sieve-manage > > > 2b) M-x eval-buffer > > > 2c) M-x find-library sieve > > > 2d) M-x eval-buffer > > > 2e) M-x toggle-debug-on-error > > > > > > to get a full backtrace and send it to me? > > > > Backtrace attached. > > Thanks. The error is here: > > (defun sieve-manage--append-to-log (&rest args) > "Append ARGS to sieve-manage log buffer. > > ARGS can be a string or a list of strings. > The buffer to use for logging is specifified via > `sieve-manage-log'. If it is nil, logging is disabled." > (when sieve-manage-log > (with-current-buffer (or (get-buffer sieve-manage-log) > (with-current-buffer <<<<<<<<<<<<<<<<<<<<<< > (get-buffer-create sieve-manage-log) > (set-buffer-multibyte nil) > (buffer-disable-undo))) > > And I admit that I don't understand this code. What is it trying to > do? Shouldn't it be just > > (when sieve-manage-log > (with-current-buffer (get-buffer-create sieve-manage-log) > (set-buffer-multibyte nil) > (buffer-disable-undo))) > > Kai, am I missing something? > > Herbert, if you make the change above, does the problem go away? Yes, this change resolves the issue: diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4a..636c7cbc5b 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -174,11 +174,9 @@ sieve-manage--append-to-log The buffer to use for logging is specifified via `sieve-manage-log'. If it is nil, logging is disabled." (when sieve-manage-log - (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) + (with-current-buffer (get-buffer-create sieve-manage-log) (set-buffer-multibyte nil) - (buffer-disable-undo))) + (buffer-disable-undo) (goto-char (point-max)) (apply #'insert args)))) Thanks. -- Herbert From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 19 03:21:24 2023 Received: (at 54154) by debbugs.gnu.org; 19 Jan 2023 08:21:24 +0000 Received: from localhost ([127.0.0.1]:42466 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIQAc-0008B5-Cz for submit@debbugs.gnu.org; Thu, 19 Jan 2023 03:21:23 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:51513) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIMBr-0001dN-EB for 54154@debbugs.gnu.org; Wed, 18 Jan 2023 23:06:17 -0500 Received: from mail.tetzco.de (unknown [IPv6:2a02:810d:1380:7900::9]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id CA03FBD078; Thu, 19 Jan 2023 05:06:08 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id 013606C00B2; Thu, 19 Jan 2023 05:25:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674102355; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kZd/z7Xr2ffc7o9xHWr862cnFjj2RxpCayAtqpgOAtw=; b=vHmULXqddN2OUvsYY1Tae4YACGCE4oMIvaVcjHYzGoubO4KkGcWR8Xal5JQQWhC2f+6m7P RdXKXcN4Wa7xHOgPxWhSKLn39ien+nyBDIj+mKqp4rgDCygCDitUoP9QfIKz19Sig7ZztE O1gjgDCfJC9Htv/R+Bug7ZiakgE1ITPHVxmqc1/DrtLUJNaGK6xNsvp+yGobp9BAW9B8H0 fqhCM5P2Jymcw+WDairghhjPk/cfhAcno0rTH4kO6+VmtrkcxU6N9JmcLCIX5WAKE/r8C4 +lRPEe+i3FZwDPvRS3CaIf68GZ0QvbGctB++2dEDPbvHQM5N+Pj31HO2tUtglyGKQvlT0Y zkKIR6+4WwXs/CEk/6weWvl0G2/rEsLDHaO/PeHBCDxL7erOB6giTRkLrWCXSNteOYAhUL BzdW1nIgoICpI4o/3cd1RoyItTu9LezaGG7DAmKGndkUaNCHu1x6RUiFQL9Lv+x0z6GRXM slOq94tS7Cd6y/K7cNgYCc3ZXDT52W+6bMhKrjZXgifQWtkDMEg4njv8Hkt4k/RFGZSQTX UTPnenbCdGlIPMOYLLmxL8hJ7P/RV1UJvR4iHrAdT8ee1DtQkMrb2S2HvZL45+Q03uDheG oJtubxAT2kTnnr89LxKaLw/RzoyN05z4sEkKRgRaxQBUMzvY4Budk= From: Kai Tetzlaff To: "Herbert J. Skuhra" Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: (Herbert J. Skuhra's message of "Thu, 19 Jan 2023 00:22:44 +0100") References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> Date: Thu, 19 Jan 2023 05:06:01 +0100 Message-ID: <874jsngod2.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: 013606C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.08 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-2.98)[-0.992]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:+,6:+,7:+]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; TAGGED_FROM(0.00)[bug]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; HAS_ATTACHMENT(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 X-Mailman-Approved-At: Thu, 19 Jan 2023 03:20:53 -0500 Cc: Eli Zaretskii , 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain "Herbert J. Skuhra" writes: > On Wed, Jan 18, 2023 at 09:17:59PM +0200, Eli Zaretskii wrote: >> > this is strange because I can reproduce it easily on different systems: >> > >> > - master on FreeBSD 13.1-STABLE >> > - emacs-29 and master on macOS 12.6.2 >> > - master on WLS2/Windows11 (Ubuntu) I can now reproduce the error, too. The problem was that at the time Lars closed the bug report by applying the attached patch (after quite a long time of it sitting dormant), I had some additional local changes for sieve.el and sieve-manage.el on a branch which I didn't get to submit. And when I tried to reproduce the error, I've still been using this branch without realizing it. Sorry for that. >> ... >> Thanks. The error is here: >> >> (defun sieve-manage--append-to-log (&rest args) >> "Append ARGS to sieve-manage log buffer. >> >> ARGS can be a string or a list of strings. >> The buffer to use for logging is specifified via >> `sieve-manage-log'. If it is nil, logging is disabled." >> (when sieve-manage-log >> (with-current-buffer (or (get-buffer sieve-manage-log) >> (with-current-buffer <<<<<<<<<<<<<<<<<<<<<< >> (get-buffer-create sieve-manage-log) >> (set-buffer-multibyte nil) >> (buffer-disable-undo))) >> >> And I admit that I don't understand this code. What is it trying to >> do? Shouldn't it be just >> >> (when sieve-manage-log >> (with-current-buffer (get-buffer-create sieve-manage-log) >> (set-buffer-multibyte nil) >> (buffer-disable-undo))) >> >> Kai, am I missing something? The additional '(or ...' was meant to only run (set-buffer-multibyte nil) (buffer-disable-undo) once, when creating the log buffer (not everytime something gets appended to the log). What is missing in my code is an additional `current-buffer'. Here's the complete fixed function: (defun sieve-manage--append-to-log (&rest args) "Append ARGS to sieve-manage log buffer. ARGS can be a string or a list of strings. The buffer to use for logging is specifified via `sieve-manage-log'. If it is nil, logging is disabled." (when sieve-manage-log (with-current-buffer (or (get-buffer sieve-manage-log) (with-current-buffer (get-buffer-create sieve-manage-log) (set-buffer-multibyte nil) (buffer-disable-undo) (current-buffer))) (goto-char (point-max)) (apply #'insert args)))) >> Herbert, if you make the change above, does the problem go away? > > Yes, this change resolves the issue: > > diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el > index 5bee4f4c4a..636c7cbc5b 100644 > --- a/lisp/net/sieve-manage.el > +++ b/lisp/net/sieve-manage.el > @@ -174,11 +174,9 @@ sieve-manage--append-to-log > The buffer to use for logging is specifified via > `sieve-manage-log'. If it is nil, logging is disabled." > (when sieve-manage-log > - (with-current-buffer (or (get-buffer sieve-manage-log) > - (with-current-buffer > - (get-buffer-create sieve-manage-log) > + (with-current-buffer (get-buffer-create sieve-manage-log) > (set-buffer-multibyte nil) > - (buffer-disable-undo))) > + (buffer-disable-undo) > (goto-char (point-max)) > (apply #'insert args)))) > Here's a patch which preserves the logic of the original code: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-Fix-bug-in-sieve-manage-append-to-log.patch Content-Description: fix sieve-manage--append-to-log for Emacs 29 >From 4198e776da13b603c56acbae0ae89cd9d31cf207 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:16:14 +0100 Subject: [PATCH] Fix bug in sieve-manage--append-to-log * lisp/net/sieve-manage.el (sieve-manage--append-to-log): Fix log buffer creation --- lisp/net/sieve-manage.el | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..ab22294a272 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -178,7 +178,8 @@ sieve-manage--append-to-log (with-current-buffer (get-buffer-create sieve-manage-log) (set-buffer-multibyte nil) - (buffer-disable-undo))) + (buffer-disable-undo) + (current-buffer))) (goto-char (point-max)) (apply #'insert args)))) -- 2.39.0 --=-=-= Content-Type: text/plain The additional changes I mentioned above solve the problem in a different way by introducing a helper function. The also add some other improvements including a new test for handling multibyte characters in sieve server responses. I'm attaching the additional patches below. They might be too large for the current emacs-29 branch. But maybe they can be applied to master? --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Fix-bug-in-sieve-manage-append-to-log-improve-sieve-.patch Content-Description: fix sieve-manage--append-to-log >From 062ade7fe5cdbae36f4b488761e6367beb6e3a41 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:08:07 +0100 Subject: [PATCH 1/4] Fix bug in sieve-manage--append-to-log, improve sieve-manage buffer config Also update/add some comments/docs. * lisp/net/sieve-manage.el sieve-manage--coding-system: New constant. (sieve-manage--set-internal-buffer-properties): New function. (sieve-manage-open-server): Use `sieve-manage--coding-system'. (sieve-manage--append-to-log): Use `sieve-manage--set-internal-buffer-properties' to fix log buffer creation. (sieve-manage-encode) (sieve-manage-make-process-buffer): Use `sieve-manage--set-internal-buffer-properties'. --- lisp/net/sieve-manage.el | 50 +++++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 14 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..4f0f9b1891b 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -58,7 +58,7 @@ ;; ;; References: ;; -;; draft-martin-managesieve-02.txt, +;; RFC5804, ;; "A Protocol for Remotely Managing Sieve Scripts", ;; by Tim Martin. ;; @@ -145,6 +145,15 @@ sieve-manage-ignore-starttls :type 'boolean) ;; Internal variables: +(defconst sieve-manage--coding-system 'raw-text-unix + "Use 'raw-text-unix coding system for (network) communication. + +Sets the coding system used for the internal (process, log) +buffers and the network stream created to communicate with the +managesieve server. Using 'raw-text encoding enables unibyte +mode and makes sure that sent/received octets (bytes) remain +untouched by the coding system. The explicit use of `-unix` +avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") intact).") (defconst sieve-manage-local-variables '(sieve-manage-server sieve-manage-port @@ -167,6 +176,24 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions +(defun sieve-manage--set-internal-buffer-properties (buffer) + "Set BUFFER properties for internally used buffers. + +Used for process and log buffers, this function makes sure that +those buffers keep received and sent data intact by: +- setting the coding system to 'sieve-manage--coding-system', +- setting `after-change-functions' to nil to avoid those + functions messing with buffer content. +Also disables undo (to save a bit of memory and improve +performance). + +Returns BUFFER." + (with-current-buffer buffer + (set-buffer-file-coding-system sieve-manage--coding-system) + (setq-local after-change-functions nil) + (buffer-disable-undo) + (current-buffer))) + (defun sieve-manage--append-to-log (&rest args) "Append ARGS to sieve-manage log buffer. @@ -175,10 +202,8 @@ sieve-manage--append-to-log `sieve-manage-log'. If it is nil, logging is disabled." (when sieve-manage-log (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) - (set-buffer-multibyte nil) - (buffer-disable-undo))) + (sieve-manage--set-internal-buffer-properties + (get-buffer-create sieve-manage-log))) (goto-char (point-max)) (apply #'insert args)))) @@ -204,7 +229,7 @@ sieve-manage--error (defun sieve-manage-encode (utf8-string) "Convert UTF8-STRING to managesieve protocol octets." - (encode-coding-string utf8-string 'raw-text t)) + (encode-coding-string utf8-string sieve-manage--coding-system t)) (defun sieve-manage-decode (octets &optional buffer) "Convert managesieve protocol OCTETS to utf-8 string. @@ -216,13 +241,11 @@ sieve-manage-decode (defun sieve-manage-make-process-buffer () (with-current-buffer - (generate-new-buffer (format " *sieve %s:%s*" - sieve-manage-server - sieve-manage-port)) + (sieve-manage--set-internal-buffer-properties + (generate-new-buffer (format " *sieve %s:%s*" + sieve-manage-server + sieve-manage-port))) (mapc #'make-local-variable sieve-manage-local-variables) - (set-buffer-multibyte nil) - (setq-local after-change-functions nil) - (buffer-disable-undo) (current-buffer))) (defun sieve-manage-erase (&optional p buffer) @@ -244,8 +267,7 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream - ;; eol type unix is required to preserve "\r\n" - :coding 'raw-text-unix + :coding `(binary . ,sieve-manage--coding-system) :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-Handle-BYE-in-sieve-manage-server-responses.patch Content-Description: handle bye in sieve-manage.el >From e09d82fec3fc86d797396b753f3b8c4411df9d1f Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:52:43 +0100 Subject: [PATCH 2/4] Handle BYE in sieve-manage server responses * lisp/net/sieve-manage.el (sieve-manage-regex-oknobye): New function. (sieve-manage-parse-oknobye): Renamed from `sieve-manage-parse-okno'. (sieve-manage-open-server) (sieve-sasl-auth) (sieve-manage-listscripts) (sieve-manage-putscript) (sieve-manage-deletescript) (sieve-manage-getscript) (sieve-manage-setactive) (sieve-manage-is-okno) (sieve-manage-parse-oknobye) (sieve-manage-parse-parse-string) (sieve-manage-parse-parse-crlf): Use `sieve-manage-regex-oknobye' to handle BYE in addition to OK and NO. --- lisp/net/sieve-manage.el | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 4f0f9b1891b..4e63603bbab 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -269,8 +269,8 @@ sieve-manage-open-server :type stream :coding `(binary . ,sieve-manage--coding-system) :capability-command "CAPABILITY\r\n" - :end-of-command "^\\(OK\\|NO\\).*\n" - :success "^OK.*\n" + :end-of-command "^\\(OK\\|NO\\|BYE\\).*\r\n" + :success "^OK.*\r\n" :return-list t :starttls-function (lambda (capabilities) @@ -329,7 +329,7 @@ sieve-sasl-auth (setq rsp nil) (goto-char (match-end 0)) rsp)) - (setq rsp (sieve-manage-is-okno)))) + (setq rsp (sieve-manage-is-oknobye)))) (accept-process-output sieve-manage-process 1) (goto-char (point-min))) (sieve-manage-erase) @@ -498,19 +498,19 @@ sieve-manage-listscripts (defun sieve-manage-havespace (name size &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name (length (sieve-manage-encode content)) sieve-manage-client-eol content)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "DELETESCRIPT \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -518,17 +518,22 @@ sieve-manage-getscript (sieve-manage-decode (sieve-manage-parse-string) output-buffer) (sieve-manage-parse-crlf) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "SETACTIVE \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) ;; Protocol parsing routines +(defun sieve-manage-regexp-oknobye () + "Return regexp for managesieve 'response-oknobye'." + (concat + "^\\(OK\\|NO\\|BYE\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" + sieve-manage-server-eol)) (defun sieve-manage-wait-for-answer () - (let ((pattern "^\\(OK\\|NO\\).*\n") + (let ((pattern (sieve-manage-regexp-oknobye)) pos) (while (not pos) (setq pos (search-forward-regexp pattern nil t)) @@ -546,10 +551,8 @@ sieve-manage-ok-p (defun sieve-manage-no-p (rsp) (string= (downcase (or (car-safe rsp) "")) "no")) -(defun sieve-manage-is-okno () - (when (looking-at (concat - "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" - sieve-manage-server-eol)) +(defun sieve-manage-is-oknobye () + (when (looking-at (sieve-manage-regexp-oknobye)) (let ((status (match-string 1)) (resp-code (match-string 3)) (response (match-string 5))) @@ -558,12 +561,12 @@ sieve-manage-is-okno (setq response (sieve-manage-is-string))) (list status resp-code response)))) -(defun sieve-manage-parse-okno () +(defun sieve-manage-parse-oknobye () (let (rsp) (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-okno))) + (setq rsp (sieve-manage-is-oknobye))) (sieve-manage-erase) rsp)) @@ -597,7 +600,7 @@ sieve-manage-parse-string (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) (unless (setq rsp (sieve-manage-is-string)) - (when (sieve-manage-no-p (sieve-manage-is-okno)) + (when (sieve-manage-no-p (sieve-manage-is-oknobye)) ;; simple `error' is enough since `sieve-manage-erase' ;; already adds the server response to the log (error (sieve-manage-erase))))) @@ -611,7 +614,7 @@ sieve-manage-parse-crlf (defun sieve-manage-parse-listscripts () (let (tmp rsp data) (while (null rsp) - (while (null (or (setq rsp (sieve-manage-is-okno)) + (while (null (or (setq rsp (sieve-manage-is-oknobye)) (setq tmp (sieve-manage-decode (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) -- 2.39.0 --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0003-Add-test-lisp-net-sieve-manage-tests.el.patch Content-Transfer-Encoding: quoted-printable Content-Description: test for sieve-manage multibyte character handling >From 3d6891878a6701afada4fcdd1d426b310366a31d Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Tue, 22 Mar 2022 20:48:09 +0100 Subject: [PATCH 3/4] Add test/lisp/net/sieve-manage-tests.el * test/lisp/net/sieve-manage-tests.el: (mk-literal-s2c) (mk-rsp-oknobye) (mk-rsp-getscript-ok) (managesieve-getscript): Test helper functions. (ert/managesieve-getscript-multibyte): Test `sieve-manage-getscript' with script names containing multibyte characters. --- test/lisp/net/sieve-manage-tests.el | 105 ++++++++++++++++++++++++++++ 1 file changed, 105 insertions(+) create mode 100644 test/lisp/net/sieve-manage-tests.el diff --git a/test/lisp/net/sieve-manage-tests.el b/test/lisp/net/sieve-mana= ge-tests.el new file mode 100644 index 00000000000..2b381a63ad1 --- /dev/null +++ b/test/lisp/net/sieve-manage-tests.el @@ -0,0 +1,105 @@ +;;; sieve-manage-tests.el --- tests for sieve-manage.el -*- lexical-= binding: t; -*- + +;; Copyright (C) 2022 Free Software Foundation, Inc. + +;; Author: Kai Tetzlaff + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see . + +;;; Commentary: + + +;;; Code: + +(require 'ert) +(require 'sieve-manage) + +(defvar sieve-script-multibyte-unix + "if body :matches \"=C3=A4\" { stop; }\n" + "Simple multibyte sieve script.") + +(defun mk-literal-s2c (string) + "Encode STRING to managesieve 'literal-s2c'." + (let ((data (sieve-manage-encode string))) + (concat (format "{%d}\r\n" (length data)) + data))) + +(defun mk-rsp-oknobye (type &optional resp-code string) + "Encode TYPE, RESP-CODE and STRING to managesieve 'response-oknobye'." + (when (memq type '(OK NO BYE)) + (concat + (mapconcat #'identity + (delq nil + (list (symbol-name type) + (when resp-code + (format "(%s)" resp-code)) + (when string + (format "\"%s\"" + (sieve-manage-encode string))))) + " ") + "\r\n"))) +;; (mk-rsp-oknobye 'OK nil "Getscript completed.") +;; (mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") + +(defun mk-rsp-getscript-ok (script) + "Encode SCRIPT to managesieve 'response-getscript'." + (concat (mk-literal-s2c script) + "\r\n" + (mk-rsp-oknobye 'OK "Getscript completed."))) +;; (mk-rsp-getscript-ok sieve-script-multibyte-unix) + +(defun managesieve-getscript (script) + "Simulate managesieve getscript response to test +`sieve-manage-getscript' function." + (let* ((script-name "test.sieve") + ;; `sieve-manage-server' and `sieve-manage-port' are used in + ;; `sieve-manage-make-process-buffer' + (sieve-manage-server) + (sieve-manage-port "sieve") + (sieve-buffer (sieve-manage-make-process-buffer)) + (output-buffer (generate-new-buffer script-name))) + ;; use cl-letf to mock some functions in call chain of + ;; sieve-manage-getscript + (cl-letf (((symbol-function 'sieve-manage-send) + ;; simulate sieve server response with a single + ;; multibyte character `=C3=A4` + (lambda (_) + (with-current-buffer sieve-buffer + (goto-char (point-min)) + (insert (mk-rsp-getscript-ok script))))) + ((symbol-function 'accept-process-output) + (lambda (&optional _ _ _ _) nil)) + ((symbol-function 'get-buffer-process) (lambda (_) nil))) + ;; extract sieve script from sieve-buffer and put it into + ;; output-buffer + (sieve-manage-getscript script-name output-buffer sieve-buffer) + ;; extract script from output-buffer and return it as a + ;; string + (with-current-buffer output-buffer + (set-buffer-modified-p nil) + (buffer-string))) + ;; cleanup + (kill-buffer sieve-buffer) + (kill-buffer output-buffer) + (when (get-buffer sieve-manage-log) + (kill-buffer sieve-manage-log))))) + +(ert-deftest managesieve-getscript-multibyte () + (should (string=3D sieve-script-multibyte-unix + (managesieve-getscript sieve-script-multibyte-unix)))) + + +;;; sieve-manage-tests.el ends here --=20 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0004-Some-minor-fixes-in-lisp-net-sieve.el.patch Content-Description: minor fixes in lisp/net/sieve.el >From 8ddb42546242d56d9437a3d33d79719618aa54ac Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 04:06:22 +0100 Subject: [PATCH 4/4] Some minor fixes in lisp/net/sieve.el * lisp/net/sieve.el (sieve-next-line) (sieve-prev-line): Handle situations where point in `sieve-buffer' is either before or after the list of server side scripts. sieve-server-script-list: New variable. (sieve-refresh-scriptlist) (sieve-upload): Use `sieve-server-script-list' to make sure that local list of server side scripts in`sieve-buffer' is up-to-date after uploading a (new) script. --- lisp/net/sieve.el | 38 +++++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 13 deletions(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index fbd07dee27c..12a85e89d7e 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -95,6 +95,8 @@ sieve-template (defvar sieve-manage-buffer nil) (defvar sieve-buffer-header-end nil) +(defvar sieve-server-script-list nil + "Current list of server-side sieve scripts.") (defvar sieve-buffer-script-name nil "The real script name of the buffer.") (make-local-variable 'sieve-buffer-script-name) @@ -235,23 +237,28 @@ sieve-next-line (interactive) (unless arg (setq arg 1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "End of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "End of list")) + (goto-char (next-overlay-change sieve-buffer-header-end)))) (defun sieve-prev-line (&optional arg) (interactive) (unless arg (setq arg -1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "Beginning of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "Beginning of list")) + (goto-char (previous-overlay-change (point-max))) + (beginning-of-line))) (defun sieve-help () "Display help for various sieve commands." @@ -319,6 +326,9 @@ sieve-refresh-scriptlist (let* ((scripts (sieve-manage-listscripts sieve-manage-buffer)) (count (length scripts)) (keys (substitute-command-keys "\\[sieve-edit-script]"))) + (setq sieve-server-script-list + (mapcar (lambda (elt) (if (consp elt) (cdr elt) elt)) + scripts)) (insert (if (null scripts) (format @@ -361,7 +371,9 @@ sieve-upload (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." - (substitute-command-keys "\\[sieve-manage]")))) + (substitute-command-keys "\\[sieve-manage]")) + (when (not (member script-name sieve-server-script-list)) + (sieve-refresh-scriptlist)))) (set-buffer-modified-p nil)))) ;;;###autoload -- 2.39.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 19 03:21:28 2023 Received: (at 54154) by debbugs.gnu.org; 19 Jan 2023 08:21:28 +0000 Received: from localhost ([127.0.0.1]:42468 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIQAm-0008BL-1S for submit@debbugs.gnu.org; Thu, 19 Jan 2023 03:21:28 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:53331) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIMth-0002kQ-CK for 54154@debbugs.gnu.org; Wed, 18 Jan 2023 23:51:35 -0500 Received: from mail.tetzco.de (ipb21a9472.dynamic.kabel-deutschland.de [178.26.148.114]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id B0285BD078; Thu, 19 Jan 2023 05:51:24 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id C0B8A6C00B2; Thu, 19 Jan 2023 06:11:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674105071; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=EyH/WH63fPO0HEPM9ybraJ6PGUZzR5T0R111JStwP2c=; b=MfEN3PAQyy4A3Tfr+gVEyRKdxIv9m8g5SnNfgZEcj6j2pXm6MEi00HhQLor3/Xy7MMXuET cxqQobvhfw9rl+q7Ix7+coS0DImkV/Sqap5VnY1sUtTr8nm7vRcoRRMOVv7VBJnfDMrK0W bHkqVj5+4DnR6v6uCwgM7s5mbE8EmDpb75xKF5/uP9u/9jd02wGQP5xu3ciyKxrIw94Kp+ aZqLESZZp9I309sF2SgqQluG4k+i64ivIDR2tm8Ckyb1s0yfaDPCCa6BV4xVUa83Rq3tD8 w30Y2HqbjcKM3Jve9zzmY4uHskiJtWSYFN/QUrZdYsu+uYtArCvaPnCHKAupxBKvz1dL4+ eAT+pnvfqxSq5IYy096PC2r6tRKJWRBOR0viy7lClNiVi9ZqVSManL1/Sot438LrbBsoiO jfaxMSNIS3lUdd7yOq3qD5US2HOKAVUd85fhI/geOkX5l7TNqczHDXEdGoD57yIf8IX3jK gEKRv7IYwEbOqMASVVVq2gv/JTUDUqg7J2SQRvVgk9bwAMTyWS5ZllDos60rhFzo3n47E1 hOQy7LOZ+h6juhqgylXckE8dJXrnK6K1PaNMwewBQea1OVqrFJu0KsWoXMwUHip4wqDySE vJi9s03P4chvthkRykjx+aEIIzHfjh70/89eEQCRV/nYYE7od7BUg= From: Kai Tetzlaff To: "Herbert J. Skuhra" Subject: Re: [update] bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: (Herbert J. Skuhra's message of "Thu, 19 Jan 2023 00:22:44 +0100") Date: Thu, 19 Jan 2023 05:50:01 +0100 References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Message-ID: <871qnrgm9k.fsf@tetzco.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: C0B8A6C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.08 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-2.98)[-0.992]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:+,6:+,7:+]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; TAGGED_FROM(0.00)[bug]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; HAS_ATTACHMENT(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 X-Mailman-Approved-At: Thu, 19 Jan 2023 03:20:53 -0500 Cc: Eli Zaretskii , 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Content-Disposition: inline Sorry, the patch with the new test/lisp/net/sieve-manage-tests.el was incomplete and contained a bug. The attached updated patch(es) should fix this (the only change is in 0003-Add-test-lisp-net-sieve-manage-tests.el.patch). "Herbert J. Skuhra" writes: > On Wed, Jan 18, 2023 at 09:17:59PM +0200, Eli Zaretskii wrote: >> > this is strange because I can reproduce it easily on different systems: >> > >> > - master on FreeBSD 13.1-STABLE >> > - emacs-29 and master on macOS 12.6.2 >> > - master on WLS2/Windows11 (Ubuntu) I can now reproduce the error, too. The problem was that at the time Lars closed the bug report by applying the attached patch (after quite a long time of it sitting dormant), I had some additional local changes for sieve.el and sieve-manage.el on a branch which I didn't get to submit. And when I tried to reproduce the error, I've still been using this branch without realizing it. Sorry for that. >> ... >> Thanks. The error is here: >> >> (defun sieve-manage--append-to-log (&rest args) >> "Append ARGS to sieve-manage log buffer. >> >> ARGS can be a string or a list of strings. >> The buffer to use for logging is specifified via >> `sieve-manage-log'. If it is nil, logging is disabled." >> (when sieve-manage-log >> (with-current-buffer (or (get-buffer sieve-manage-log) >> (with-current-buffer <<<<<<<<<<<<<<<<<<<<<< >> (get-buffer-create sieve-manage-log) >> (set-buffer-multibyte nil) >> (buffer-disable-undo))) >> >> And I admit that I don't understand this code. What is it trying to >> do? Shouldn't it be just >> >> (when sieve-manage-log >> (with-current-buffer (get-buffer-create sieve-manage-log) >> (set-buffer-multibyte nil) >> (buffer-disable-undo))) >> >> Kai, am I missing something? The additional '(or ...' was meant to only run (set-buffer-multibyte nil) (buffer-disable-undo) once, when creating the log buffer (not everytime something gets appended to the log). What is missing in my code is an additional `current-buffer'. Here's the complete fixed function: (defun sieve-manage--append-to-log (&rest args) "Append ARGS to sieve-manage log buffer. ARGS can be a string or a list of strings. The buffer to use for logging is specifified via `sieve-manage-log'. If it is nil, logging is disabled." (when sieve-manage-log (with-current-buffer (or (get-buffer sieve-manage-log) (with-current-buffer (get-buffer-create sieve-manage-log) (set-buffer-multibyte nil) (buffer-disable-undo) (current-buffer))) (goto-char (point-max)) (apply #'insert args)))) >> Herbert, if you make the change above, does the problem go away? > > Yes, this change resolves the issue: > > diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el > index 5bee4f4c4a..636c7cbc5b 100644 > --- a/lisp/net/sieve-manage.el > +++ b/lisp/net/sieve-manage.el > @@ -174,11 +174,9 @@ sieve-manage--append-to-log > The buffer to use for logging is specifified via > `sieve-manage-log'. If it is nil, logging is disabled." > (when sieve-manage-log > - (with-current-buffer (or (get-buffer sieve-manage-log) > - (with-current-buffer > - (get-buffer-create sieve-manage-log) > + (with-current-buffer (get-buffer-create sieve-manage-log) > (set-buffer-multibyte nil) > - (buffer-disable-undo))) > + (buffer-disable-undo) > (goto-char (point-max)) > (apply #'insert args)))) > Here's a patch which preserves the logic of the original code: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-Fix-bug-in-sieve-manage-append-to-log.patch Content-Description: fix sieve-manage--append-to-log for Emacs 29 >From 4198e776da13b603c56acbae0ae89cd9d31cf207 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:16:14 +0100 Subject: [PATCH] Fix bug in sieve-manage--append-to-log * lisp/net/sieve-manage.el (sieve-manage--append-to-log): Fix log buffer creation --- lisp/net/sieve-manage.el | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..ab22294a272 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -178,7 +178,8 @@ sieve-manage--append-to-log (with-current-buffer (get-buffer-create sieve-manage-log) (set-buffer-multibyte nil) - (buffer-disable-undo))) + (buffer-disable-undo) + (current-buffer))) (goto-char (point-max)) (apply #'insert args)))) -- 2.39.0 --=-=-= Content-Type: text/plain Content-Disposition: inline The additional changes I mentioned above solve the problem in a different way by introducing a helper function. The also add some other improvements including a new test for handling multibyte characters in sieve server responses. I'm attaching the additional patches below. They might be too large for the current emacs-29 branch. But maybe they can be applied to master? --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Fix-bug-in-sieve-manage-append-to-log-improve-sieve-.patch Content-Description: fix sieve-manage--append-to-log >From 062ade7fe5cdbae36f4b488761e6367beb6e3a41 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:08:07 +0100 Subject: [PATCH 1/4] Fix bug in sieve-manage--append-to-log, improve sieve-manage buffer config Also update/add some comments/docs. * lisp/net/sieve-manage.el sieve-manage--coding-system: New constant. (sieve-manage--set-internal-buffer-properties): New function. (sieve-manage-open-server): Use `sieve-manage--coding-system'. (sieve-manage--append-to-log): Use `sieve-manage--set-internal-buffer-properties' to fix log buffer creation. (sieve-manage-encode) (sieve-manage-make-process-buffer): Use `sieve-manage--set-internal-buffer-properties'. --- lisp/net/sieve-manage.el | 50 +++++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 14 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..4f0f9b1891b 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -58,7 +58,7 @@ ;; ;; References: ;; -;; draft-martin-managesieve-02.txt, +;; RFC5804, ;; "A Protocol for Remotely Managing Sieve Scripts", ;; by Tim Martin. ;; @@ -145,6 +145,15 @@ sieve-manage-ignore-starttls :type 'boolean) ;; Internal variables: +(defconst sieve-manage--coding-system 'raw-text-unix + "Use 'raw-text-unix coding system for (network) communication. + +Sets the coding system used for the internal (process, log) +buffers and the network stream created to communicate with the +managesieve server. Using 'raw-text encoding enables unibyte +mode and makes sure that sent/received octets (bytes) remain +untouched by the coding system. The explicit use of `-unix` +avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") intact).") (defconst sieve-manage-local-variables '(sieve-manage-server sieve-manage-port @@ -167,6 +176,24 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions +(defun sieve-manage--set-internal-buffer-properties (buffer) + "Set BUFFER properties for internally used buffers. + +Used for process and log buffers, this function makes sure that +those buffers keep received and sent data intact by: +- setting the coding system to 'sieve-manage--coding-system', +- setting `after-change-functions' to nil to avoid those + functions messing with buffer content. +Also disables undo (to save a bit of memory and improve +performance). + +Returns BUFFER." + (with-current-buffer buffer + (set-buffer-file-coding-system sieve-manage--coding-system) + (setq-local after-change-functions nil) + (buffer-disable-undo) + (current-buffer))) + (defun sieve-manage--append-to-log (&rest args) "Append ARGS to sieve-manage log buffer. @@ -175,10 +202,8 @@ sieve-manage--append-to-log `sieve-manage-log'. If it is nil, logging is disabled." (when sieve-manage-log (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) - (set-buffer-multibyte nil) - (buffer-disable-undo))) + (sieve-manage--set-internal-buffer-properties + (get-buffer-create sieve-manage-log))) (goto-char (point-max)) (apply #'insert args)))) @@ -204,7 +229,7 @@ sieve-manage--error (defun sieve-manage-encode (utf8-string) "Convert UTF8-STRING to managesieve protocol octets." - (encode-coding-string utf8-string 'raw-text t)) + (encode-coding-string utf8-string sieve-manage--coding-system t)) (defun sieve-manage-decode (octets &optional buffer) "Convert managesieve protocol OCTETS to utf-8 string. @@ -216,13 +241,11 @@ sieve-manage-decode (defun sieve-manage-make-process-buffer () (with-current-buffer - (generate-new-buffer (format " *sieve %s:%s*" - sieve-manage-server - sieve-manage-port)) + (sieve-manage--set-internal-buffer-properties + (generate-new-buffer (format " *sieve %s:%s*" + sieve-manage-server + sieve-manage-port))) (mapc #'make-local-variable sieve-manage-local-variables) - (set-buffer-multibyte nil) - (setq-local after-change-functions nil) - (buffer-disable-undo) (current-buffer))) (defun sieve-manage-erase (&optional p buffer) @@ -244,8 +267,7 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream - ;; eol type unix is required to preserve "\r\n" - :coding 'raw-text-unix + :coding `(binary . ,sieve-manage--coding-system) :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-Handle-BYE-in-sieve-manage-server-responses.patch Content-Description: handle BYE in sieve-manage.el >From e09d82fec3fc86d797396b753f3b8c4411df9d1f Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:52:43 +0100 Subject: [PATCH 2/4] Handle BYE in sieve-manage server responses * lisp/net/sieve-manage.el (sieve-manage-regex-oknobye): New function. (sieve-manage-parse-oknobye): Renamed from `sieve-manage-parse-okno'. (sieve-manage-open-server) (sieve-sasl-auth) (sieve-manage-listscripts) (sieve-manage-putscript) (sieve-manage-deletescript) (sieve-manage-getscript) (sieve-manage-setactive) (sieve-manage-is-okno) (sieve-manage-parse-oknobye) (sieve-manage-parse-parse-string) (sieve-manage-parse-parse-crlf): Use `sieve-manage-regex-oknobye' to handle BYE in addition to OK and NO. --- lisp/net/sieve-manage.el | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 4f0f9b1891b..4e63603bbab 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -269,8 +269,8 @@ sieve-manage-open-server :type stream :coding `(binary . ,sieve-manage--coding-system) :capability-command "CAPABILITY\r\n" - :end-of-command "^\\(OK\\|NO\\).*\n" - :success "^OK.*\n" + :end-of-command "^\\(OK\\|NO\\|BYE\\).*\r\n" + :success "^OK.*\r\n" :return-list t :starttls-function (lambda (capabilities) @@ -329,7 +329,7 @@ sieve-sasl-auth (setq rsp nil) (goto-char (match-end 0)) rsp)) - (setq rsp (sieve-manage-is-okno)))) + (setq rsp (sieve-manage-is-oknobye)))) (accept-process-output sieve-manage-process 1) (goto-char (point-min))) (sieve-manage-erase) @@ -498,19 +498,19 @@ sieve-manage-listscripts (defun sieve-manage-havespace (name size &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name (length (sieve-manage-encode content)) sieve-manage-client-eol content)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "DELETESCRIPT \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -518,17 +518,22 @@ sieve-manage-getscript (sieve-manage-decode (sieve-manage-parse-string) output-buffer) (sieve-manage-parse-crlf) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "SETACTIVE \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) ;; Protocol parsing routines +(defun sieve-manage-regexp-oknobye () + "Return regexp for managesieve 'response-oknobye'." + (concat + "^\\(OK\\|NO\\|BYE\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" + sieve-manage-server-eol)) (defun sieve-manage-wait-for-answer () - (let ((pattern "^\\(OK\\|NO\\).*\n") + (let ((pattern (sieve-manage-regexp-oknobye)) pos) (while (not pos) (setq pos (search-forward-regexp pattern nil t)) @@ -546,10 +551,8 @@ sieve-manage-ok-p (defun sieve-manage-no-p (rsp) (string= (downcase (or (car-safe rsp) "")) "no")) -(defun sieve-manage-is-okno () - (when (looking-at (concat - "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" - sieve-manage-server-eol)) +(defun sieve-manage-is-oknobye () + (when (looking-at (sieve-manage-regexp-oknobye)) (let ((status (match-string 1)) (resp-code (match-string 3)) (response (match-string 5))) @@ -558,12 +561,12 @@ sieve-manage-is-okno (setq response (sieve-manage-is-string))) (list status resp-code response)))) -(defun sieve-manage-parse-okno () +(defun sieve-manage-parse-oknobye () (let (rsp) (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-okno))) + (setq rsp (sieve-manage-is-oknobye))) (sieve-manage-erase) rsp)) @@ -597,7 +600,7 @@ sieve-manage-parse-string (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) (unless (setq rsp (sieve-manage-is-string)) - (when (sieve-manage-no-p (sieve-manage-is-okno)) + (when (sieve-manage-no-p (sieve-manage-is-oknobye)) ;; simple `error' is enough since `sieve-manage-erase' ;; already adds the server response to the log (error (sieve-manage-erase))))) @@ -611,7 +614,7 @@ sieve-manage-parse-crlf (defun sieve-manage-parse-listscripts () (let (tmp rsp data) (while (null rsp) - (while (null (or (setq rsp (sieve-manage-is-okno)) + (while (null (or (setq rsp (sieve-manage-is-oknobye)) (setq tmp (sieve-manage-decode (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) -- 2.39.0 --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0003-Add-test-lisp-net-sieve-manage-tests.el.patch Content-Transfer-Encoding: quoted-printable Content-Description: test for sieve-manage multibyte character handling >From 8ef9ad845b5a07e75b26ec226a558ea32a593237 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Tue, 22 Mar 2022 20:48:09 +0100 Subject: [PATCH 3/4] Add test/lisp/net/sieve-manage-tests.el * test/lisp/net/sieve-manage-tests.el: (mk-literal-s2c) (mk-rsp-oknobye) (mk-rsp-getscript-ok) (managesieve-getscript): Test helper functions. (ert/managesieve-getscript-multibyte): Test `sieve-manage-getscript' with script names containing multibyte characters. --- test/lisp/net/sieve-manage-tests.el | 106 ++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 test/lisp/net/sieve-manage-tests.el diff --git a/test/lisp/net/sieve-manage-tests.el b/test/lisp/net/sieve-mana= ge-tests.el new file mode 100644 index 00000000000..d4c616b3364 --- /dev/null +++ b/test/lisp/net/sieve-manage-tests.el @@ -0,0 +1,106 @@ +;;; sieve-manage-tests.el --- tests for sieve-manage.el -*- lexical-= binding: t; -*- + +;; Copyright (C) 2022 Free Software Foundation, Inc. + +;; Author: Kai Tetzlaff + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see . + +;;; Commentary: + + +;;; Code: + +(require 'ert) +(require 'sieve-manage) + +(defvar sieve-script-multibyte-unix + "if body :matches \"=C3=A4\" { stop; }\n" + "Simple multibyte sieve script.") + +(defun mk-literal-s2c (string) + "Encode STRING to managesieve 'literal-s2c'." + (let ((data (sieve-manage-encode string))) + (concat (format "{%d}\r\n" (length data)) + data))) + +(defun mk-rsp-oknobye (type &optional resp-code string) + "Encode TYPE, RESP-CODE and STRING to managesieve 'response-oknobye'." + (when (memq type '(OK NO BYE)) + (concat + (mapconcat #'identity + (delq nil + (list (symbol-name type) + (when resp-code + (format "(%s)" resp-code)) + (when string + (format "\"%s\"" + (sieve-manage-encode string))))) + " ") + "\r\n"))) +;; (mk-rsp-oknobye 'OK nil "Getscript completed.") +;; (mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") + +(defun mk-rsp-getscript-ok (script) + "Encode SCRIPT to managesieve 'response-getscript'." + (concat (mk-literal-s2c script) + "\r\n" + (mk-rsp-oknobye 'OK "Getscript completed."))) +;; (mk-rsp-getscript-ok sieve-script-multibyte-unix) + +(defun managesieve-getscript (script) + "Simulate managesieve getscript response to test +`sieve-manage-getscript' function." + (let* ((script-name "test.sieve") + ;; `sieve-manage-server' and `sieve-manage-port' are used in + ;; `sieve-manage-make-process-buffer' + (sieve-manage-server) + (sieve-manage-port "sieve") + (sieve-buffer (sieve-manage-make-process-buffer)) + (output-buffer (generate-new-buffer script-name))) + ;; use cl-letf to mock some functions in call chain of + ;; sieve-manage-getscript + (cl-letf (((symbol-function 'sieve-manage-send) + ;; simulate sieve server response with a single + ;; multibyte character `=C3=A4` + (lambda (_) + (with-current-buffer sieve-buffer + (goto-char (point-min)) + (insert (mk-rsp-getscript-ok script))))) + ((symbol-function 'accept-process-output) + (lambda (&optional _ _ _ _) nil)) + ((symbol-function 'get-buffer-process) (lambda (_) nil))) + ;; extract sieve script from sieve-buffer and put it into + ;; output-buffer + (sieve-manage-getscript script-name output-buffer sieve-buffer) + ;; extract script from output-buffer and return it as a string + (let ((script (with-current-buffer output-buffer + (set-buffer-modified-p nil) + (buffer-string)))) + ;; cleanup + (kill-buffer sieve-buffer) + (kill-buffer output-buffer) + (when (get-buffer sieve-manage-log) + (kill-buffer sieve-manage-log)) + ;; return script + script)))) + +(ert-deftest ert/managesieve-getscript-multibyte () + (should (string=3D sieve-script-multibyte-unix + (managesieve-getscript sieve-script-multibyte-unix)))) + + +;;; sieve-manage-tests.el ends here --=20 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0004-Some-minor-fixes-in-lisp-net-sieve.el.patch Content-Description: minor fixes in lisp/net/sieve.el >From 59457ec8ec88e4c0b22499f782b8db24fb7b5cc4 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 04:06:22 +0100 Subject: [PATCH 4/4] Some minor fixes in lisp/net/sieve.el * lisp/net/sieve.el (sieve-next-line) (sieve-prev-line): Handle situations where point in `sieve-buffer' is either before or after the list of server side scripts. sieve-server-script-list: New variable. (sieve-refresh-scriptlist) (sieve-upload): Use `sieve-server-script-list' to make sure that local list of server side scripts in`sieve-buffer' is up-to-date after uploading a (new) script. --- lisp/net/sieve.el | 38 +++++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 13 deletions(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index fbd07dee27c..12a85e89d7e 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -95,6 +95,8 @@ sieve-template (defvar sieve-manage-buffer nil) (defvar sieve-buffer-header-end nil) +(defvar sieve-server-script-list nil + "Current list of server-side sieve scripts.") (defvar sieve-buffer-script-name nil "The real script name of the buffer.") (make-local-variable 'sieve-buffer-script-name) @@ -235,23 +237,28 @@ sieve-next-line (interactive) (unless arg (setq arg 1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "End of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "End of list")) + (goto-char (next-overlay-change sieve-buffer-header-end)))) (defun sieve-prev-line (&optional arg) (interactive) (unless arg (setq arg -1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "Beginning of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "Beginning of list")) + (goto-char (previous-overlay-change (point-max))) + (beginning-of-line))) (defun sieve-help () "Display help for various sieve commands." @@ -319,6 +326,9 @@ sieve-refresh-scriptlist (let* ((scripts (sieve-manage-listscripts sieve-manage-buffer)) (count (length scripts)) (keys (substitute-command-keys "\\[sieve-edit-script]"))) + (setq sieve-server-script-list + (mapcar (lambda (elt) (if (consp elt) (cdr elt) elt)) + scripts)) (insert (if (null scripts) (format @@ -361,7 +371,9 @@ sieve-upload (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." - (substitute-command-keys "\\[sieve-manage]")))) + (substitute-command-keys "\\[sieve-manage]")) + (when (not (member script-name sieve-server-script-list)) + (sieve-refresh-scriptlist)))) (set-buffer-modified-p nil)))) ;;;###autoload -- 2.39.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 19 07:38:33 2023 Received: (at 54154) by debbugs.gnu.org; 19 Jan 2023 12:38:33 +0000 Received: from localhost ([127.0.0.1]:42839 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIUBc-0000ug-9x for submit@debbugs.gnu.org; Thu, 19 Jan 2023 07:38:33 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:35783) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIUBX-0000uP-VD for 54154@debbugs.gnu.org; Thu, 19 Jan 2023 07:38:30 -0500 Received: from mail.tetzco.de (ipb21a9472.dynamic.kabel-deutschland.de [178.26.148.114]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id 989E7BD078; Thu, 19 Jan 2023 13:38:21 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id 46E456C00B2; Thu, 19 Jan 2023 13:58:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674133087; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=J2Yvx3mqvjTCCgnQTK4CUYyufNtXFt/BN12KdRdn4K4=; b=cAyS1RJiDxV2HtfPVkzA3hFMXwmIEbN2pwRE2fzRPUvxTFdZ5mSTrwrhfaridvVuUmjEKl Fh4sl/PZEQP/MLX5UN09ZIxO9y7meUWupjlrafP5thTwfK2ytO4UA6LTkcm8GBuQkKXUl4 Qc39UDiy8Qc3oln0mNuedzySGnhJn+7H0pFY1s2b+JmyaYubGiq7A2aVWJy94XMptN6/kQ X2IAWldbGiWjs3b2VrDbD5QkYckmOsXNk/+WlJMa24jA6pbMBNaMtbgGaBrlyv6oRT9XG9 a1SWZ3Kc4fEP2taEOGY/u0wu3K07V9B9boGPVmDLHpIonHBbQ5zOH5+h926sj4DZ2uaHBz ezV61Fgf7S3kfqsUEal6be+vwLmUK23nQOb8apIXCYgmvkfZULZIAtxm3Jrlr8LhhsNgA7 rmMNO48KPGtaOrw7M5hUSX/i3fRuTVPRbNn/FKdiSG8ET3ZzKiPSmGoFWIVm0OEiDl4mVa sFz131joQd4bJszL9o0FfVscs5VV4bNPCN/FB1oZ53PBcu1Y5GKybeI3aU500tJJSHn31C MrQsaeTYtVmdQ2Og39OOCP3cz1wARcu9+8K+tsKId/p2arHr9Viij87+8ppVdCYzYK/UNS mSVieQ2yMCqnALiPo4PfcEelEh5hHF9qadmafbsiQdqZeJTkP1Fyw= From: Kai Tetzlaff To: Eli Zaretskii Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: <83tu0nynla.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 19 Jan 2023 09:45:21 +0200") References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> Date: Thu, 19 Jan 2023 13:38:13 +0100 Message-ID: <873586d7ii.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: 46E456C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.07 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-2.97)[-0.991]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:+,6:+,7:+]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; TAGGED_FROM(0.00)[bug]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; HAS_ATTACHMENT(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Hello Eli, thanks for looking into this! Eli Zaretskii writes: > The duplicate use of with-current-buffer is sub-optimal, > IMO. What about the simpler code below: > > (when sieve-manage-log > (let* ((existing-log-buffer (get-buffer sieve-manage-log)) > (log-buffer (or existing-log-buffer > (get-buffer-create sieve-manage-log)))) > (with-current-buffer log-buffer > (unless existing-log-buffer > ;; Do this only once, when creating the log buffer. > (set-buffer-multibyte nil) > (buffer-disable-undo)) > (goto-char (point-max)) > (apply #'insert args))))) Yes, that provides more insight into what the code intends to do. Here's the patch (with additional updated doc string): --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-Fix-bug-in-sieve-manage-append-to-log-emacs-29-only.patch Content-Description: fix sieve-manage--append-to-log for emacs-29 >From 62d03f302125c0b1aab2e3ae4f5b12b531d30d74 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:16:14 +0100 Subject: [PATCH] ; Fix bug in sieve-manage--append-to-log (emacs-29 only) This is emacs-29 only, use more elaborate fix for Emacs 30.x (master). * lisp/net/sieve-manage.el (sieve-manage--append-to-log): Fix log buffer creation. --- lisp/net/sieve-manage.el | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..4866f788bff 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -168,19 +168,25 @@ sieve-manage-capability ;; Internal utility functions (defun sieve-manage--append-to-log (&rest args) - "Append ARGS to sieve-manage log buffer. + "Append ARGS to `sieve-manage-log' buffer. ARGS can be a string or a list of strings. -The buffer to use for logging is specifified via -`sieve-manage-log'. If it is nil, logging is disabled." +The buffer to use for logging is specifified via `sieve-manage-log'. +If it is nil, logging is disabled. + +When the `sieve-manage-log' buffer doesn't exist, it gets created (and +configured with some initial settings)." (when sieve-manage-log - (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) - (set-buffer-multibyte nil) - (buffer-disable-undo))) - (goto-char (point-max)) - (apply #'insert args)))) + (let* ((existing-log-buffer (get-buffer sieve-manage-log)) + (log-buffer (or existing-log-buffer + (get-buffer-create sieve-manage-log)))) + (with-current-buffer log-buffer + (unless existing-log-buffer + ;; Do this only once, when creating the log buffer. + (set-buffer-multibyte nil) + (buffer-disable-undo)) + (goto-char (point-max)) + (apply #'insert args))))) (defun sieve-manage--message (format-string &rest args) "Wrapper around `message' which also logs to sieve manage log. -- 2.39.0 --=-=-= Content-Type: text/plain >> ;; Internal utility functions >> +(defun sieve-manage--set-internal-buffer-properties (buffer) >> + "Set BUFFER properties for internally used buffers. >> + >> +Used for process and log buffers, this function makes sure that >> +those buffers keep received and sent data intact by: >> +- setting the coding system to 'sieve-manage--coding-system', >> +- setting `after-change-functions' to nil to avoid those >> + functions messing with buffer content. >> +Also disables undo (to save a bit of memory and improve >> +performance). >> + >> +Returns BUFFER." >> + (with-current-buffer buffer >> + (set-buffer-file-coding-system sieve-manage--coding-system) >> + (setq-local after-change-functions nil) >> + (buffer-disable-undo) >> + (current-buffer))) >> + >> (defun sieve-manage--append-to-log (&rest args) >> "Append ARGS to sieve-manage log buffer. >> >> @@ -175,10 +202,8 @@ sieve-manage--append-to-log >> `sieve-manage-log'. If it is nil, logging is disabled." >> (when sieve-manage-log >> (with-current-buffer (or (get-buffer sieve-manage-log) >> - (with-current-buffer >> - (get-buffer-create sieve-manage-log) >> - (set-buffer-multibyte nil) >> - (buffer-disable-undo))) >> + (sieve-manage--set-internal-buffer-properties >> + (get-buffer-create sieve-manage-log))) >> (goto-char (point-max)) >> (apply #'insert args)))) > > This still uses a less-than-elegant implementation that calls > with-current-buffer twice. Yes, true. But since `sieve-manage--set-internal-buffer-properties' is used in two different places, the more elegant solution you suggested above would require duplicating the body of the function in those places. I just didn't see a better way. In particular because `set-buffer-file-coding-system' and `setq-local' only work with (current-buffer). If you can point me to code which can replace these with something that takes BUFFER arguments, I can rewrite `sieve-manage--set-internal-buffer-properties' and avoid using `with-current-buffer'. >> (defun sieve-manage-encode (utf8-string) >> "Convert UTF8-STRING to managesieve protocol octets." >> - (encode-coding-string utf8-string 'raw-text t)) >> + (encode-coding-string utf8-string sieve-manage--coding-system t)) > > Why is the argument called utf8-string? If it's indeed a string > encoded in UTF-8, why do you need to encode it again with > raw-text-unix? it should be a no-op in that case. So please tell more > about the underlying issue. I chose the name as a hint to the user that the incoming string should be UTF-8 encoded. But that is probably misleading since the string itself doesn't have an encoding? So let's change the function to: (defun sieve-manage-encode (str) "Convert STR to managesieve protocol octets." (encode-coding-string str sieve-manage--coding-system t)) Regarding the potential double encoding: When sending data over the network connection, `sieve-manage-encode' intends to make sure that `utf8-string' data is converted to a byte/octet representation. I tried to explain that in the doc string of `sieve-manage--coding-system': (defconst sieve-manage--coding-system 'raw-text-unix "Use 'raw-text-unix coding system for (network) communication. Sets the coding system used for the internal (process, log) buffers and the network stream created to communicate with the managesieve server. Using 'raw-text encoding enables unibyte mode and makes sure that sent/received octets (bytes) remain untouched by the coding system. The explicit use of `-unix` avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") intact).") The original problem was that when communicating with the sievemanage server, we need to handle length elements where we need make sure that calculated values take into account that UTF-8 characters may comprise multiple octets. Even after reading the relevant sections of the documentation multiple times I was (and am still) not sure what exactly the various coding system settings do and how they interact with buffers and networking functions. So forgive me if what I'm doing there looks weird to your expert eyes. When working on the original patch, I had several uhoh moments where data sent to or received from the network seemed to have been automatically modified by the coding system (unfortunately, I don't remember the exact details). So I tried to eliminate any such automatic modifications by using 'binary or 'raw-text encodings on code paths which handle network data. Basically, my thinking was: 'better do things twice/thrice/... before introducing new points of failure'. >> @@ -244,8 +267,7 @@ sieve-manage-open-server >> (open-network-stream >> "SIEVE" buffer server port >> :type stream >> - ;; eol type unix is required to preserve "\r\n" >> - :coding 'raw-text-unix >> + :coding `(binary . ,sieve-manage--coding-system) > > Same question about encoding with raw-text-unix here: using it means > some other code will need to encode the text with a real encoding, > which in this case is UTF-8 (AFAIU the managesieve protocol RFC). So > why not use utf-8-unix here? Same as above: I'm just not sure that this is the right thing. But after thinking about it some more, I made the following changes (as an experiment): 1. set `sieve-manage--coding-system' to 'utf-8-unix and 2. changed the call to `open-network-stream' in the patch above to >> @@ -244,8 +267,7 @@ sieve-manage-open-server >> (open-network-stream >> "SIEVE" buffer server port >> :type stream >> - ;; eol type unix is required to preserve "\r\n" >> - :coding 'raw-text-unix >> + :coding sieve-manage--coding-system instead of the previous, asymmetric mix of `binary and `sieve-manage--coding-system'. With these changes, there are still no issues when connecting to my managesieve server which still contains a script with a name that contains utf-8 multibyte characters. Also, the test I wrote still works with that change. So if you think that this makes things clearer, I'm happy to make these changes. I'm just don't feel confident enough to do this without additional guidance. I was also experimenting with some additional changes with the hope to to just use coding system settings instead of calling `sieve-manage-encode'/`sieve-manage-decode'. But I couldn't get that to work. I added an updated set of Emacs 30.x patches with the changes described above (plus an additional change in sieve.el which makes sure that the sieve-manage buffer containing the list of available sieve scripts is also using coding system 'utf-8-unix). > Should the addition of BYE support be mentioned in NEWS? I can certainly do that if you think that this is useful. It just seems to be more of an internal detail which probably doesn't mean much to most users. > On balance, I think the additional patches should go to master, > indeed. But let's resolve the issues mentioned above first. Ok, awaiting further input... --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Fix-bug-in-sieve-manage-append-to-log-improve-sieve-.patch >From f8d255124777c7da5c029ac7be04d770cd4dc0f3 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:08:07 +0100 Subject: [PATCH 1/4] Fix bug in sieve-manage--append-to-log, improve sieve-manage buffer config Also update/add some comments/docs. * lisp/net/sieve-manage.el (sieve-manage-coding-system-for-read) (sieve-manage-coding-system-for-write): Remove unused constants. (sieve-manage-encode): Change misleading argument name (utf8-string -> str). (sieve-manage--coding-system): New constant. (sieve-manage--set-internal-buffer-properties): New function. (sieve-manage-open-server): Use `sieve-manage--coding-system'. (sieve-manage--append-to-log): Use `sieve-manage--set-internal-buffer-properties' to fix log buffer creation. (sieve-manage-encode) (sieve-manage-make-process-buffer): Use `sieve-manage--set-internal-buffer-properties'. --- lisp/net/sieve-manage.el | 65 +++++++++++++++++++++++++++------------- 1 file changed, 44 insertions(+), 21 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..7721ca139b7 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -58,7 +58,7 @@ ;; ;; References: ;; -;; draft-martin-managesieve-02.txt, +;; RFC5804, ;; "A Protocol for Remotely Managing Sieve Scripts", ;; by Tim Martin. ;; @@ -145,6 +145,15 @@ sieve-manage-ignore-starttls :type 'boolean) ;; Internal variables: +(defconst sieve-manage--coding-system 'utf-8-unix + "Use \\='utf-8-unix coding system for (network) communication. + +Defines the coding system used for the internal (process, log) +buffers and the network stream created to communicate with the +managesieve server. Using \\='utf-8-unix encoding corresponds to +the use of UTF-8 in RFC5804 (managesieve). The explicit use of +\\='-unix\\=' avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") +sequences intact).") (defconst sieve-manage-local-variables '(sieve-manage-server sieve-manage-port @@ -154,8 +163,6 @@ sieve-manage-local-variables sieve-manage-client-eol sieve-manage-server-eol sieve-manage-capability)) -(defconst sieve-manage-coding-system-for-read 'binary) -(defconst sieve-manage-coding-system-for-write 'binary) (defvar sieve-manage-stream nil) (defvar sieve-manage-auth nil) (defvar sieve-manage-server nil) @@ -167,18 +174,37 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions +(defun sieve-manage--set-internal-buffer-properties (buffer) + "Set BUFFER properties for internally used buffers. + +Used for process and log buffers, this function makes sure that +those buffers keep received and sent data intact by: +- setting the coding system to 'sieve-manage--coding-system', +- setting `after-change-functions' to nil to avoid those + functions messing with buffer content. +Also disables undo (to save a bit of memory and improve +performance). + +Returns BUFFER." + (with-current-buffer buffer + (set-buffer-file-coding-system sieve-manage--coding-system) + (setq-local after-change-functions nil) + (buffer-disable-undo) + (current-buffer))) + (defun sieve-manage--append-to-log (&rest args) - "Append ARGS to sieve-manage log buffer. + "Append ARGS to `sieve-manage-log' buffer. ARGS can be a string or a list of strings. -The buffer to use for logging is specifified via -`sieve-manage-log'. If it is nil, logging is disabled." +The buffer to use for logging is specifified via `sieve-manage-log'. +If it is nil, logging is disabled. + +When the `sieve-manage-log' buffer doesn't exist, it gets created (and +configured with some initial settings)." (when sieve-manage-log (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) - (set-buffer-multibyte nil) - (buffer-disable-undo))) + (sieve-manage--set-internal-buffer-properties + (get-buffer-create sieve-manage-log))) (goto-char (point-max)) (apply #'insert args)))) @@ -202,9 +228,9 @@ sieve-manage--error (sieve-manage--append-to-log msg "\n") (error msg))) -(defun sieve-manage-encode (utf8-string) - "Convert UTF8-STRING to managesieve protocol octets." - (encode-coding-string utf8-string 'raw-text t)) +(defun sieve-manage-encode (str) + "Convert STR to managesieve protocol octets." + (encode-coding-string str sieve-manage--coding-system t)) (defun sieve-manage-decode (octets &optional buffer) "Convert managesieve protocol OCTETS to utf-8 string. @@ -216,13 +242,11 @@ sieve-manage-decode (defun sieve-manage-make-process-buffer () (with-current-buffer - (generate-new-buffer (format " *sieve %s:%s*" - sieve-manage-server - sieve-manage-port)) + (sieve-manage--set-internal-buffer-properties + (generate-new-buffer (format " *sieve %s:%s*" + sieve-manage-server + sieve-manage-port))) (mapc #'make-local-variable sieve-manage-local-variables) - (set-buffer-multibyte nil) - (setq-local after-change-functions nil) - (buffer-disable-undo) (current-buffer))) (defun sieve-manage-erase (&optional p buffer) @@ -244,8 +268,7 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream - ;; eol type unix is required to preserve "\r\n" - :coding 'raw-text-unix + :coding sieve-manage--coding-system :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-Handle-BYE-in-sieve-manage-server-responses.patch >From 84d6b0cb6559b27b506556aa32894cc6d2752508 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:52:43 +0100 Subject: [PATCH 2/4] Handle BYE in sieve-manage server responses * lisp/net/sieve-manage.el (sieve-manage-regex-oknobye): New function. (sieve-manage-parse-oknobye): Renamed from `sieve-manage-parse-okno'. (sieve-manage-open-server) (sieve-sasl-auth) (sieve-manage-listscripts) (sieve-manage-putscript) (sieve-manage-deletescript) (sieve-manage-getscript) (sieve-manage-setactive) (sieve-manage-is-okno) (sieve-manage-parse-oknobye) (sieve-manage-parse-parse-string) (sieve-manage-parse-parse-crlf): Use `sieve-manage-regex-oknobye' to handle BYE in addition to OK and NO. --- lisp/net/sieve-manage.el | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 7721ca139b7..b1a77219c61 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -270,8 +270,8 @@ sieve-manage-open-server :type stream :coding sieve-manage--coding-system :capability-command "CAPABILITY\r\n" - :end-of-command "^\\(OK\\|NO\\).*\n" - :success "^OK.*\n" + :end-of-command "^\\(OK\\|NO\\|BYE\\).*\r\n" + :success "^OK.*\r\n" :return-list t :starttls-function (lambda (capabilities) @@ -330,7 +330,7 @@ sieve-sasl-auth (setq rsp nil) (goto-char (match-end 0)) rsp)) - (setq rsp (sieve-manage-is-okno)))) + (setq rsp (sieve-manage-is-oknobye)))) (accept-process-output sieve-manage-process 1) (goto-char (point-min))) (sieve-manage-erase) @@ -499,19 +499,19 @@ sieve-manage-listscripts (defun sieve-manage-havespace (name size &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name (length (sieve-manage-encode content)) sieve-manage-client-eol content)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "DELETESCRIPT \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -519,17 +519,22 @@ sieve-manage-getscript (sieve-manage-decode (sieve-manage-parse-string) output-buffer) (sieve-manage-parse-crlf) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "SETACTIVE \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) ;; Protocol parsing routines +(defun sieve-manage-regexp-oknobye () + "Return regexp for managesieve 'response-oknobye'." + (concat + "^\\(OK\\|NO\\|BYE\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" + sieve-manage-server-eol)) (defun sieve-manage-wait-for-answer () - (let ((pattern "^\\(OK\\|NO\\).*\n") + (let ((pattern (sieve-manage-regexp-oknobye)) pos) (while (not pos) (setq pos (search-forward-regexp pattern nil t)) @@ -547,10 +552,8 @@ sieve-manage-ok-p (defun sieve-manage-no-p (rsp) (string= (downcase (or (car-safe rsp) "")) "no")) -(defun sieve-manage-is-okno () - (when (looking-at (concat - "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" - sieve-manage-server-eol)) +(defun sieve-manage-is-oknobye () + (when (looking-at (sieve-manage-regexp-oknobye)) (let ((status (match-string 1)) (resp-code (match-string 3)) (response (match-string 5))) @@ -559,12 +562,12 @@ sieve-manage-is-okno (setq response (sieve-manage-is-string))) (list status resp-code response)))) -(defun sieve-manage-parse-okno () +(defun sieve-manage-parse-oknobye () (let (rsp) (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-okno))) + (setq rsp (sieve-manage-is-oknobye))) (sieve-manage-erase) rsp)) @@ -598,7 +601,7 @@ sieve-manage-parse-string (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) (unless (setq rsp (sieve-manage-is-string)) - (when (sieve-manage-no-p (sieve-manage-is-okno)) + (when (sieve-manage-no-p (sieve-manage-is-oknobye)) ;; simple `error' is enough since `sieve-manage-erase' ;; already adds the server response to the log (error (sieve-manage-erase))))) @@ -612,7 +615,7 @@ sieve-manage-parse-crlf (defun sieve-manage-parse-listscripts () (let (tmp rsp data) (while (null rsp) - (while (null (or (setq rsp (sieve-manage-is-okno)) + (while (null (or (setq rsp (sieve-manage-is-oknobye)) (setq tmp (sieve-manage-decode (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) -- 2.39.0 --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0003-Add-test-lisp-net-sieve-manage-tests.el.patch Content-Transfer-Encoding: quoted-printable >From 07ace6346d66862fc84cb3fd1764f6b676558430 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Tue, 22 Mar 2022 20:48:09 +0100 Subject: [PATCH 3/4] Add test/lisp/net/sieve-manage-tests.el * test/lisp/net/sieve-manage-tests.el: (mk-literal-s2c) (mk-rsp-oknobye) (mk-rsp-getscript-ok) (managesieve-getscript): Test helper functions. (ert/managesieve-getscript-multibyte): Test `sieve-manage-getscript' with script names containing multibyte characters. --- test/lisp/net/sieve-manage-tests.el | 106 ++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 test/lisp/net/sieve-manage-tests.el diff --git a/test/lisp/net/sieve-manage-tests.el b/test/lisp/net/sieve-mana= ge-tests.el new file mode 100644 index 00000000000..d4c616b3364 --- /dev/null +++ b/test/lisp/net/sieve-manage-tests.el @@ -0,0 +1,106 @@ +;;; sieve-manage-tests.el --- tests for sieve-manage.el -*- lexical-= binding: t; -*- + +;; Copyright (C) 2022 Free Software Foundation, Inc. + +;; Author: Kai Tetzlaff + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see . + +;;; Commentary: + + +;;; Code: + +(require 'ert) +(require 'sieve-manage) + +(defvar sieve-script-multibyte-unix + "if body :matches \"=C3=A4\" { stop; }\n" + "Simple multibyte sieve script.") + +(defun mk-literal-s2c (string) + "Encode STRING to managesieve 'literal-s2c'." + (let ((data (sieve-manage-encode string))) + (concat (format "{%d}\r\n" (length data)) + data))) + +(defun mk-rsp-oknobye (type &optional resp-code string) + "Encode TYPE, RESP-CODE and STRING to managesieve 'response-oknobye'." + (when (memq type '(OK NO BYE)) + (concat + (mapconcat #'identity + (delq nil + (list (symbol-name type) + (when resp-code + (format "(%s)" resp-code)) + (when string + (format "\"%s\"" + (sieve-manage-encode string))))) + " ") + "\r\n"))) +;; (mk-rsp-oknobye 'OK nil "Getscript completed.") +;; (mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") + +(defun mk-rsp-getscript-ok (script) + "Encode SCRIPT to managesieve 'response-getscript'." + (concat (mk-literal-s2c script) + "\r\n" + (mk-rsp-oknobye 'OK "Getscript completed."))) +;; (mk-rsp-getscript-ok sieve-script-multibyte-unix) + +(defun managesieve-getscript (script) + "Simulate managesieve getscript response to test +`sieve-manage-getscript' function." + (let* ((script-name "test.sieve") + ;; `sieve-manage-server' and `sieve-manage-port' are used in + ;; `sieve-manage-make-process-buffer' + (sieve-manage-server) + (sieve-manage-port "sieve") + (sieve-buffer (sieve-manage-make-process-buffer)) + (output-buffer (generate-new-buffer script-name))) + ;; use cl-letf to mock some functions in call chain of + ;; sieve-manage-getscript + (cl-letf (((symbol-function 'sieve-manage-send) + ;; simulate sieve server response with a single + ;; multibyte character `=C3=A4` + (lambda (_) + (with-current-buffer sieve-buffer + (goto-char (point-min)) + (insert (mk-rsp-getscript-ok script))))) + ((symbol-function 'accept-process-output) + (lambda (&optional _ _ _ _) nil)) + ((symbol-function 'get-buffer-process) (lambda (_) nil))) + ;; extract sieve script from sieve-buffer and put it into + ;; output-buffer + (sieve-manage-getscript script-name output-buffer sieve-buffer) + ;; extract script from output-buffer and return it as a string + (let ((script (with-current-buffer output-buffer + (set-buffer-modified-p nil) + (buffer-string)))) + ;; cleanup + (kill-buffer sieve-buffer) + (kill-buffer output-buffer) + (when (get-buffer sieve-manage-log) + (kill-buffer sieve-manage-log)) + ;; return script + script)))) + +(ert-deftest ert/managesieve-getscript-multibyte () + (should (string=3D sieve-script-multibyte-unix + (managesieve-getscript sieve-script-multibyte-unix)))) + + +;;; sieve-manage-tests.el ends here --=20 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0004-Some-minor-fixes-in-lisp-net-sieve.el.patch >From 4dde8d5a544c94ebc3f6001e46dca484d3632902 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 04:06:22 +0100 Subject: [PATCH 4/4] Some minor fixes in lisp/net/sieve.el * lisp/net/sieve.el (sieve-next-line) (sieve-prev-line): Handle situations where point in `sieve-buffer' is either before or after the list of server side scripts. (sieve-server-script-list): New variable. (sieve-refresh-scriptlist) (sieve-upload): Use `sieve-server-script-list' to make sure that local list of server side scripts in`sieve-buffer' is up-to-date after uploading a (new) script. (sieve-edit-script): Make sure that the newly created user facing sieve-manage buffer is using 'utf-8-unix encoding. --- lisp/net/sieve.el | 40 +++++++++++++++++++++++++++------------- 1 file changed, 27 insertions(+), 13 deletions(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index fbd07dee27c..1e5aae7825c 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -95,6 +95,8 @@ sieve-template (defvar sieve-manage-buffer nil) (defvar sieve-buffer-header-end nil) +(defvar sieve-server-script-list nil + "Current list of server-side sieve scripts.") (defvar sieve-buffer-script-name nil "The real script name of the buffer.") (make-local-variable 'sieve-buffer-script-name) @@ -204,6 +206,8 @@ sieve-edit-script (if (not (string-equal name sieve-new-script)) (let ((newbuf (generate-new-buffer name)) err) + (with-current-buffer newbuf + (set-buffer-file-coding-system sieve-manage--coding-system)) (setq err (sieve-manage-getscript name newbuf sieve-manage-buffer)) (switch-to-buffer newbuf) (if (sieve-manage-ok-p err) @@ -235,23 +239,28 @@ sieve-next-line (interactive) (unless arg (setq arg 1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "End of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "End of list")) + (goto-char (next-overlay-change sieve-buffer-header-end)))) (defun sieve-prev-line (&optional arg) (interactive) (unless arg (setq arg -1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "Beginning of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "Beginning of list")) + (goto-char (previous-overlay-change (point-max))) + (beginning-of-line))) (defun sieve-help () "Display help for various sieve commands." @@ -319,6 +328,9 @@ sieve-refresh-scriptlist (let* ((scripts (sieve-manage-listscripts sieve-manage-buffer)) (count (length scripts)) (keys (substitute-command-keys "\\[sieve-edit-script]"))) + (setq sieve-server-script-list + (mapcar (lambda (elt) (if (consp elt) (cdr elt) elt)) + scripts)) (insert (if (null scripts) (format @@ -361,7 +373,9 @@ sieve-upload (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." - (substitute-command-keys "\\[sieve-manage]")))) + (substitute-command-keys "\\[sieve-manage]")) + (when (not (member script-name sieve-server-script-list)) + (sieve-refresh-scriptlist)))) (set-buffer-modified-p nil)))) ;;;###autoload -- 2.39.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 19 08:22:51 2023 Received: (at 54154) by debbugs.gnu.org; 19 Jan 2023 13:22:51 +0000 Received: from localhost ([127.0.0.1]:42883 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIUsU-00022L-Ue for submit@debbugs.gnu.org; Thu, 19 Jan 2023 08:22:51 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:57297) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIUsQ-000226-Mv for 54154@debbugs.gnu.org; Thu, 19 Jan 2023 08:22:49 -0500 Received: from mail.tetzco.de (ipb21a9472.dynamic.kabel-deutschland.de [178.26.148.114]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id B634EBD078; Thu, 19 Jan 2023 14:22:40 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id D03986C00B2; Thu, 19 Jan 2023 14:42:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674135752; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=piaygrErYiQj7+9i8+FB+j4zUAYhiub1rhkKpeD4/FE=; b=RsBwGz4j7PJwhE9Vh3gyWqTaII2ICblf3OCz9fyceUOPO4Hi6sqsKJqtRbX/TbxxvUetR1 ujznpi3EG3nmcG32DlQUjf3cby+MTHF2tKNM78KZuPPUetZhBnO7a3SX7JMzMtpZsWbEqa 2Au9f5rsvDCg+1oXRapQoADzlLJUFB27J95zoar0wKeYR5vu/GZZ+gPG2quDXX+bDFHdt9 40ddxLObniFB/8SNMl6p2xKr8Ig5nM0owT2CIpxjZd8eByciOf0vLCGJfjJvIpJx/rlGXC 2WvujfBVy+qEGYkCLk7LbLYOVCy8kTRq+x0nVR9nYBiEx35OMMJB9Gm2HTCVb/MtEUyQyT yKu9PTTqq48dXHYfc5KvQV0fMxCRmgixvtvavgOP9agc1QeK+KPsgG2BkdC8keS7/DQ8cW b2CVq9mpj5yLxrsfGlZMnCwKBsm6Hr2jFkPNCBrvXi7RZqesbV4JsNjBDKu48L+V7ryrLo RcsRKVI7eGSAWhj8yirw0kyk8IkZwFJRmiE9RUdG/G1Zhcl3fxU17F/2g4jcQQNN0m3RH7 39LapoC/UTxkaQ9d8PllclsFbX1qagIeNlXwMol+WVNJoFbD16lJHqkEeGfslwHFH3oyM2 J7D0HMoxVbs2YNxP/FieLd7AdCckNUrppVF3sx5f1sqAEJSyuvuYE= From: Kai Tetzlaff To: Eli Zaretskii Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: <83tu0nynla.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 19 Jan 2023 09:45:21 +0200") References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> Date: Thu, 19 Jan 2023 14:22:39 +0100 Message-ID: <87r0vqbqw0.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: D03986C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-5.85 / 30.00]; NEURAL_HAM(-3.00)[-1.000]; BAYES_HAM(-2.75)[98.91%]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_FROM(0.00)[bug]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hmm, my previous mail with the updated patches was sent to early. The changes I made now cause issues when actually downloading a script from the server. So ignore my last message for now. I will send an updated version. From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 19 09:09:12 2023 Received: (at 54154) by debbugs.gnu.org; 19 Jan 2023 14:09:13 +0000 Received: from localhost ([127.0.0.1]:42920 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIVbM-0003D0-8S for submit@debbugs.gnu.org; Thu, 19 Jan 2023 09:09:12 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59230) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIVbJ-0003Co-Tr for 54154@debbugs.gnu.org; Thu, 19 Jan 2023 09:09:10 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIVbC-0008CJ-UR; Thu, 19 Jan 2023 09:09:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=ktvviMzlPCzmF5VmiJa//QQDmjFrmoJcAV9fL3VFYPs=; b=I7bDLxgXcoQU lPgpbFIeg77eziJhYZR7lptkdnGS1vIcR4cFsjU5BfTwWpWNoPUaTvkkCffx2apSt1zMxKXS8u3/1 n6NuBJCcMwHllmPipjTj0f5Wxb8XgF39YsSc7FkiMgTNIYjC/o6mJQTIXFLvUYDUdJcDwQJOQSiqy 0SCEt+lxLVvj8iEEN6yahkMCgT8EDO/5lGfO8/UFaFXc4mEhBLFIal7qIkbOcZC8GdDiwdaRZfirb dHxn4DyRjrWBl73oAzeQAxHYCQEttZh1R4RR55PbzR5AXm26paE5bx7UAI5da5Lt/VyTSNWqCIaks 9hwz+KRZKlS14CA7fWB+pA==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIVbC-0007X4-0Y; Thu, 19 Jan 2023 09:09:02 -0500 Date: Thu, 19 Jan 2023 16:08:58 +0200 Message-Id: <834jsmpqf9.fsf@gnu.org> From: Eli Zaretskii To: Kai Tetzlaff In-Reply-To: <873586d7ii.fsf@tetzco.de> (message from Kai Tetzlaff on Thu, 19 Jan 2023 13:38:13 +0100) Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Kai Tetzlaff > Cc: herbert@gojira.at, larsi@gnus.org, 54154@debbugs.gnu.org > Date: Thu, 19 Jan 2023 13:38:13 +0100 > > Eli Zaretskii writes: > > The duplicate use of with-current-buffer is sub-optimal, > > IMO. What about the simpler code below: > > > > (when sieve-manage-log > > (let* ((existing-log-buffer (get-buffer sieve-manage-log)) > > (log-buffer (or existing-log-buffer > > (get-buffer-create sieve-manage-log)))) > > (with-current-buffer log-buffer > > (unless existing-log-buffer > > ;; Do this only once, when creating the log buffer. > > (set-buffer-multibyte nil) > > (buffer-disable-undo)) > > (goto-char (point-max)) > > (apply #'insert args))))) > > Yes, that provides more insight into what the code intends to do. Here's > the patch (with additional updated doc string): Thanks, installed on the emacs-29 branch. > > This still uses a less-than-elegant implementation that calls > > with-current-buffer twice. > > Yes, true. But since `sieve-manage--set-internal-buffer-properties' is > used in two different places, the more elegant solution you suggested > above would require duplicating the body of the function in those > places. I just didn't see a better way. I'm not sure why you need to force the encoding of the process buffer, when you already set the coding-system to be used for decoding stuff from the process. Is that really needed? But if you really need this, then just make the insertion of the text into the buffer you create optional: then for the process-buffer pass nil as the text to insert, and you can do the with-current-buffer dance only inside that function. > >> (defun sieve-manage-encode (utf8-string) > >> "Convert UTF8-STRING to managesieve protocol octets." > >> - (encode-coding-string utf8-string 'raw-text t)) > >> + (encode-coding-string utf8-string sieve-manage--coding-system t)) > > > > Why is the argument called utf8-string? If it's indeed a string > > encoded in UTF-8, why do you need to encode it again with > > raw-text-unix? it should be a no-op in that case. So please tell more > > about the underlying issue. > > I chose the name as a hint to the user that the incoming string should > be UTF-8 encoded. But that is probably misleading since the string > itself doesn't have an encoding? So let's change the function to: > > (defun sieve-manage-encode (str) > "Convert STR to managesieve protocol octets." > (encode-coding-string str sieve-manage--coding-system t)) > > Regarding the potential double encoding: When sending data over the > network connection, `sieve-manage-encode' intends to make sure that > `utf8-string' data is converted to a byte/octet representation. I tried > to explain that in the doc string of `sieve-manage--coding-system': > > (defconst sieve-manage--coding-system 'raw-text-unix > "Use 'raw-text-unix coding system for (network) communication. > > Sets the coding system used for the internal (process, log) > buffers and the network stream created to communicate with the > managesieve server. Using 'raw-text encoding enables unibyte > mode and makes sure that sent/received octets (bytes) remain > untouched by the coding system. The explicit use of `-unix` > avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") intact).") > > The original problem was that when communicating with the sievemanage > server, we need to handle length elements where we need make sure that > calculated values take into account that UTF-8 characters may comprise > multiple octets. > > Even after reading the relevant sections of the documentation multiple > times I was (and am still) not sure what exactly the various coding > system settings do and how they interact with buffers and networking > functions. So forgive me if what I'm doing there looks weird to your > expert eyes. > > When working on the original patch, I had several uhoh moments where > data sent to or received from the network seemed to have been > automatically modified by the coding system (unfortunately, I don't > remember the exact details). So I tried to eliminate any such automatic > modifications by using 'binary or 'raw-text encodings on code paths > which handle network data. Basically, my thinking was: 'better do things > twice/thrice/... before introducing new points of failure'. Since you seem to be encoding and decoding to/from UTF-8 by hand in sieve-manage-encode/decode, you should use 'binary' as the process-codings-system for the network connection to the server, and that's it. I see no reason to encode again using raw-text-unix. What you should do is call sieve-manage-encode inside sieve-manage-send, and count the bytes there after encoding the payload. > >> @@ -244,8 +267,7 @@ sieve-manage-open-server > >> (open-network-stream > >> "SIEVE" buffer server port > >> :type stream > >> - ;; eol type unix is required to preserve "\r\n" > >> - :coding 'raw-text-unix > >> + :coding `(binary . ,sieve-manage--coding-system) > > > > Same question about encoding with raw-text-unix here: using it means > > some other code will need to encode the text with a real encoding, > > which in this case is UTF-8 (AFAIU the managesieve protocol RFC). So > > why not use utf-8-unix here? > > Same as above: I'm just not sure that this is the right thing. See above. > But after thinking about it some more, I made the following changes (as > an experiment): > > 1. set `sieve-manage--coding-system' to 'utf-8-unix and > 2. changed the call to `open-network-stream' in the patch above to > >> @@ -244,8 +267,7 @@ sieve-manage-open-server > >> (open-network-stream > >> "SIEVE" buffer server port > >> :type stream > >> - ;; eol type unix is required to preserve "\r\n" > >> - :coding 'raw-text-unix > >> + :coding sieve-manage--coding-system > instead of the previous, asymmetric mix of `binary and > `sieve-manage--coding-system'. This could work, but AFAIU, you need to specify the content length in bytes for the PUTSCRIPT command, so you must encode the content yourself. Thus my suggestion to use 'binary' in the :coding attribute of the process, and instead encode/decode using sieve-manage-encode/decode to/from UTF-8 inside sieve-manage-send and sieve-manage-parse-* functions. > > Should the addition of BYE support be mentioned in NEWS? > > I can certainly do that if you think that this is useful. It just seems > to be more of an internal detail which probably doesn't mean much to > most users. Isn't BYE provides some capabilities that user/callers would like to know about? From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 19 09:16:29 2023 Received: (at 54154) by debbugs.gnu.org; 19 Jan 2023 14:16:29 +0000 Received: from localhost ([127.0.0.1]:42944 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIViO-0003PW-8E for submit@debbugs.gnu.org; Thu, 19 Jan 2023 09:16:29 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:43219) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIViK-0003Ox-Gy for 54154@debbugs.gnu.org; Thu, 19 Jan 2023 09:16:26 -0500 Received: from mail.tetzco.de (unknown [IPv6:2a02:810d:1380:7900::9]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id AAD06BD078; Thu, 19 Jan 2023 15:16:18 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id ECDF86C00B2; Thu, 19 Jan 2023 15:36:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674138970; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=+mzs+WLC2QhRGO/+r5+onbP56ir44h9frquuuKWTZs0=; b=udiHIC4oanNrDHZquDt/8KIzlJR+GFiLviSKf+dGfP27VnOd7R2yGLzWVfCtquhODHlwOB 8PEu5SVwbcuYi+s3t8LsKzZlO8h4Yp5DmBojN6XMhFW/aLTghN696YzyEhXUPkU6SR4uKE mj8JNtbOPPFFFpxAHZyrXF4QZLHJ3+857YGnry5Ae6qgtqknE7O4FTLCeYAJlvbDUR0clA kY9gA1u8dSzn0O7TPgeyQePrNvzs822sBZ6DJAm87QyEXm9W8K8wS492kYwe2SHQE0SDH9 DNMqo1HCl5GDivFuNr7GB5sMuty3AN+9UTLNmqYl/grQQb9CgZqBpdY9WJBGNF+UIV1OXL Qs6tD4HtSg76tt5fYDzoUV1Ny4ieNR+KHF80LU60nsWkn9ifisZj4Z+bYxK0uM38z5Ur3H 8CL9bBmgHQU+vqUt7OczD61OXkVMW3aDaAGg0klLIcCil781FdF7kmwVWA+yOqI0l5bXp1 NKFIz5YLpAxSDxioZQxqQLJap+RG2iyXvuu5trh07oDpYfUShaXHQ0Y0bDAY4Be7+E7cU+ 2f49z7DR4G57hTqFENXtdZRO5rQrz1CXaaYtHxjIFpjlGSHCjPJHPMctwxFRibtETcf16V L3rVH+dU4/oxgDvlWse6641dJQoHyvElEwelu+gkK7EsBj+qjcQbs= From: Kai Tetzlaff To: Eli Zaretskii Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: <83tu0nynla.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 19 Jan 2023 09:45:21 +0200") References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> Date: Thu, 19 Jan 2023 15:16:16 +0100 Message-ID: <87h6wmboen.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: ECDF86C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.07 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-2.97)[-0.991]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:+,6:+,7:+]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; TAGGED_FROM(0.00)[bug]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; HAS_ATTACHMENT(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Here's the update. As a summary, this change: 2. changed the call to `open-network-stream' in the patch above to >> @@ -244,8 +267,7 @@ sieve-manage-open-server >> (open-network-stream >> "SIEVE" buffer server port >> :type stream >> - ;; eol type unix is required to preserve "\r\n" >> - :coding 'raw-text-unix >> + :coding sieve-manage--coding-system needed to be reverted to what I had earlier: >> @@ -244,8 +267,7 @@ sieve-manage-open-server >> (open-network-stream >> "SIEVE" buffer server port >> :type stream >> - ;; eol type unix is required to preserve "\r\n" >> - :coding 'raw-text-unix >> + :coding `(binary . ,sieve-manage--coding-system) I updated the text below and the patches accordingly. Hello Eli, thanks for looking into this! Eli Zaretskii writes: > The duplicate use of with-current-buffer is sub-optimal, > IMO. What about the simpler code below: > > (when sieve-manage-log > (let* ((existing-log-buffer (get-buffer sieve-manage-log)) > (log-buffer (or existing-log-buffer > (get-buffer-create sieve-manage-log)))) > (with-current-buffer log-buffer > (unless existing-log-buffer > ;; Do this only once, when creating the log buffer. > (set-buffer-multibyte nil) > (buffer-disable-undo)) > (goto-char (point-max)) > (apply #'insert args))))) Yes, that provides more insight into what the code intends to do. Here's the patch (with additional updated doc string): --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-Fix-bug-in-sieve-manage-append-to-log-emacs-29-only.patch Content-Description: fix sieve-manage--append-to-log for emacs-29 >From 62d03f302125c0b1aab2e3ae4f5b12b531d30d74 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:16:14 +0100 Subject: [PATCH] ; Fix bug in sieve-manage--append-to-log (emacs-29 only) This is emacs-29 only, use more elaborate fix for Emacs 30.x (master). * lisp/net/sieve-manage.el (sieve-manage--append-to-log): Fix log buffer creation. --- lisp/net/sieve-manage.el | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..4866f788bff 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -168,19 +168,25 @@ sieve-manage-capability ;; Internal utility functions (defun sieve-manage--append-to-log (&rest args) - "Append ARGS to sieve-manage log buffer. + "Append ARGS to `sieve-manage-log' buffer. ARGS can be a string or a list of strings. -The buffer to use for logging is specifified via -`sieve-manage-log'. If it is nil, logging is disabled." +The buffer to use for logging is specifified via `sieve-manage-log'. +If it is nil, logging is disabled. + +When the `sieve-manage-log' buffer doesn't exist, it gets created (and +configured with some initial settings)." (when sieve-manage-log - (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) - (set-buffer-multibyte nil) - (buffer-disable-undo))) - (goto-char (point-max)) - (apply #'insert args)))) + (let* ((existing-log-buffer (get-buffer sieve-manage-log)) + (log-buffer (or existing-log-buffer + (get-buffer-create sieve-manage-log)))) + (with-current-buffer log-buffer + (unless existing-log-buffer + ;; Do this only once, when creating the log buffer. + (set-buffer-multibyte nil) + (buffer-disable-undo)) + (goto-char (point-max)) + (apply #'insert args))))) (defun sieve-manage--message (format-string &rest args) "Wrapper around `message' which also logs to sieve manage log. -- 2.39.0 --=-=-= Content-Type: text/plain >> ;; Internal utility functions >> +(defun sieve-manage--set-internal-buffer-properties (buffer) >> + "Set BUFFER properties for internally used buffers. >> + >> +Used for process and log buffers, this function makes sure that >> +those buffers keep received and sent data intact by: >> +- setting the coding system to 'sieve-manage--coding-system', >> +- setting `after-change-functions' to nil to avoid those >> + functions messing with buffer content. >> +Also disables undo (to save a bit of memory and improve >> +performance). >> + >> +Returns BUFFER." >> + (with-current-buffer buffer >> + (set-buffer-file-coding-system sieve-manage--coding-system) >> + (setq-local after-change-functions nil) >> + (buffer-disable-undo) >> + (current-buffer))) >> + >> (defun sieve-manage--append-to-log (&rest args) >> "Append ARGS to sieve-manage log buffer. >> >> @@ -175,10 +202,8 @@ sieve-manage--append-to-log >> `sieve-manage-log'. If it is nil, logging is disabled." >> (when sieve-manage-log >> (with-current-buffer (or (get-buffer sieve-manage-log) >> - (with-current-buffer >> - (get-buffer-create sieve-manage-log) >> - (set-buffer-multibyte nil) >> - (buffer-disable-undo))) >> + (sieve-manage--set-internal-buffer-properties >> + (get-buffer-create sieve-manage-log))) >> (goto-char (point-max)) >> (apply #'insert args)))) > > This still uses a less-than-elegant implementation that calls > with-current-buffer twice. Yes, true. But since `sieve-manage--set-internal-buffer-properties' is used in two different places, the more elegant solution you suggested above would require duplicating the body of the function in those places. I just didn't see a better way. In particular because `set-buffer-file-coding-system' and `setq-local' only work with (current-buffer). If you can point me to code which can replace these with something that takes BUFFER arguments, I can rewrite `sieve-manage--set-internal-buffer-properties' and avoid using `with-current-buffer'. >> (defun sieve-manage-encode (utf8-string) >> "Convert UTF8-STRING to managesieve protocol octets." >> - (encode-coding-string utf8-string 'raw-text t)) >> + (encode-coding-string utf8-string sieve-manage--coding-system t)) > > Why is the argument called utf8-string? If it's indeed a string > encoded in UTF-8, why do you need to encode it again with > raw-text-unix? it should be a no-op in that case. So please tell more > about the underlying issue. I chose the name as a hint to the user that the incoming string should be UTF-8 encoded. But that is probably misleading since the string itself doesn't have an encoding? So let's change the function to: (defun sieve-manage-encode (str) "Convert STR to managesieve protocol octets." (encode-coding-string str sieve-manage--coding-system t)) Regarding the potential double encoding: When sending data over the network connection, `sieve-manage-encode' intends to make sure that `utf8-string' data is converted to a byte/octet representation. I tried to explain that in the doc string of `sieve-manage--coding-system': (defconst sieve-manage--coding-system 'raw-text-unix "Use 'raw-text-unix coding system for (network) communication. Sets the coding system used for the internal (process, log) buffers and the network stream created to communicate with the managesieve server. Using 'raw-text encoding enables unibyte mode and makes sure that sent/received octets (bytes) remain untouched by the coding system. The explicit use of `-unix` avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") intact).") The original problem was that when communicating with the sievemanage server, we need to handle length elements where we need make sure that calculated values take into account that UTF-8 characters may comprise multiple octets. Even after reading the relevant sections of the documentation multiple times I was (and am still) not sure what exactly the various coding system settings do and how they interact with buffers and networking functions. So forgive me if what I'm doing there looks weird to your expert eyes. When working on the original patch, I had several uhoh moments where data sent to or received from the network seemed to have been automatically modified by the coding system (unfortunately, I don't remember the exact details). So I tried to eliminate any such automatic modifications by using 'binary or 'raw-text encodings on code paths which handle network data. Basically, my thinking was: 'better do things twice/thrice/... before introducing new points of failure'. >> @@ -244,8 +267,7 @@ sieve-manage-open-server >> (open-network-stream >> "SIEVE" buffer server port >> :type stream >> - ;; eol type unix is required to preserve "\r\n" >> - :coding 'raw-text-unix >> + :coding `(binary . ,sieve-manage--coding-system) > > Same question about encoding with raw-text-unix here: using it means > some other code will need to encode the text with a real encoding, > which in this case is UTF-8 (AFAIU the managesieve protocol RFC). So > why not use utf-8-unix here? Same as above: I'm just not sure that this is the right thing. But after thinking about it some more, I made the following change (as an experiment): 1. set `sieve-manage--coding-system' to 'utf-8-unix (and update the doc string) With this change, there are still no issues when connecting to my managesieve server which still contains a script with a name that contains utf-8 multibyte characters. Also, the test I wrote still works with that change. So if you think that this makes things clearer, I'm happy to make the change. I just don't feel confident enough to do this without additional guidance. I was also experimenting with some additional changes with the hope to to just use coding system settings instead of calling `sieve-manage-encode'/`sieve-manage-decode'. But I couldn't get that to work. I added an updated set of Emacs 30.x patches with the changes described above (plus an additional change in sieve.el which makes sure that the sieve-manage buffer containing the list of available sieve scripts is also using coding system 'utf-8-unix from `sieve-manage--coding-system'). > Should the addition of BYE support be mentioned in NEWS? I can certainly do that if you think that this is useful. It just seems to be more of an internal detail which probably doesn't mean much to most users. > On balance, I think the additional patches should go to master, > indeed. But let's resolve the issues mentioned above first. Ok, awaiting further input... --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Fix-bug-in-sieve-manage-append-to-log-improve-sieve-.patch >From 977734f16874636c4f2f5e3bb41a86e4338247c4 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:08:07 +0100 Subject: [PATCH 1/4] Fix bug in sieve-manage--append-to-log, improve sieve-manage buffer config Also update/add some comments/docs. * lisp/net/sieve-manage.el (sieve-manage-coding-system-for-read) (sieve-manage-coding-system-for-write): Remove unused constants. (sieve-manage-encode): Change misleading argument name (utf8-string -> str). (sieve-manage--coding-system): New constant. (sieve-manage--set-internal-buffer-properties): New function. (sieve-manage-open-server): Use `sieve-manage--coding-system'. (sieve-manage--append-to-log): Use `sieve-manage--set-internal-buffer-properties' to fix log buffer creation. (sieve-manage-encode) (sieve-manage-make-process-buffer): Use `sieve-manage--set-internal-buffer-properties'. --- lisp/net/sieve-manage.el | 65 +++++++++++++++++++++++++++------------- 1 file changed, 44 insertions(+), 21 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..adfecc7b309 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -58,7 +58,7 @@ ;; ;; References: ;; -;; draft-martin-managesieve-02.txt, +;; RFC5804, ;; "A Protocol for Remotely Managing Sieve Scripts", ;; by Tim Martin. ;; @@ -145,6 +145,15 @@ sieve-manage-ignore-starttls :type 'boolean) ;; Internal variables: +(defconst sieve-manage--coding-system 'utf-8-unix + "Use \\='utf-8-unix coding system for (network) communication. + +Defines the coding system used for the internal (process, log) +buffers and the network stream created to communicate with the +managesieve server. Using \\='utf-8-unix encoding corresponds to +the use of UTF-8 in RFC5804 (managesieve). The explicit use of +\\='-unix\\=' avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") +sequences intact).") (defconst sieve-manage-local-variables '(sieve-manage-server sieve-manage-port @@ -154,8 +163,6 @@ sieve-manage-local-variables sieve-manage-client-eol sieve-manage-server-eol sieve-manage-capability)) -(defconst sieve-manage-coding-system-for-read 'binary) -(defconst sieve-manage-coding-system-for-write 'binary) (defvar sieve-manage-stream nil) (defvar sieve-manage-auth nil) (defvar sieve-manage-server nil) @@ -167,18 +174,37 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions +(defun sieve-manage--set-internal-buffer-properties (buffer) + "Set BUFFER properties for internally used buffers. + +Used for process and log buffers, this function makes sure that +those buffers keep received and sent data intact by: +- setting the coding system to 'sieve-manage--coding-system', +- setting `after-change-functions' to nil to avoid those + functions messing with buffer content. +Also disables undo (to save a bit of memory and improve +performance). + +Returns BUFFER." + (with-current-buffer buffer + (set-buffer-file-coding-system sieve-manage--coding-system) + (setq-local after-change-functions nil) + (buffer-disable-undo) + (current-buffer))) + (defun sieve-manage--append-to-log (&rest args) - "Append ARGS to sieve-manage log buffer. + "Append ARGS to `sieve-manage-log' buffer. ARGS can be a string or a list of strings. -The buffer to use for logging is specifified via -`sieve-manage-log'. If it is nil, logging is disabled." +The buffer to use for logging is specifified via `sieve-manage-log'. +If it is nil, logging is disabled. + +When the `sieve-manage-log' buffer doesn't exist, it gets created (and +configured with some initial settings)." (when sieve-manage-log (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) - (set-buffer-multibyte nil) - (buffer-disable-undo))) + (sieve-manage--set-internal-buffer-properties + (get-buffer-create sieve-manage-log))) (goto-char (point-max)) (apply #'insert args)))) @@ -202,9 +228,9 @@ sieve-manage--error (sieve-manage--append-to-log msg "\n") (error msg))) -(defun sieve-manage-encode (utf8-string) - "Convert UTF8-STRING to managesieve protocol octets." - (encode-coding-string utf8-string 'raw-text t)) +(defun sieve-manage-encode (str) + "Convert STR to managesieve protocol octets." + (encode-coding-string str sieve-manage--coding-system t)) (defun sieve-manage-decode (octets &optional buffer) "Convert managesieve protocol OCTETS to utf-8 string. @@ -216,13 +242,11 @@ sieve-manage-decode (defun sieve-manage-make-process-buffer () (with-current-buffer - (generate-new-buffer (format " *sieve %s:%s*" - sieve-manage-server - sieve-manage-port)) + (sieve-manage--set-internal-buffer-properties + (generate-new-buffer (format " *sieve %s:%s*" + sieve-manage-server + sieve-manage-port))) (mapc #'make-local-variable sieve-manage-local-variables) - (set-buffer-multibyte nil) - (setq-local after-change-functions nil) - (buffer-disable-undo) (current-buffer))) (defun sieve-manage-erase (&optional p buffer) @@ -244,8 +268,7 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream - ;; eol type unix is required to preserve "\r\n" - :coding 'raw-text-unix + :coding `(binary . ,sieve-manage--coding-system) :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-Handle-BYE-in-sieve-manage-server-responses.patch >From 55e3dfae66be5df73aff96fccd8dc5704a2c5dc5 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:52:43 +0100 Subject: [PATCH 2/4] Handle BYE in sieve-manage server responses * lisp/net/sieve-manage.el (sieve-manage-regex-oknobye): New function. (sieve-manage-parse-oknobye): Renamed from `sieve-manage-parse-okno'. (sieve-manage-open-server) (sieve-sasl-auth) (sieve-manage-listscripts) (sieve-manage-putscript) (sieve-manage-deletescript) (sieve-manage-getscript) (sieve-manage-setactive) (sieve-manage-is-okno) (sieve-manage-parse-oknobye) (sieve-manage-parse-parse-string) (sieve-manage-parse-parse-crlf): Use `sieve-manage-regex-oknobye' to handle BYE in addition to OK and NO. --- lisp/net/sieve-manage.el | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index adfecc7b309..ed3dcef8d04 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -270,8 +270,8 @@ sieve-manage-open-server :type stream :coding `(binary . ,sieve-manage--coding-system) :capability-command "CAPABILITY\r\n" - :end-of-command "^\\(OK\\|NO\\).*\n" - :success "^OK.*\n" + :end-of-command "^\\(OK\\|NO\\|BYE\\).*\r\n" + :success "^OK.*\r\n" :return-list t :starttls-function (lambda (capabilities) @@ -330,7 +330,7 @@ sieve-sasl-auth (setq rsp nil) (goto-char (match-end 0)) rsp)) - (setq rsp (sieve-manage-is-okno)))) + (setq rsp (sieve-manage-is-oknobye)))) (accept-process-output sieve-manage-process 1) (goto-char (point-min))) (sieve-manage-erase) @@ -499,19 +499,19 @@ sieve-manage-listscripts (defun sieve-manage-havespace (name size &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name (length (sieve-manage-encode content)) sieve-manage-client-eol content)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "DELETESCRIPT \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -519,17 +519,22 @@ sieve-manage-getscript (sieve-manage-decode (sieve-manage-parse-string) output-buffer) (sieve-manage-parse-crlf) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "SETACTIVE \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) ;; Protocol parsing routines +(defun sieve-manage-regexp-oknobye () + "Return regexp for managesieve 'response-oknobye'." + (concat + "^\\(OK\\|NO\\|BYE\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" + sieve-manage-server-eol)) (defun sieve-manage-wait-for-answer () - (let ((pattern "^\\(OK\\|NO\\).*\n") + (let ((pattern (sieve-manage-regexp-oknobye)) pos) (while (not pos) (setq pos (search-forward-regexp pattern nil t)) @@ -547,10 +552,8 @@ sieve-manage-ok-p (defun sieve-manage-no-p (rsp) (string= (downcase (or (car-safe rsp) "")) "no")) -(defun sieve-manage-is-okno () - (when (looking-at (concat - "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" - sieve-manage-server-eol)) +(defun sieve-manage-is-oknobye () + (when (looking-at (sieve-manage-regexp-oknobye)) (let ((status (match-string 1)) (resp-code (match-string 3)) (response (match-string 5))) @@ -559,12 +562,12 @@ sieve-manage-is-okno (setq response (sieve-manage-is-string))) (list status resp-code response)))) -(defun sieve-manage-parse-okno () +(defun sieve-manage-parse-oknobye () (let (rsp) (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-okno))) + (setq rsp (sieve-manage-is-oknobye))) (sieve-manage-erase) rsp)) @@ -598,7 +601,7 @@ sieve-manage-parse-string (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) (unless (setq rsp (sieve-manage-is-string)) - (when (sieve-manage-no-p (sieve-manage-is-okno)) + (when (sieve-manage-no-p (sieve-manage-is-oknobye)) ;; simple `error' is enough since `sieve-manage-erase' ;; already adds the server response to the log (error (sieve-manage-erase))))) @@ -612,7 +615,7 @@ sieve-manage-parse-crlf (defun sieve-manage-parse-listscripts () (let (tmp rsp data) (while (null rsp) - (while (null (or (setq rsp (sieve-manage-is-okno)) + (while (null (or (setq rsp (sieve-manage-is-oknobye)) (setq tmp (sieve-manage-decode (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) -- 2.39.0 --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0003-Add-test-lisp-net-sieve-manage-tests.el.patch Content-Transfer-Encoding: quoted-printable >From 786325d3adf7514967ae74c2db84c532f19dc070 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Tue, 22 Mar 2022 20:48:09 +0100 Subject: [PATCH 3/4] Add test/lisp/net/sieve-manage-tests.el * test/lisp/net/sieve-manage-tests.el: (mk-literal-s2c) (mk-rsp-oknobye) (mk-rsp-getscript-ok) (managesieve-getscript): Test helper functions. (ert/managesieve-getscript-multibyte): Test `sieve-manage-getscript' with script names containing multibyte characters. --- test/lisp/net/sieve-manage-tests.el | 106 ++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 test/lisp/net/sieve-manage-tests.el diff --git a/test/lisp/net/sieve-manage-tests.el b/test/lisp/net/sieve-mana= ge-tests.el new file mode 100644 index 00000000000..d4c616b3364 --- /dev/null +++ b/test/lisp/net/sieve-manage-tests.el @@ -0,0 +1,106 @@ +;;; sieve-manage-tests.el --- tests for sieve-manage.el -*- lexical-= binding: t; -*- + +;; Copyright (C) 2022 Free Software Foundation, Inc. + +;; Author: Kai Tetzlaff + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see . + +;;; Commentary: + + +;;; Code: + +(require 'ert) +(require 'sieve-manage) + +(defvar sieve-script-multibyte-unix + "if body :matches \"=C3=A4\" { stop; }\n" + "Simple multibyte sieve script.") + +(defun mk-literal-s2c (string) + "Encode STRING to managesieve 'literal-s2c'." + (let ((data (sieve-manage-encode string))) + (concat (format "{%d}\r\n" (length data)) + data))) + +(defun mk-rsp-oknobye (type &optional resp-code string) + "Encode TYPE, RESP-CODE and STRING to managesieve 'response-oknobye'." + (when (memq type '(OK NO BYE)) + (concat + (mapconcat #'identity + (delq nil + (list (symbol-name type) + (when resp-code + (format "(%s)" resp-code)) + (when string + (format "\"%s\"" + (sieve-manage-encode string))))) + " ") + "\r\n"))) +;; (mk-rsp-oknobye 'OK nil "Getscript completed.") +;; (mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") + +(defun mk-rsp-getscript-ok (script) + "Encode SCRIPT to managesieve 'response-getscript'." + (concat (mk-literal-s2c script) + "\r\n" + (mk-rsp-oknobye 'OK "Getscript completed."))) +;; (mk-rsp-getscript-ok sieve-script-multibyte-unix) + +(defun managesieve-getscript (script) + "Simulate managesieve getscript response to test +`sieve-manage-getscript' function." + (let* ((script-name "test.sieve") + ;; `sieve-manage-server' and `sieve-manage-port' are used in + ;; `sieve-manage-make-process-buffer' + (sieve-manage-server) + (sieve-manage-port "sieve") + (sieve-buffer (sieve-manage-make-process-buffer)) + (output-buffer (generate-new-buffer script-name))) + ;; use cl-letf to mock some functions in call chain of + ;; sieve-manage-getscript + (cl-letf (((symbol-function 'sieve-manage-send) + ;; simulate sieve server response with a single + ;; multibyte character `=C3=A4` + (lambda (_) + (with-current-buffer sieve-buffer + (goto-char (point-min)) + (insert (mk-rsp-getscript-ok script))))) + ((symbol-function 'accept-process-output) + (lambda (&optional _ _ _ _) nil)) + ((symbol-function 'get-buffer-process) (lambda (_) nil))) + ;; extract sieve script from sieve-buffer and put it into + ;; output-buffer + (sieve-manage-getscript script-name output-buffer sieve-buffer) + ;; extract script from output-buffer and return it as a string + (let ((script (with-current-buffer output-buffer + (set-buffer-modified-p nil) + (buffer-string)))) + ;; cleanup + (kill-buffer sieve-buffer) + (kill-buffer output-buffer) + (when (get-buffer sieve-manage-log) + (kill-buffer sieve-manage-log)) + ;; return script + script)))) + +(ert-deftest ert/managesieve-getscript-multibyte () + (should (string=3D sieve-script-multibyte-unix + (managesieve-getscript sieve-script-multibyte-unix)))) + + +;;; sieve-manage-tests.el ends here --=20 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0004-Some-minor-fixes-in-lisp-net-sieve.el.patch >From 6c63f891589e6525bf91ef53a0a8114a836a7781 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 04:06:22 +0100 Subject: [PATCH 4/4] Some minor fixes in lisp/net/sieve.el * lisp/net/sieve.el (sieve-next-line) (sieve-prev-line): Handle situations where point in `sieve-buffer' is either before or after the list of server side scripts. (sieve-server-script-list): New variable. (sieve-refresh-scriptlist) (sieve-upload): Use `sieve-server-script-list' to make sure that local list of server side scripts in`sieve-buffer' is up-to-date after uploading a (new) script. (sieve-edit-script): Make sure that the newly created user facing sieve-manage buffer is using 'utf-8-unix encoding. --- lisp/net/sieve.el | 40 +++++++++++++++++++++++++++------------- 1 file changed, 27 insertions(+), 13 deletions(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index fbd07dee27c..1e5aae7825c 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -95,6 +95,8 @@ sieve-template (defvar sieve-manage-buffer nil) (defvar sieve-buffer-header-end nil) +(defvar sieve-server-script-list nil + "Current list of server-side sieve scripts.") (defvar sieve-buffer-script-name nil "The real script name of the buffer.") (make-local-variable 'sieve-buffer-script-name) @@ -204,6 +206,8 @@ sieve-edit-script (if (not (string-equal name sieve-new-script)) (let ((newbuf (generate-new-buffer name)) err) + (with-current-buffer newbuf + (set-buffer-file-coding-system sieve-manage--coding-system)) (setq err (sieve-manage-getscript name newbuf sieve-manage-buffer)) (switch-to-buffer newbuf) (if (sieve-manage-ok-p err) @@ -235,23 +239,28 @@ sieve-next-line (interactive) (unless arg (setq arg 1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "End of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "End of list")) + (goto-char (next-overlay-change sieve-buffer-header-end)))) (defun sieve-prev-line (&optional arg) (interactive) (unless arg (setq arg -1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "Beginning of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "Beginning of list")) + (goto-char (previous-overlay-change (point-max))) + (beginning-of-line))) (defun sieve-help () "Display help for various sieve commands." @@ -319,6 +328,9 @@ sieve-refresh-scriptlist (let* ((scripts (sieve-manage-listscripts sieve-manage-buffer)) (count (length scripts)) (keys (substitute-command-keys "\\[sieve-edit-script]"))) + (setq sieve-server-script-list + (mapcar (lambda (elt) (if (consp elt) (cdr elt) elt)) + scripts)) (insert (if (null scripts) (format @@ -361,7 +373,9 @@ sieve-upload (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." - (substitute-command-keys "\\[sieve-manage]")))) + (substitute-command-keys "\\[sieve-manage]")) + (when (not (member script-name sieve-server-script-list)) + (sieve-refresh-scriptlist)))) (set-buffer-modified-p nil)))) ;;;###autoload -- 2.39.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 19 10:59:52 2023 Received: (at 54154) by debbugs.gnu.org; 19 Jan 2023 15:59:52 +0000 Received: from localhost ([127.0.0.1]:44504 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIXKQ-0006iS-Uh for submit@debbugs.gnu.org; Thu, 19 Jan 2023 10:59:52 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:44849) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIXKO-0006iF-Ll for 54154@debbugs.gnu.org; Thu, 19 Jan 2023 10:59:50 -0500 Received: from mail.tetzco.de (unknown [IPv6:2a02:810d:1380:7900::9]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id BC9EFBD078; Thu, 19 Jan 2023 16:59:41 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id B570B6C00B2; Thu, 19 Jan 2023 17:19:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674145169; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=opyxAiYS6r5yN+PKCZO0BXy5fXrMDLDPbKjZT3ZFGvc=; b=TPYTGDefl0fWulE47ovY3lKPKiKpQE2Cv/tYZoA3n9GmDP70Q42luxzi+M8cYvr7oSNLKj XWYpqLUWtFphm5dQ81FYxtTH7gsV7flUJy0lAvDxW1FCQRLebaCdVvv4+Wl8E4nqujnT+h vi6ymmHiwAQpfx7PgUydS/j06JiTaUYkHQaS1XM9zDGZv4CPt8WwnhOP5/Y8IbO47Ulusj OlAuTxMEpzYrkMCXvTdw4jI21pC/ofBzHc/lImd7+Uy/z4na87T6TgkFxeBppfUMBFaKPr nUaH1EaV8NiOvFRHCU/klenvn7TS0ZAVKg3mNx1r+YkSDlJg/DOwaP4FZFyKR6we1rZShS MDBHQgaCE+FwAhSZwR4Xo1B57Cw7lKrj10pOUDWXF5BzfwBDoA9WKxw20CZeTF8WN73KN6 PVem1mr+dmZL6AmDGwW/3w+mRjW13R816hkXafp74Ks60CXofNQbX/G3Yh5Tm2d/F2Unbi EXt3vhprLP59svJ3I9W5xbHBkAsNX3Y1J3NUuKDiSLJsrxqVVuP0N7gaLHnmf6tBWcoFtN DNGtP3ocK9EUC4+Clmj9Uw6NGPqKr2lKH1+Y950ZMI+iduyUJgj4AM2KcRKNAhlWyXOk/j pdc/QIKwwSREpbPMhb+Ks9+5jv3BDZR9itaNnyq4BWemE4FAGpMy8= From: Kai Tetzlaff To: Eli Zaretskii Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: <834jsmpqf9.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 19 Jan 2023 16:08:58 +0200") References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> Date: Thu, 19 Jan 2023 16:59:36 +0100 Message-ID: <87bkmua51z.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: B570B6C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.82 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-3.00)[-0.999]; GENERIC_REPUTATION(-0.73)[-0.7266375176278]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:+]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; ARC_NA(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_FROM(0.00)[bug]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Eli Zaretskii writes: >> > This still uses a less-than-elegant implementation that calls >> > with-current-buffer twice. >> >> Yes, true. But since `sieve-manage--set-internal-buffer-properties' is >> used in two different places, the more elegant solution you suggested >> above would require duplicating the body of the function in those >> places. I just didn't see a better way. > > I'm not sure why you need to force the encoding of the process buffer, > when you already set the coding-system to be used for decoding stuff > from the process. Is that really needed? Not sure if it is really needed. But I wanted to make sure that both, the process buffer and the log buffer use identical settings. Otherwise, the content of the log buffer might be misleading. > But if you really need this, then just make the insertion of the text > into the buffer you create optional: then for the process-buffer pass > nil as the text to insert, and you can do the with-current-buffer > dance only inside that function. Sorry, you lost me there. I don't understand what you want to tell me. Which (optional) text in which buffer? > Since you seem to be encoding and decoding to/from UTF-8 by hand in > sieve-manage-encode/decode, you should use 'binary' as the > process-codings-system for the network connection to the server, and > that's it. That works. Done. > What you should do is call sieve-manage-encode inside > sieve-manage-send, and count the bytes there after encoding the > payload. Unfortunately, that is too late since the sent data - in case that the sent text may contain CRLF sequences - contains its own length. So in order to insert the correct length, I need to encode before sending. See: (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name (length (sieve-manage-encode content)) sieve-manage-client-eol content)) (sieve-manage-parse-oknobye))) >> 1. set `sieve-manage--coding-system' to 'utf-8-unix and >> 2. changed the call to `open-network-stream' in the patch above to >> >> @@ -244,8 +267,7 @@ sieve-manage-open-server >> >> (open-network-stream >> >> "SIEVE" buffer server port >> >> :type stream >> >> - ;; eol type unix is required to preserve "\r\n" >> >> - :coding 'raw-text-unix >> >> + :coding sieve-manage--coding-system >> instead of the previous, asymmetric mix of `binary and >> `sieve-manage--coding-system'. > > This could work, but AFAIU, you need to specify the content length in > bytes for the PUTSCRIPT command, so you must encode the content > yourself. Thus my suggestion to use 'binary' in the :coding attribute > of the process, and instead encode/decode using > sieve-manage-encode/decode to/from UTF-8 inside sieve-manage-send and > sieve-manage-parse-* functions. Yes, that's what I explained above (before reading this part of your reply). Unfortunately, just using `sieve-manage--coding-system' for the :coding property didn't work, but I'm now using 'binary' encoding for both directions. >> > Should the addition of BYE support be mentioned in NEWS? >> >> I can certainly do that if you think that this is useful. It just seems >> to be more of an internal detail which probably doesn't mean much to >> most users. > > Isn't BYE provides some capabilities that user/callers would like to > know about? >From RFC5804: response-nobye = ("NO" / "BYE") [SP "(" resp-code ")"] [SP string] CRLF ;; The string contains human-readable text ;; encoded as UTF-8. As far as I understand, the difference between NO and BYE is that BYE is just a different (and more drastic, because the server will also disconnect) way of signalling an error. Fortunately, the human readable is typically included in these responses and will be shown to the user. Here is some more info about where BYE SHOULD (not MUST) be used: The BYE response SHOULD be used if the server wishes to close the connection. A server may wish to do this because the client was idle for too long or there were too many failed authentication attempts. This response can be issued at any time and should be immediately followed by a server hang-up of the connection. ... If I remember correctly, the timeout case was the reason why I added the BYE handling (since during my experiments, I sometimes used the debugger to understand what's going on which introduced long delays between connection establishment/authentication and sending the first request and resulted in a BYE instead of a NO). There's also one additional (more interesting) case: REFERRAL This response code may be returned with a BYE result from any command, and includes a mandatory parameter that indicates what server to access to manage this user's Sieve scripts. The server will be specified by a Sieve URL (see Section 3). The scriptname portion of the URL MUST NOT be specified. The client should authenticate to the specified server and use it for all further commands in the current session. However, even my updated sieve-manage code doesn't handle REFERRALs. So I still think that to understand the difference between a BYE and a NO would require the user to take a (deeper) dive into the RFC. But to avoid a longer discussion, how about: * Changes in Specialized Modes and Packages in Emacs 30.1 ... ** sieve-manage -- *** Added (partial) handling of BYE responses The managesieve client in `sieve-manage' now handles BYE responses sent be the server (in addition to OK and NO). This makes the implementation more robust in case of e.g. timeouts and authentication failures. Note: The special case of a REFERRAL/BYE responses is still not handled by the client (s. RFC5804 for more details). --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Fix-bug-in-sieve-manage-append-to-log-improve-sieve-.patch >From 977734f16874636c4f2f5e3bb41a86e4338247c4 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:08:07 +0100 Subject: [PATCH 1/4] Fix bug in sieve-manage--append-to-log, improve sieve-manage buffer config Also update/add some comments/docs. * lisp/net/sieve-manage.el (sieve-manage-coding-system-for-read) (sieve-manage-coding-system-for-write): Remove unused constants. (sieve-manage-encode): Change misleading argument name (utf8-string -> str). (sieve-manage--coding-system): New constant. (sieve-manage--set-internal-buffer-properties): New function. (sieve-manage-open-server): Use `sieve-manage--coding-system'. (sieve-manage--append-to-log): Use `sieve-manage--set-internal-buffer-properties' to fix log buffer creation. (sieve-manage-encode) (sieve-manage-make-process-buffer): Use `sieve-manage--set-internal-buffer-properties'. --- lisp/net/sieve-manage.el | 65 +++++++++++++++++++++++++++------------- 1 file changed, 44 insertions(+), 21 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..adfecc7b309 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -58,7 +58,7 @@ ;; ;; References: ;; -;; draft-martin-managesieve-02.txt, +;; RFC5804, ;; "A Protocol for Remotely Managing Sieve Scripts", ;; by Tim Martin. ;; @@ -145,6 +145,15 @@ sieve-manage-ignore-starttls :type 'boolean) ;; Internal variables: +(defconst sieve-manage--coding-system 'utf-8-unix + "Use \\='utf-8-unix coding system for (network) communication. + +Defines the coding system used for the internal (process, log) +buffers and the network stream created to communicate with the +managesieve server. Using \\='utf-8-unix encoding corresponds to +the use of UTF-8 in RFC5804 (managesieve). The explicit use of +\\='-unix\\=' avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") +sequences intact).") (defconst sieve-manage-local-variables '(sieve-manage-server sieve-manage-port @@ -154,8 +163,6 @@ sieve-manage-local-variables sieve-manage-client-eol sieve-manage-server-eol sieve-manage-capability)) -(defconst sieve-manage-coding-system-for-read 'binary) -(defconst sieve-manage-coding-system-for-write 'binary) (defvar sieve-manage-stream nil) (defvar sieve-manage-auth nil) (defvar sieve-manage-server nil) @@ -167,18 +174,37 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions +(defun sieve-manage--set-internal-buffer-properties (buffer) + "Set BUFFER properties for internally used buffers. + +Used for process and log buffers, this function makes sure that +those buffers keep received and sent data intact by: +- setting the coding system to 'sieve-manage--coding-system', +- setting `after-change-functions' to nil to avoid those + functions messing with buffer content. +Also disables undo (to save a bit of memory and improve +performance). + +Returns BUFFER." + (with-current-buffer buffer + (set-buffer-file-coding-system sieve-manage--coding-system) + (setq-local after-change-functions nil) + (buffer-disable-undo) + (current-buffer))) + (defun sieve-manage--append-to-log (&rest args) - "Append ARGS to sieve-manage log buffer. + "Append ARGS to `sieve-manage-log' buffer. ARGS can be a string or a list of strings. -The buffer to use for logging is specifified via -`sieve-manage-log'. If it is nil, logging is disabled." +The buffer to use for logging is specifified via `sieve-manage-log'. +If it is nil, logging is disabled. + +When the `sieve-manage-log' buffer doesn't exist, it gets created (and +configured with some initial settings)." (when sieve-manage-log (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) - (set-buffer-multibyte nil) - (buffer-disable-undo))) + (sieve-manage--set-internal-buffer-properties + (get-buffer-create sieve-manage-log))) (goto-char (point-max)) (apply #'insert args)))) @@ -202,9 +228,9 @@ sieve-manage--error (sieve-manage--append-to-log msg "\n") (error msg))) -(defun sieve-manage-encode (utf8-string) - "Convert UTF8-STRING to managesieve protocol octets." - (encode-coding-string utf8-string 'raw-text t)) +(defun sieve-manage-encode (str) + "Convert STR to managesieve protocol octets." + (encode-coding-string str sieve-manage--coding-system t)) (defun sieve-manage-decode (octets &optional buffer) "Convert managesieve protocol OCTETS to utf-8 string. @@ -216,13 +242,11 @@ sieve-manage-decode (defun sieve-manage-make-process-buffer () (with-current-buffer - (generate-new-buffer (format " *sieve %s:%s*" - sieve-manage-server - sieve-manage-port)) + (sieve-manage--set-internal-buffer-properties + (generate-new-buffer (format " *sieve %s:%s*" + sieve-manage-server + sieve-manage-port))) (mapc #'make-local-variable sieve-manage-local-variables) - (set-buffer-multibyte nil) - (setq-local after-change-functions nil) - (buffer-disable-undo) (current-buffer))) (defun sieve-manage-erase (&optional p buffer) @@ -244,8 +268,7 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream - ;; eol type unix is required to preserve "\r\n" - :coding 'raw-text-unix + :coding `(binary . ,sieve-manage--coding-system) :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-Handle-BYE-in-sieve-manage-server-responses.patch >From 55e3dfae66be5df73aff96fccd8dc5704a2c5dc5 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:52:43 +0100 Subject: [PATCH 2/4] Handle BYE in sieve-manage server responses * lisp/net/sieve-manage.el (sieve-manage-regex-oknobye): New function. (sieve-manage-parse-oknobye): Renamed from `sieve-manage-parse-okno'. (sieve-manage-open-server) (sieve-sasl-auth) (sieve-manage-listscripts) (sieve-manage-putscript) (sieve-manage-deletescript) (sieve-manage-getscript) (sieve-manage-setactive) (sieve-manage-is-okno) (sieve-manage-parse-oknobye) (sieve-manage-parse-parse-string) (sieve-manage-parse-parse-crlf): Use `sieve-manage-regex-oknobye' to handle BYE in addition to OK and NO. --- lisp/net/sieve-manage.el | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index adfecc7b309..ed3dcef8d04 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -270,8 +270,8 @@ sieve-manage-open-server :type stream :coding `(binary . ,sieve-manage--coding-system) :capability-command "CAPABILITY\r\n" - :end-of-command "^\\(OK\\|NO\\).*\n" - :success "^OK.*\n" + :end-of-command "^\\(OK\\|NO\\|BYE\\).*\r\n" + :success "^OK.*\r\n" :return-list t :starttls-function (lambda (capabilities) @@ -330,7 +330,7 @@ sieve-sasl-auth (setq rsp nil) (goto-char (match-end 0)) rsp)) - (setq rsp (sieve-manage-is-okno)))) + (setq rsp (sieve-manage-is-oknobye)))) (accept-process-output sieve-manage-process 1) (goto-char (point-min))) (sieve-manage-erase) @@ -499,19 +499,19 @@ sieve-manage-listscripts (defun sieve-manage-havespace (name size &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name (length (sieve-manage-encode content)) sieve-manage-client-eol content)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "DELETESCRIPT \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -519,17 +519,22 @@ sieve-manage-getscript (sieve-manage-decode (sieve-manage-parse-string) output-buffer) (sieve-manage-parse-crlf) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "SETACTIVE \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) ;; Protocol parsing routines +(defun sieve-manage-regexp-oknobye () + "Return regexp for managesieve 'response-oknobye'." + (concat + "^\\(OK\\|NO\\|BYE\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" + sieve-manage-server-eol)) (defun sieve-manage-wait-for-answer () - (let ((pattern "^\\(OK\\|NO\\).*\n") + (let ((pattern (sieve-manage-regexp-oknobye)) pos) (while (not pos) (setq pos (search-forward-regexp pattern nil t)) @@ -547,10 +552,8 @@ sieve-manage-ok-p (defun sieve-manage-no-p (rsp) (string= (downcase (or (car-safe rsp) "")) "no")) -(defun sieve-manage-is-okno () - (when (looking-at (concat - "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" - sieve-manage-server-eol)) +(defun sieve-manage-is-oknobye () + (when (looking-at (sieve-manage-regexp-oknobye)) (let ((status (match-string 1)) (resp-code (match-string 3)) (response (match-string 5))) @@ -559,12 +562,12 @@ sieve-manage-is-okno (setq response (sieve-manage-is-string))) (list status resp-code response)))) -(defun sieve-manage-parse-okno () +(defun sieve-manage-parse-oknobye () (let (rsp) (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-okno))) + (setq rsp (sieve-manage-is-oknobye))) (sieve-manage-erase) rsp)) @@ -598,7 +601,7 @@ sieve-manage-parse-string (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) (unless (setq rsp (sieve-manage-is-string)) - (when (sieve-manage-no-p (sieve-manage-is-okno)) + (when (sieve-manage-no-p (sieve-manage-is-oknobye)) ;; simple `error' is enough since `sieve-manage-erase' ;; already adds the server response to the log (error (sieve-manage-erase))))) @@ -612,7 +615,7 @@ sieve-manage-parse-crlf (defun sieve-manage-parse-listscripts () (let (tmp rsp data) (while (null rsp) - (while (null (or (setq rsp (sieve-manage-is-okno)) + (while (null (or (setq rsp (sieve-manage-is-oknobye)) (setq tmp (sieve-manage-decode (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) -- 2.39.0 --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0003-Add-test-lisp-net-sieve-manage-tests.el.patch Content-Transfer-Encoding: quoted-printable >From 786325d3adf7514967ae74c2db84c532f19dc070 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Tue, 22 Mar 2022 20:48:09 +0100 Subject: [PATCH 3/4] Add test/lisp/net/sieve-manage-tests.el * test/lisp/net/sieve-manage-tests.el: (mk-literal-s2c) (mk-rsp-oknobye) (mk-rsp-getscript-ok) (managesieve-getscript): Test helper functions. (ert/managesieve-getscript-multibyte): Test `sieve-manage-getscript' with script names containing multibyte characters. --- test/lisp/net/sieve-manage-tests.el | 106 ++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 test/lisp/net/sieve-manage-tests.el diff --git a/test/lisp/net/sieve-manage-tests.el b/test/lisp/net/sieve-mana= ge-tests.el new file mode 100644 index 00000000000..d4c616b3364 --- /dev/null +++ b/test/lisp/net/sieve-manage-tests.el @@ -0,0 +1,106 @@ +;;; sieve-manage-tests.el --- tests for sieve-manage.el -*- lexical-= binding: t; -*- + +;; Copyright (C) 2022 Free Software Foundation, Inc. + +;; Author: Kai Tetzlaff + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see . + +;;; Commentary: + + +;;; Code: + +(require 'ert) +(require 'sieve-manage) + +(defvar sieve-script-multibyte-unix + "if body :matches \"=C3=A4\" { stop; }\n" + "Simple multibyte sieve script.") + +(defun mk-literal-s2c (string) + "Encode STRING to managesieve 'literal-s2c'." + (let ((data (sieve-manage-encode string))) + (concat (format "{%d}\r\n" (length data)) + data))) + +(defun mk-rsp-oknobye (type &optional resp-code string) + "Encode TYPE, RESP-CODE and STRING to managesieve 'response-oknobye'." + (when (memq type '(OK NO BYE)) + (concat + (mapconcat #'identity + (delq nil + (list (symbol-name type) + (when resp-code + (format "(%s)" resp-code)) + (when string + (format "\"%s\"" + (sieve-manage-encode string))))) + " ") + "\r\n"))) +;; (mk-rsp-oknobye 'OK nil "Getscript completed.") +;; (mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") + +(defun mk-rsp-getscript-ok (script) + "Encode SCRIPT to managesieve 'response-getscript'." + (concat (mk-literal-s2c script) + "\r\n" + (mk-rsp-oknobye 'OK "Getscript completed."))) +;; (mk-rsp-getscript-ok sieve-script-multibyte-unix) + +(defun managesieve-getscript (script) + "Simulate managesieve getscript response to test +`sieve-manage-getscript' function." + (let* ((script-name "test.sieve") + ;; `sieve-manage-server' and `sieve-manage-port' are used in + ;; `sieve-manage-make-process-buffer' + (sieve-manage-server) + (sieve-manage-port "sieve") + (sieve-buffer (sieve-manage-make-process-buffer)) + (output-buffer (generate-new-buffer script-name))) + ;; use cl-letf to mock some functions in call chain of + ;; sieve-manage-getscript + (cl-letf (((symbol-function 'sieve-manage-send) + ;; simulate sieve server response with a single + ;; multibyte character `=C3=A4` + (lambda (_) + (with-current-buffer sieve-buffer + (goto-char (point-min)) + (insert (mk-rsp-getscript-ok script))))) + ((symbol-function 'accept-process-output) + (lambda (&optional _ _ _ _) nil)) + ((symbol-function 'get-buffer-process) (lambda (_) nil))) + ;; extract sieve script from sieve-buffer and put it into + ;; output-buffer + (sieve-manage-getscript script-name output-buffer sieve-buffer) + ;; extract script from output-buffer and return it as a string + (let ((script (with-current-buffer output-buffer + (set-buffer-modified-p nil) + (buffer-string)))) + ;; cleanup + (kill-buffer sieve-buffer) + (kill-buffer output-buffer) + (when (get-buffer sieve-manage-log) + (kill-buffer sieve-manage-log)) + ;; return script + script)))) + +(ert-deftest ert/managesieve-getscript-multibyte () + (should (string=3D sieve-script-multibyte-unix + (managesieve-getscript sieve-script-multibyte-unix)))) + + +;;; sieve-manage-tests.el ends here --=20 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0004-Some-minor-fixes-in-lisp-net-sieve.el.patch >From 6c63f891589e6525bf91ef53a0a8114a836a7781 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 04:06:22 +0100 Subject: [PATCH 4/4] Some minor fixes in lisp/net/sieve.el * lisp/net/sieve.el (sieve-next-line) (sieve-prev-line): Handle situations where point in `sieve-buffer' is either before or after the list of server side scripts. (sieve-server-script-list): New variable. (sieve-refresh-scriptlist) (sieve-upload): Use `sieve-server-script-list' to make sure that local list of server side scripts in`sieve-buffer' is up-to-date after uploading a (new) script. (sieve-edit-script): Make sure that the newly created user facing sieve-manage buffer is using 'utf-8-unix encoding. --- lisp/net/sieve.el | 40 +++++++++++++++++++++++++++------------- 1 file changed, 27 insertions(+), 13 deletions(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index fbd07dee27c..1e5aae7825c 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -95,6 +95,8 @@ sieve-template (defvar sieve-manage-buffer nil) (defvar sieve-buffer-header-end nil) +(defvar sieve-server-script-list nil + "Current list of server-side sieve scripts.") (defvar sieve-buffer-script-name nil "The real script name of the buffer.") (make-local-variable 'sieve-buffer-script-name) @@ -204,6 +206,8 @@ sieve-edit-script (if (not (string-equal name sieve-new-script)) (let ((newbuf (generate-new-buffer name)) err) + (with-current-buffer newbuf + (set-buffer-file-coding-system sieve-manage--coding-system)) (setq err (sieve-manage-getscript name newbuf sieve-manage-buffer)) (switch-to-buffer newbuf) (if (sieve-manage-ok-p err) @@ -235,23 +239,28 @@ sieve-next-line (interactive) (unless arg (setq arg 1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "End of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "End of list")) + (goto-char (next-overlay-change sieve-buffer-header-end)))) (defun sieve-prev-line (&optional arg) (interactive) (unless arg (setq arg -1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "Beginning of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "Beginning of list")) + (goto-char (previous-overlay-change (point-max))) + (beginning-of-line))) (defun sieve-help () "Display help for various sieve commands." @@ -319,6 +328,9 @@ sieve-refresh-scriptlist (let* ((scripts (sieve-manage-listscripts sieve-manage-buffer)) (count (length scripts)) (keys (substitute-command-keys "\\[sieve-edit-script]"))) + (setq sieve-server-script-list + (mapcar (lambda (elt) (if (consp elt) (cdr elt) elt)) + scripts)) (insert (if (null scripts) (format @@ -361,7 +373,9 @@ sieve-upload (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." - (substitute-command-keys "\\[sieve-manage]")))) + (substitute-command-keys "\\[sieve-manage]")) + (when (not (member script-name sieve-server-script-list)) + (sieve-refresh-scriptlist)))) (set-buffer-modified-p nil)))) ;;;###autoload -- 2.39.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 19 12:41:32 2023 Received: (at 54154) by debbugs.gnu.org; 19 Jan 2023 17:41:32 +0000 Received: from localhost ([127.0.0.1]:44679 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIYuq-0003al-7O for submit@debbugs.gnu.org; Thu, 19 Jan 2023 12:41:32 -0500 Received: from eggs.gnu.org ([209.51.188.92]:60060) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIYun-0003aX-UE for 54154@debbugs.gnu.org; Thu, 19 Jan 2023 12:41:30 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIYue-00025i-3L; Thu, 19 Jan 2023 12:41:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=UmuGlETW+8Ms8zeUDEK0nZMqh8vmteajtD3Jg2kVfA4=; b=iZDJa5SI2ZYM aYFWTbwOiFNCkNYq/PeA8X6kZnmaWRTuINyUN22uDAGyiiZArXAMm4yNZ+dRRaLKJPtcJuZ0VqWPd e0zb0HkLjIArow8WISV1hjXRCg9Go1QRTWJnVikIaZ/Vjnao/NO1IaB7Yv3m7sGTT7V7JGZDJYm6w 2a1mVISUuI53VgCUcBOTT33cwmW+kPQ2dEh6XeTEjPr2Q59ewFOu64QKMjrHCcqSvi2Zxv1zbZRip VtxrCyYPmIensWgdytlPD6khWfCjv8WewX5z6yXKgXwWfyjxZsMRBi7ubuTh/21wDgYKpRd3R3hul BZFduyfvaGaklIJ8mAMKBA==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIYua-0002vA-Il; Thu, 19 Jan 2023 12:41:19 -0500 Date: Thu, 19 Jan 2023 19:41:13 +0200 Message-Id: <83v8l2o212.fsf@gnu.org> From: Eli Zaretskii To: Kai Tetzlaff In-Reply-To: <87bkmua51z.fsf@tetzco.de> (message from Kai Tetzlaff on Thu, 19 Jan 2023 16:59:36 +0100) Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Kai Tetzlaff > Cc: herbert@gojira.at, larsi@gnus.org, 54154@debbugs.gnu.org > Date: Thu, 19 Jan 2023 16:59:36 +0100 > > >> Yes, true. But since `sieve-manage--set-internal-buffer-properties' is > >> used in two different places, the more elegant solution you suggested > >> above would require duplicating the body of the function in those > >> places. I just didn't see a better way. > > > > I'm not sure why you need to force the encoding of the process buffer, > > when you already set the coding-system to be used for decoding stuff > > from the process. Is that really needed? > > Not sure if it is really needed. But I wanted to make sure that both, > the process buffer and the log buffer use identical settings. Otherwise, > the content of the log buffer might be misleading. I don't think it could mislead, but OK. > > But if you really need this, then just make the insertion of the text > > into the buffer you create optional: then for the process-buffer pass > > nil as the text to insert, and you can do the with-current-buffer > > dance only inside that function. > > Sorry, you lost me there. I don't understand what you want to tell me. > Which (optional) text in which buffer? I meant this: (defun sieve-manage--set-buffer-and-append-text (buffer-name &rest args) (let ((existing-buffer (get-buffer buffer-name)) new-buffer) (if existing-buffer (setq new-buffer existing-buffer) (setq new-buffer (get-buffer-create buffer-name))) (with-current-buffer new-buffer (when (not existing-buffer) (set-buffer-file-coding-system sieve-manage--coding-system) (setq-local after-change-functions nil) (buffer-disable-undo) ; What happened to set-buffer-multibyte? ) (goto-char (point-max)) (apply #'insert args)))) Then you can call it from sieve-manage-make-process-buffer like this: (sieve-manage--set-buffer-and-append-text (format " *sieve %s:%s*" sieve-manage-server sieve-manage-port) "") i.e. with an empty string, so nothing gets inserted into the process buffer. Or you could instead change the signature to accept a single &optional argument that is a list, and then you could make the last two lines in the function above conditional on that argument being non-nil. > > Since you seem to be encoding and decoding to/from UTF-8 by hand in > > sieve-manage-encode/decode, you should use 'binary' as the > > process-codings-system for the network connection to the server, and > > that's it. > > That works. Done. > > > What you should do is call sieve-manage-encode inside > > sieve-manage-send, and count the bytes there after encoding the > > payload. > > Unfortunately, that is too late since the sent data - in case that the > sent text may contain CRLF sequences - contains its own length. So in > order to insert the correct length, I need to encode before sending. > See: > > (defun sieve-manage-putscript (name content &optional buffer) > (with-current-buffer (or buffer (current-buffer)) > (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name > (length (sieve-manage-encode content)) > sieve-manage-client-eol content)) > (sieve-manage-parse-oknobye))) This is because you pass both the text and the number to 'format'. But that is not carved in stone: the "%d" part can never produce any non-ASCII characters, so there's no need to encode it together with CONTENT. You could do this instead: (defun sieve-manage-send (command &optional payload) (let ((encoded (if payload (encode-coding-string payload 'utf-8-unix))) size cmdstr) (if encoded (setq size (format " {%d+}%s" (length encoded) sieve-manage-client-eol))) (setq cmdstr (concat command size encoded)) (sieve-manage--append-to-log cmdstr) (process-send-string sieve-manage-process cmdstr))) And then you call this like below: (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) I hope this clarifies my proposal. From debbugs-submit-bounces@debbugs.gnu.org Thu Jan 19 16:33:29 2023 Received: (at 54154) by debbugs.gnu.org; 19 Jan 2023 21:33:29 +0000 Received: from localhost ([127.0.0.1]:44872 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIcXH-0001GF-Hv for submit@debbugs.gnu.org; Thu, 19 Jan 2023 16:33:29 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:48629) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIcXC-0001Fu-7N for 54154@debbugs.gnu.org; Thu, 19 Jan 2023 16:33:25 -0500 Received: from mail.tetzco.de (ipb21a9472.dynamic.kabel-deutschland.de [178.26.148.114]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id 88D32BD078; Thu, 19 Jan 2023 22:33:16 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id DC4396C00B2; Thu, 19 Jan 2023 22:53:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674165188; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=26r7YvUIhD0Xo0LV3v01iVRkMh3PSr+lzY6LEZPLZtE=; b=LEcBBTm5J4wWXm44Hd6+M6xCZCVaB2bOouyf8QEKyBSKIkdLgXm74ukCP26gUSZOpg46co BBRQD76UsmOZSgXtKSjsf8larRHRZYE6lZDuE99tAjjDNdiFQ2GOOrGfjcUEJbkRTbSjBc UfkVRA4ntrTw3PzQV+tueerq2IFEAWmz0yEk6yYduWEZbLvTsbHK0GyC/gVrm1G2+QLPrD kA8D3neASAq1/Ac+RYXsYty0W0YaqK5gbLOXcu/w8n2xuqvPkPGNXT6ZISdus+Ei9rhQq9 XTBb7oYb1qaOrxVg/EVnU8ob5PlmYsoX2qe/bESl9WM7u2oYmsJVBPj5GWbZd6DoFFivUt +7rLlLXB0AGayff4hHQ8+X4nv0WTnY5Vj7Qb3Gg2HcRqJFBXig+EtMTh8TZhxI1OdwcHp9 osePGbh82C9SoXWCyXmaL/+dH/ZibIEBWWfME5nrQCAB6Ydx6/+mV6epTt6ZvElL3Ielvi e3bBuqGrZWC3XepZMd+2aKp2cLiJOvsLPOFkIthS8CByxF5YT27PVo1DSeA3AeWrhg7Nru 45o9NzSQvuKwBpr1YXRv3WfNa7Z9kFh+WEdB9KWq44l/4NmijvShhMbDeMTuGOHSFZK56v PTVWTL3GjcFvGyBO+Li6a8aplEAPUc6Zg+jzIZB8iQ0VXid10ZuuU= From: Kai Tetzlaff To: Eli Zaretskii Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: <83v8l2o212.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 19 Jan 2023 19:41:13 +0200") References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> Date: Thu, 19 Jan 2023 22:33:14 +0100 Message-ID: <87sfg65hwl.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: DC4396C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.82 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-3.00)[-0.999]; GENERIC_REPUTATION(-0.73)[-0.72663819828647]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:+]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; ARC_NA(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_FROM(0.00)[bug]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Eli Zaretskii writes: >> From: Kai Tetzlaff >> Cc: herbert@gojira.at, larsi@gnus.org, 54154@debbugs.gnu.org >> Date: Thu, 19 Jan 2023 16:59:36 +0100 >> >> >> Yes, true. But since `sieve-manage--set-internal-buffer-properties' is >> >> used in two different places, the more elegant solution you suggested >> >> above would require duplicating the body of the function in those >> >> places. I just didn't see a better way. >> > >> > I'm not sure why you need to force the encoding of the process buffer, >> > when you already set the coding-system to be used for decoding stuff >> > from the process. Is that really needed? >> >> Not sure if it is really needed. But I wanted to make sure that both, >> the process buffer and the log buffer use identical settings. Otherwise, >> the content of the log buffer might be misleading. > > I don't think it could mislead, but OK. > >> > But if you really need this, then just make the insertion of the text >> > into the buffer you create optional: then for the process-buffer pass >> > nil as the text to insert, and you can do the with-current-buffer >> > dance only inside that function. >> >> Sorry, you lost me there. I don't understand what you want to tell me. >> Which (optional) text in which buffer? > > I meant this: > > (defun sieve-manage--set-buffer-and-append-text (buffer-name &rest args) > (let ((existing-buffer (get-buffer buffer-name)) > new-buffer) > (if existing-buffer > (setq new-buffer existing-buffer) > (setq new-buffer (get-buffer-create buffer-name))) > (with-current-buffer new-buffer > (when (not existing-buffer) > (set-buffer-file-coding-system sieve-manage--coding-system) > (setq-local after-change-functions nil) > (buffer-disable-undo) > ; What happened to set-buffer-multibyte? > ) > (goto-char (point-max)) > (apply #'insert args)))) > > Then you can call it from sieve-manage-make-process-buffer like this: > > (sieve-manage--set-buffer-and-append-text > (format " *sieve %s:%s*" sieve-manage-server sieve-manage-port) > "") > > i.e. with an empty string, so nothing gets inserted into the process > buffer. Or you could instead change the signature to accept a single > &optional argument that is a list, and then you could make the last > two lines in the function above conditional on that argument being > non-nil. Ok, I now implemented it like this: (defun sieve-manage--set-buffer-maybe-append-text (buffer-name &rest args) "Append ARGS to buffer named BUFFER-NAME and return buffer. To be used with process and log buffers. When the buffer doesn't exist, it gets created and configure as follows: - set coding system to 'sieve-manage--coding-system', - set buffer to single-byte mode, - set `after-change-functions' to nil to avoid those functions messing with buffer content, - disable undo (to save a bit of memory and improve performance). ARGS can be a nil, a string or a list of strings. If no ARGS are provided, the content of buffer will not be modified." (let* ((existing-buffer (get-buffer buffer-name)) (buffer (or existing-buffer (get-buffer-create buffer-name)))) (with-current-buffer buffer (unless existing-buffer (set-buffer-file-coding-system sieve-manage--coding-system) (set-buffer-multibyte nil) (setq-local after-change-functions nil) (buffer-disable-undo)) (when args (goto-char (point-max)) (apply #'insert args))) buffer)) (defun sieve-manage--append-to-log (&rest args) "Append ARGS to `sieve-manage-log' buffer. If `sieve-manage-log' is nil, logging is disabled. See also `sieve-manage--set-buffer-maybe-append-text'." (when sieve-manage-log (apply #'sieve-manage--set-buffer-maybe-append-text sieve-manage-log args))) (defun sieve-manage-make-process-buffer () (let ((buffer (sieve-manage--set-buffer-maybe-append-text (format " *sieve %s:%s*" sieve-manage-server sieve-manage-port)))) (with-current-buffer buffer (mapc #'make-local-variable sieve-manage-local-variables)) buffer)) Is that better, now? I also added the (set-buffer-multibyte nil) back into the mix. My understanding was that it was not needed when using the 'raw-text-unix coding system but it is now after switching to 'utf-8-unix? >> > What you should do is call sieve-manage-encode inside >> > sieve-manage-send, and count the bytes there after encoding the >> > payload. >> >> Unfortunately, that is too late since the sent data - in case that the >> sent text may contain CRLF sequences - contains its own length. So in >> order to insert the correct length, I need to encode before sending. >> See: >> >> (defun sieve-manage-putscript (name content &optional buffer) >> (with-current-buffer (or buffer (current-buffer)) >> (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name >> (length (sieve-manage-encode content)) >> sieve-manage-client-eol content)) >> (sieve-manage-parse-oknobye))) > > This is because you pass both the text and the number to 'format'. > But that is not carved in stone: the "%d" part can never produce any > non-ASCII characters, so there's no need to encode it together with > CONTENT. You could do this instead: > > (defun sieve-manage-send (command &optional payload) > (let ((encoded (if payload (encode-coding-string payload 'utf-8-unix))) > size cmdstr) > (if encoded > (setq size (format " {%d+}%s" > (length encoded) sieve-manage-client-eol))) > (setq cmdstr (concat command size encoded)) > (sieve-manage--append-to-log cmdstr) > (process-send-string sieve-manage-process cmdstr))) > > And then you call this like below: > > (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) > (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) > > I hope this clarifies my proposal. Yes, it does. Actually, I like it. RFC5804 specifies three variants for string encoding: string = quoted / literal-c2s / literal-s2c Only the first two are relevant for `sieve-menage-send' ('literal-s2c' is for messages from s(server) to c(lient)). For PUTSCRIPT, we need to use 'literal-c2s' which requires the additional length element (since 'quoted' is a) limited in the character set and b) may not exceed 1024 characters). So I would just modify the your `sieve-manage-send' like this: (defun sieve-manage-send (command &optional payload-str) "Send COMMAND with optional PAYLOAD-STR. If non-nil, PAYLOAD-STR will be appended to COMMAND using the \\='literal-s2c\\' representation according to RFC5804." (let ((encoded (when payload-str (sieve-manage-encode payload-str))) literal-c2s cmdstr) (when encoded (setq literal-c2s (format " {%d+}%s%s" (length encoded) sieve-manage-client-eol encoded))) (setq cmdstr (concat (sieve-manage-encode command) literal-c2s sieve-manage-client-eol)) (sieve-manage--append-to-log cmdstr) (process-send-string sieve-manage-process cmdstr))) Apart from renaming some of the variables, this adds encoding of `command' itself (since command may contain multibyte characters in script names) and an additional `sieve-manage-client-eol' at the end of `cmdstr'. As before, updated patches are attached. --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Fix-bug-in-sieve-manage-append-to-log-improve-sieve-.patch >From 977734f16874636c4f2f5e3bb41a86e4338247c4 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:08:07 +0100 Subject: [PATCH 1/4] Fix bug in sieve-manage--append-to-log, improve sieve-manage buffer config Also update/add some comments/docs. * lisp/net/sieve-manage.el (sieve-manage-coding-system-for-read) (sieve-manage-coding-system-for-write): Remove unused constants. (sieve-manage-encode): Change misleading argument name (utf8-string -> str). (sieve-manage--coding-system): New constant. (sieve-manage--set-internal-buffer-properties): New function. (sieve-manage-open-server): Use `sieve-manage--coding-system'. (sieve-manage--append-to-log): Use `sieve-manage--set-internal-buffer-properties' to fix log buffer creation. (sieve-manage-encode) (sieve-manage-make-process-buffer): Use `sieve-manage--set-internal-buffer-properties'. --- lisp/net/sieve-manage.el | 65 +++++++++++++++++++++++++++------------- 1 file changed, 44 insertions(+), 21 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..adfecc7b309 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -58,7 +58,7 @@ ;; ;; References: ;; -;; draft-martin-managesieve-02.txt, +;; RFC5804, ;; "A Protocol for Remotely Managing Sieve Scripts", ;; by Tim Martin. ;; @@ -145,6 +145,15 @@ sieve-manage-ignore-starttls :type 'boolean) ;; Internal variables: +(defconst sieve-manage--coding-system 'utf-8-unix + "Use \\='utf-8-unix coding system for (network) communication. + +Defines the coding system used for the internal (process, log) +buffers and the network stream created to communicate with the +managesieve server. Using \\='utf-8-unix encoding corresponds to +the use of UTF-8 in RFC5804 (managesieve). The explicit use of +\\='-unix\\=' avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") +sequences intact).") (defconst sieve-manage-local-variables '(sieve-manage-server sieve-manage-port @@ -154,8 +163,6 @@ sieve-manage-local-variables sieve-manage-client-eol sieve-manage-server-eol sieve-manage-capability)) -(defconst sieve-manage-coding-system-for-read 'binary) -(defconst sieve-manage-coding-system-for-write 'binary) (defvar sieve-manage-stream nil) (defvar sieve-manage-auth nil) (defvar sieve-manage-server nil) @@ -167,18 +174,37 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions +(defun sieve-manage--set-internal-buffer-properties (buffer) + "Set BUFFER properties for internally used buffers. + +Used for process and log buffers, this function makes sure that +those buffers keep received and sent data intact by: +- setting the coding system to 'sieve-manage--coding-system', +- setting `after-change-functions' to nil to avoid those + functions messing with buffer content. +Also disables undo (to save a bit of memory and improve +performance). + +Returns BUFFER." + (with-current-buffer buffer + (set-buffer-file-coding-system sieve-manage--coding-system) + (setq-local after-change-functions nil) + (buffer-disable-undo) + (current-buffer))) + (defun sieve-manage--append-to-log (&rest args) - "Append ARGS to sieve-manage log buffer. + "Append ARGS to `sieve-manage-log' buffer. ARGS can be a string or a list of strings. -The buffer to use for logging is specifified via -`sieve-manage-log'. If it is nil, logging is disabled." +The buffer to use for logging is specifified via `sieve-manage-log'. +If it is nil, logging is disabled. + +When the `sieve-manage-log' buffer doesn't exist, it gets created (and +configured with some initial settings)." (when sieve-manage-log (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) - (set-buffer-multibyte nil) - (buffer-disable-undo))) + (sieve-manage--set-internal-buffer-properties + (get-buffer-create sieve-manage-log))) (goto-char (point-max)) (apply #'insert args)))) @@ -202,9 +228,9 @@ sieve-manage--error (sieve-manage--append-to-log msg "\n") (error msg))) -(defun sieve-manage-encode (utf8-string) - "Convert UTF8-STRING to managesieve protocol octets." - (encode-coding-string utf8-string 'raw-text t)) +(defun sieve-manage-encode (str) + "Convert STR to managesieve protocol octets." + (encode-coding-string str sieve-manage--coding-system t)) (defun sieve-manage-decode (octets &optional buffer) "Convert managesieve protocol OCTETS to utf-8 string. @@ -216,13 +242,11 @@ sieve-manage-decode (defun sieve-manage-make-process-buffer () (with-current-buffer - (generate-new-buffer (format " *sieve %s:%s*" - sieve-manage-server - sieve-manage-port)) + (sieve-manage--set-internal-buffer-properties + (generate-new-buffer (format " *sieve %s:%s*" + sieve-manage-server + sieve-manage-port))) (mapc #'make-local-variable sieve-manage-local-variables) - (set-buffer-multibyte nil) - (setq-local after-change-functions nil) - (buffer-disable-undo) (current-buffer))) (defun sieve-manage-erase (&optional p buffer) @@ -244,8 +268,7 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream - ;; eol type unix is required to preserve "\r\n" - :coding 'raw-text-unix + :coding `(binary . ,sieve-manage--coding-system) :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-Handle-BYE-in-sieve-manage-server-responses.patch >From c35bbe4a8ece09d0755386009a97ba4a91839753 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:52:43 +0100 Subject: [PATCH 2/4] Handle BYE in sieve-manage server responses * lisp/net/sieve-manage.el (sieve-manage-regex-oknobye): New function. (sieve-manage-parse-oknobye): Renamed from `sieve-manage-parse-okno'. (sieve-manage-open-server) (sieve-sasl-auth) (sieve-manage-listscripts) (sieve-manage-putscript) (sieve-manage-deletescript) (sieve-manage-getscript) (sieve-manage-setactive) (sieve-manage-is-okno) (sieve-manage-parse-oknobye) (sieve-manage-parse-parse-string) (sieve-manage-parse-parse-crlf): Use `sieve-manage-regex-oknobye' to handle BYE in addition to OK and NO. --- lisp/net/sieve-manage.el | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 70d100ead36..b8f91962194 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -278,8 +278,8 @@ sieve-manage-open-server :type stream :coding 'binary :capability-command "CAPABILITY\r\n" - :end-of-command "^\\(OK\\|NO\\).*\n" - :success "^OK.*\n" + :end-of-command "^\\(OK\\|NO\\|BYE\\).*\r\n" + :success "^OK.*\r\n" :return-list t :starttls-function (lambda (capabilities) @@ -338,7 +338,7 @@ sieve-sasl-auth (setq rsp nil) (goto-char (match-end 0)) rsp)) - (setq rsp (sieve-manage-is-okno)))) + (setq rsp (sieve-manage-is-oknobye)))) (accept-process-output sieve-manage-process 1) (goto-char (point-min))) (sieve-manage-erase) @@ -507,17 +507,17 @@ sieve-manage-listscripts (defun sieve-manage-havespace (name size &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "DELETESCRIPT \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -525,17 +525,22 @@ sieve-manage-getscript (sieve-manage-decode (sieve-manage-parse-string) output-buffer) (sieve-manage-parse-crlf) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "SETACTIVE \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) ;; Protocol parsing routines +(defun sieve-manage-regexp-oknobye () + "Return regexp for managesieve 'response-oknobye'." + (concat + "^\\(OK\\|NO\\|BYE\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" + sieve-manage-server-eol)) (defun sieve-manage-wait-for-answer () - (let ((pattern "^\\(OK\\|NO\\).*\n") + (let ((pattern (sieve-manage-regexp-oknobye)) pos) (while (not pos) (setq pos (search-forward-regexp pattern nil t)) @@ -553,10 +558,8 @@ sieve-manage-ok-p (defun sieve-manage-no-p (rsp) (string= (downcase (or (car-safe rsp) "")) "no")) -(defun sieve-manage-is-okno () - (when (looking-at (concat - "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" - sieve-manage-server-eol)) +(defun sieve-manage-is-oknobye () + (when (looking-at (sieve-manage-regexp-oknobye)) (let ((status (match-string 1)) (resp-code (match-string 3)) (response (match-string 5))) @@ -565,12 +568,12 @@ sieve-manage-is-okno (setq response (sieve-manage-is-string))) (list status resp-code response)))) -(defun sieve-manage-parse-okno () +(defun sieve-manage-parse-oknobye () (let (rsp) (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-okno))) + (setq rsp (sieve-manage-is-oknobye))) (sieve-manage-erase) rsp)) @@ -604,7 +607,7 @@ sieve-manage-parse-string (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) (unless (setq rsp (sieve-manage-is-string)) - (when (sieve-manage-no-p (sieve-manage-is-okno)) + (when (sieve-manage-no-p (sieve-manage-is-oknobye)) ;; simple `error' is enough since `sieve-manage-erase' ;; already adds the server response to the log (error (sieve-manage-erase))))) @@ -618,7 +621,7 @@ sieve-manage-parse-crlf (defun sieve-manage-parse-listscripts () (let (tmp rsp data) (while (null rsp) - (while (null (or (setq rsp (sieve-manage-is-okno)) + (while (null (or (setq rsp (sieve-manage-is-oknobye)) (setq tmp (sieve-manage-decode (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) -- 2.39.0 --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0003-Add-test-lisp-net-sieve-manage-tests.el.patch Content-Transfer-Encoding: quoted-printable >From 8a04e8aaaf3abac0fb9d42bb535633c05be909a2 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Tue, 22 Mar 2022 20:48:09 +0100 Subject: [PATCH 3/4] Add test/lisp/net/sieve-manage-tests.el * test/lisp/net/sieve-manage-tests.el: (mk-literal-s2c) (mk-rsp-oknobye) (mk-rsp-getscript-ok) (managesieve-getscript): Test helper functions. (ert/managesieve-getscript-multibyte): Test `sieve-manage-getscript' with script names containing multibyte characters. --- test/lisp/net/sieve-manage-tests.el | 104 ++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 test/lisp/net/sieve-manage-tests.el diff --git a/test/lisp/net/sieve-manage-tests.el b/test/lisp/net/sieve-mana= ge-tests.el new file mode 100644 index 00000000000..010c9071608 --- /dev/null +++ b/test/lisp/net/sieve-manage-tests.el @@ -0,0 +1,104 @@ +;;; sieve-manage-tests.el --- tests for sieve-manage.el -*- lexical-= binding: t; -*- + +;; Copyright (C) 2022 Free Software Foundation, Inc. + +;; Author: Kai Tetzlaff + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see . + +;;; Commentary: + +;;; Code: + +(require 'ert) +(require 'sieve-manage) + +(defvar sieve-script-multibyte-unix + "if body :matches \"=C3=A4\" { stop; }\n" + "Simple multibyte sieve script.") + +(defun mk-literal-s2c (string) + "Encode STRING to managesieve 'literal-s2c'." + (let ((data (sieve-manage-encode string))) + (concat (format "{%d}\r\n" (length data)) + data))) + +(defun mk-rsp-oknobye (type &optional resp-code string) + "Encode TYPE, RESP-CODE and STRING to managesieve 'response-oknobye'." + (when (memq type '(OK NO BYE)) + (concat + (mapconcat #'identity + (delq nil + (list (symbol-name type) + (when resp-code + (format "(%s)" resp-code)) + (when string + (format "\"%s\"" + (sieve-manage-encode string))))) + " ") + "\r\n"))) +;; (mk-rsp-oknobye 'OK nil "Getscript completed.") +;; (mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") + +(defun mk-rsp-getscript-ok (script) + "Encode SCRIPT to managesieve 'response-getscript'." + (concat (mk-literal-s2c script) + "\r\n" + (mk-rsp-oknobye 'OK "Getscript completed."))) +;; (mk-rsp-getscript-ok sieve-script-multibyte-unix) + +(defun managesieve-getscript (script) + "Simulate managesieve getscript response to test +`sieve-manage-getscript' function." + (let* ((script-name "test.sieve") + ;; `sieve-manage-server' and `sieve-manage-port' are used in + ;; `sieve-manage-make-process-buffer' + (sieve-manage-server) + (sieve-manage-port "sieve") + (sieve-buffer (sieve-manage-make-process-buffer)) + (output-buffer (generate-new-buffer script-name))) + ;; use cl-letf to mock some functions in call chain of + ;; sieve-manage-getscript + (cl-letf (((symbol-function 'sieve-manage-send) + ;; simulate sieve server response with a single + ;; multibyte character `=C3=A4` + (lambda (_) + (with-current-buffer sieve-buffer + (goto-char (point-min)) + (insert (mk-rsp-getscript-ok script))))) + ((symbol-function 'accept-process-output) + (lambda (&optional _ _ _ _) nil)) + ((symbol-function 'get-buffer-process) (lambda (_) nil))) + ;; extract sieve script from sieve-buffer and put it into + ;; output-buffer + (sieve-manage-getscript script-name output-buffer sieve-buffer) + ;; extract script from output-buffer and return it as a string + (let ((script (with-current-buffer output-buffer + (set-buffer-modified-p nil) + (buffer-string)))) + ;; cleanup + (kill-buffer sieve-buffer) + (kill-buffer output-buffer) + (when (get-buffer sieve-manage-log) + (kill-buffer sieve-manage-log)) + ;; return script + script)))) + +(ert-deftest ert/managesieve-getscript-multibyte () + (should (string=3D sieve-script-multibyte-unix + (managesieve-getscript sieve-script-multibyte-unix)))) + +;;; sieve-manage-tests.el ends here --=20 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0004-Some-minor-fixes-in-lisp-net-sieve.el.patch >From 24cef4c762f5c5f5de9799b57bf6789ff0724b1d Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 04:06:22 +0100 Subject: [PATCH 4/4] Some minor fixes in lisp/net/sieve.el * lisp/net/sieve.el (sieve-next-line) (sieve-prev-line): Handle situations where point in `sieve-buffer' is either before or after the list of server side scripts. (sieve-server-script-list): New variable. (sieve-refresh-scriptlist) (sieve-upload): Use `sieve-server-script-list' to make sure that local list of server side scripts in`sieve-buffer' is up-to-date after uploading a (new) script. (sieve-edit-script): Make sure that the newly created user facing sieve-manage buffer is using 'utf-8-unix encoding. --- lisp/net/sieve.el | 40 +++++++++++++++++++++++++++------------- 1 file changed, 27 insertions(+), 13 deletions(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index fbd07dee27c..1e5aae7825c 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -95,6 +95,8 @@ sieve-template (defvar sieve-manage-buffer nil) (defvar sieve-buffer-header-end nil) +(defvar sieve-server-script-list nil + "Current list of server-side sieve scripts.") (defvar sieve-buffer-script-name nil "The real script name of the buffer.") (make-local-variable 'sieve-buffer-script-name) @@ -204,6 +206,8 @@ sieve-edit-script (if (not (string-equal name sieve-new-script)) (let ((newbuf (generate-new-buffer name)) err) + (with-current-buffer newbuf + (set-buffer-file-coding-system sieve-manage--coding-system)) (setq err (sieve-manage-getscript name newbuf sieve-manage-buffer)) (switch-to-buffer newbuf) (if (sieve-manage-ok-p err) @@ -235,23 +239,28 @@ sieve-next-line (interactive) (unless arg (setq arg 1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "End of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "End of list")) + (goto-char (next-overlay-change sieve-buffer-header-end)))) (defun sieve-prev-line (&optional arg) (interactive) (unless arg (setq arg -1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "Beginning of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "Beginning of list")) + (goto-char (previous-overlay-change (point-max))) + (beginning-of-line))) (defun sieve-help () "Display help for various sieve commands." @@ -319,6 +328,9 @@ sieve-refresh-scriptlist (let* ((scripts (sieve-manage-listscripts sieve-manage-buffer)) (count (length scripts)) (keys (substitute-command-keys "\\[sieve-edit-script]"))) + (setq sieve-server-script-list + (mapcar (lambda (elt) (if (consp elt) (cdr elt) elt)) + scripts)) (insert (if (null scripts) (format @@ -361,7 +373,9 @@ sieve-upload (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." - (substitute-command-keys "\\[sieve-manage]")))) + (substitute-command-keys "\\[sieve-manage]")) + (when (not (member script-name sieve-server-script-list)) + (sieve-refresh-scriptlist)))) (set-buffer-modified-p nil)))) ;;;###autoload -- 2.39.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Jan 20 01:54:36 2023 Received: (at 54154) by debbugs.gnu.org; 20 Jan 2023 06:54:36 +0000 Received: from localhost ([127.0.0.1]:45295 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIlII-0007bQ-BV for submit@debbugs.gnu.org; Fri, 20 Jan 2023 01:54:35 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:43733) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pIlIE-0007b7-DA for 54154@debbugs.gnu.org; Fri, 20 Jan 2023 01:54:32 -0500 Received: from mail.tetzco.de (ipb21a9472.dynamic.kabel-deutschland.de [178.26.148.114]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id B18E7BD078; Fri, 20 Jan 2023 07:54:22 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id 9A2346C00B2; Fri, 20 Jan 2023 08:14:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674198851; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=RspsHUxUz+HsJ7xPFZPZzAiT32RUZedSW6oWPfMSXRs=; b=SRrZ71zCD/+UQKiZBBJHUFyz/ACN0dH2QbWDJg6KObA2pvhj8z1TzW6BgsqlrrCl0qooNC I645ca+nsSC5dyCIKY72R/MjNhxJyxnP1/VTmkyC+SJft8FsJ1yCDPFTLPjL1tHeGpnluT V+rDcWcYRG5dIuvmETVP+MiJfO/D9hAHbrcWDrvohS72N4OcXm1LbOdvzU1JlsYm6Z4UUL KsPQpbVVYczXfzL0ToHpFrOcrDTuviaKoNI6+XybWOLJ5hxXYdHzh21K7EO1jXD5aBX+rs qlVOHpL1jeUnEOfPuU9wtLsaElSJ0+HqMjplUDaTtVDoZPejpoCw11dOCYSUnxmYMaj0eT mMmEovwKgJAtjVsQCu5csPI/tTHe1JPdSqjtrIajcVPfn/y6ciTJXCLrnfHDe/oLDLObNW PUHINwIJDWFqVPTMKOiO2lPzIJjxDzrAJ8UpgczN2saxM+kdiEwMtKvZm/hg6Zx2zMnoQM DLTtQ9xtzncvgs2Q+n+KSofYHdl/WLG51JV6lo5gaXCm/wf6KW6tAOp/0CJbT4q9//qtxv wY7zsw57nUxG+3+76FnZYBb2CsiqamWGfn6EE4y21na+AuwC4FqJcdNeyu0Pj3Z12ulUUU HKbiS2TtjnAoQJJeHFOHhoGxJXfuL0gRus9QVg3rmshVzymD6Y5Wo= From: Kai Tetzlaff To: Eli Zaretskii Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: <83v8l2o212.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 19 Jan 2023 19:41:13 +0200") References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> Date: Fri, 20 Jan 2023 07:54:15 +0100 Message-ID: <87pmb966i0.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: 9A2346C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.82 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-3.00)[-0.999]; GENERIC_REPUTATION(-0.73)[-0.72663820055784]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:+]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; ARC_NA(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_FROM(0.00)[bug]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Eli Zaretskii writes: Sorry, the first patch in the last email was outdated. Please check the updated ones below. >> From: Kai Tetzlaff >> Cc: herbert@gojira.at, larsi@gnus.org, 54154@debbugs.gnu.org >> Date: Thu, 19 Jan 2023 16:59:36 +0100 >> >> >> Yes, true. But since `sieve-manage--set-internal-buffer-properties' is >> >> used in two different places, the more elegant solution you suggested >> >> above would require duplicating the body of the function in those >> >> places. I just didn't see a better way. >> > >> > I'm not sure why you need to force the encoding of the process buffer, >> > when you already set the coding-system to be used for decoding stuff >> > from the process. Is that really needed? >> >> Not sure if it is really needed. But I wanted to make sure that both, >> the process buffer and the log buffer use identical settings. Otherwise, >> the content of the log buffer might be misleading. > > I don't think it could mislead, but OK. > >> > But if you really need this, then just make the insertion of the text >> > into the buffer you create optional: then for the process-buffer pass >> > nil as the text to insert, and you can do the with-current-buffer >> > dance only inside that function. >> >> Sorry, you lost me there. I don't understand what you want to tell me. >> Which (optional) text in which buffer? > > I meant this: > > (defun sieve-manage--set-buffer-and-append-text (buffer-name &rest args) > (let ((existing-buffer (get-buffer buffer-name)) > new-buffer) > (if existing-buffer > (setq new-buffer existing-buffer) > (setq new-buffer (get-buffer-create buffer-name))) > (with-current-buffer new-buffer > (when (not existing-buffer) > (set-buffer-file-coding-system sieve-manage--coding-system) > (setq-local after-change-functions nil) > (buffer-disable-undo) > ; What happened to set-buffer-multibyte? > ) > (goto-char (point-max)) > (apply #'insert args)))) > > Then you can call it from sieve-manage-make-process-buffer like this: > > (sieve-manage--set-buffer-and-append-text > (format " *sieve %s:%s*" sieve-manage-server sieve-manage-port) > "") > > i.e. with an empty string, so nothing gets inserted into the process > buffer. Or you could instead change the signature to accept a single > &optional argument that is a list, and then you could make the last > two lines in the function above conditional on that argument being > non-nil. Ok, I now implemented it like this: (defun sieve-manage--set-buffer-maybe-append-text (buffer-name &rest args) "Append ARGS to buffer named BUFFER-NAME and return buffer. To be used with process and log buffers. When the buffer doesn't exist, it gets created and configure as follows: - set coding system to 'sieve-manage--coding-system', - set buffer to single-byte mode, - set `after-change-functions' to nil to avoid those functions messing with buffer content, - disable undo (to save a bit of memory and improve performance). ARGS can be a nil, a string or a list of strings. If no ARGS are provided, the content of buffer will not be modified." (let* ((existing-buffer (get-buffer buffer-name)) (buffer (or existing-buffer (get-buffer-create buffer-name)))) (with-current-buffer buffer (unless existing-buffer (set-buffer-file-coding-system sieve-manage--coding-system) (set-buffer-multibyte nil) (setq-local after-change-functions nil) (buffer-disable-undo)) (when args (goto-char (point-max)) (apply #'insert args))) buffer)) (defun sieve-manage--append-to-log (&rest args) "Append ARGS to `sieve-manage-log' buffer. If `sieve-manage-log' is nil, logging is disabled. See also `sieve-manage--set-buffer-maybe-append-text'." (when sieve-manage-log (apply #'sieve-manage--set-buffer-maybe-append-text sieve-manage-log args))) (defun sieve-manage-make-process-buffer () (let ((buffer (sieve-manage--set-buffer-maybe-append-text (format " *sieve %s:%s*" sieve-manage-server sieve-manage-port)))) (with-current-buffer buffer (mapc #'make-local-variable sieve-manage-local-variables)) buffer)) Is that better, now? I also added the (set-buffer-multibyte nil) back into the mix. My understanding was that it was not needed when using the 'raw-text-unix coding system but it is now after switching to 'utf-8-unix? >> > What you should do is call sieve-manage-encode inside >> > sieve-manage-send, and count the bytes there after encoding the >> > payload. >> >> Unfortunately, that is too late since the sent data - in case that the >> sent text may contain CRLF sequences - contains its own length. So in >> order to insert the correct length, I need to encode before sending. >> See: >> >> (defun sieve-manage-putscript (name content &optional buffer) >> (with-current-buffer (or buffer (current-buffer)) >> (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name >> (length (sieve-manage-encode content)) >> sieve-manage-client-eol content)) >> (sieve-manage-parse-oknobye))) > > This is because you pass both the text and the number to 'format'. > But that is not carved in stone: the "%d" part can never produce any > non-ASCII characters, so there's no need to encode it together with > CONTENT. You could do this instead: > > (defun sieve-manage-send (command &optional payload) > (let ((encoded (if payload (encode-coding-string payload 'utf-8-unix))) > size cmdstr) > (if encoded > (setq size (format " {%d+}%s" > (length encoded) sieve-manage-client-eol))) > (setq cmdstr (concat command size encoded)) > (sieve-manage--append-to-log cmdstr) > (process-send-string sieve-manage-process cmdstr))) > > And then you call this like below: > > (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) > (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) > > I hope this clarifies my proposal. Yes, it does. Actually, I like it. RFC5804 specifies three variants for string encoding: string = quoted / literal-c2s / literal-s2c Only the first two are relevant for `sieve-menage-send' ('literal-s2c' is for messages from s(server) to c(lient)). For PUTSCRIPT, we need to use 'literal-c2s' which requires the additional length element (since 'quoted' is a) limited in the character set and b) may not exceed 1024 characters). So I would just modify the your `sieve-manage-send' like this: (defun sieve-manage-send (command &optional payload-str) "Send COMMAND with optional PAYLOAD-STR. If non-nil, PAYLOAD-STR will be appended to COMMAND using the \\='literal-s2c\\' representation according to RFC5804." (let ((encoded (when payload-str (sieve-manage-encode payload-str))) literal-c2s cmdstr) (when encoded (setq literal-c2s (format " {%d+}%s%s" (length encoded) sieve-manage-client-eol encoded))) (setq cmdstr (concat (sieve-manage-encode command) literal-c2s sieve-manage-client-eol)) (sieve-manage--append-to-log cmdstr) (process-send-string sieve-manage-process cmdstr))) Apart from renaming some of the variables, this adds encoding of `command' itself (since command may contain multibyte characters in script names) and an additional `sieve-manage-client-eol' at the end of `cmdstr'. As before, updated patches are attached. --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Fix-bug-in-sieve-manage-append-to-log-and-do-some-re.patch >From a8e07ffdfb150f63c77bfb2a3f3ca59e2633cfbd Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:08:07 +0100 Subject: [PATCH 1/4] Fix bug in sieve-manage--append-to-log and do some refactoring - simplify buffer creation and setup for process and log buffers - handle all encoding tasks in `sieve-manage-send' (and thus simplify `sieve-manage-putscript') - add/update doc-strings - update some comments * lisp/net/sieve-manage.el (sieve-manage-coding-system-for-read) (sieve-manage-coding-system-for-write): Remove unused constants. (sieve-manage-encode): Change misleading argument name (utf8-string -> str). (sieve-manage--coding-system): New constant. (sieve-manage-encode): Use `sieve-manage--coding-system', rename arguments, update doc string. (sieve-manage--set-buffer-maybe-append-text): New function. (sieve-manage-open-server): Change :coding property of `open-network-stream' from 'raw-text-unix to 'binary. (sieve-manage--append-to-log): Use `sieve-manage--set-buffer-maybe-append-text' to fix log buffer creation. (sieve-manage-decode) (sieve-manage-make-process-buffer): Use `sieve-manage--set-buffer-maybe-append-text'. (sieve-manage-send): Handle all encoding steps (including the 'literal-c2s' encoding previously done in `sieve-manage-putscript'). (sieve-manage-putscript): Move all encoding steps to `sieve-manage-send'. --- lisp/net/sieve-manage.el | 115 ++++++++++++++++++++++++++------------- 1 file changed, 78 insertions(+), 37 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..70d100ead36 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -58,7 +58,7 @@ ;; ;; References: ;; -;; draft-martin-managesieve-02.txt, +;; RFC5804, ;; "A Protocol for Remotely Managing Sieve Scripts", ;; by Tim Martin. ;; @@ -145,6 +145,15 @@ sieve-manage-ignore-starttls :type 'boolean) ;; Internal variables: +(defconst sieve-manage--coding-system 'utf-8-unix + "Use \\='utf-8-unix coding system for (network) communication. + +Defines the coding system used for the internal (process, log) +buffers and the network stream created to communicate with the +managesieve server. Using \\='utf-8-unix encoding corresponds to +the use of UTF-8 in RFC5804 (managesieve). The explicit use of +\\='-unix\\=' avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") +sequences intact).") (defconst sieve-manage-local-variables '(sieve-manage-server sieve-manage-port @@ -154,8 +163,6 @@ sieve-manage-local-variables sieve-manage-client-eol sieve-manage-server-eol sieve-manage-capability)) -(defconst sieve-manage-coding-system-for-read 'binary) -(defconst sieve-manage-coding-system-for-write 'binary) (defvar sieve-manage-stream nil) (defvar sieve-manage-auth nil) (defvar sieve-manage-server nil) @@ -167,20 +174,45 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions +(defun sieve-manage--set-buffer-maybe-append-text (buffer-name + &rest args) + "Append ARGS to buffer named BUFFER-NAME and return buffer. + +To be used with process and log buffers. When the buffer doesn't +exist, it gets created and configure as follows: +- set coding system to 'sieve-manage--coding-system', +- set buffer to single-byte mode, +- set `after-change-functions' to nil to avoid those + functions messing with buffer content, +- disable undo (to save a bit of memory and improve + performance). + +ARGS can be a nil, a string or a list of strings. If no +ARGS are provided, the content of buffer will not be +modified." + (let* ((existing-buffer (get-buffer buffer-name)) + (buffer (or existing-buffer + (get-buffer-create buffer-name)))) + (with-current-buffer buffer + (unless existing-buffer + (set-buffer-file-coding-system sieve-manage--coding-system) + (set-buffer-multibyte nil) + (setq-local after-change-functions nil) + (buffer-disable-undo)) + (when args + (goto-char (point-max)) + (apply #'insert args))) + buffer)) + (defun sieve-manage--append-to-log (&rest args) - "Append ARGS to sieve-manage log buffer. + "Append ARGS to `sieve-manage-log' buffer. -ARGS can be a string or a list of strings. -The buffer to use for logging is specifified via -`sieve-manage-log'. If it is nil, logging is disabled." +If `sieve-manage-log' is nil, logging is disabled. See also +`sieve-manage--set-buffer-maybe-append-text'." (when sieve-manage-log - (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) - (set-buffer-multibyte nil) - (buffer-disable-undo))) - (goto-char (point-max)) - (apply #'insert args)))) + (apply #'sieve-manage--set-buffer-maybe-append-text + sieve-manage-log + args))) (defun sieve-manage--message (format-string &rest args) "Wrapper around `message' which also logs to sieve manage log. @@ -202,9 +234,9 @@ sieve-manage--error (sieve-manage--append-to-log msg "\n") (error msg))) -(defun sieve-manage-encode (utf8-string) - "Convert UTF8-STRING to managesieve protocol octets." - (encode-coding-string utf8-string 'raw-text t)) +(defun sieve-manage-encode (str) + "Convert STR to managesieve protocol octets." + (encode-coding-string str sieve-manage--coding-system t)) (defun sieve-manage-decode (octets &optional buffer) "Convert managesieve protocol OCTETS to utf-8 string. @@ -212,18 +244,18 @@ sieve-manage-decode If optional BUFFER is non-nil, insert decoded string into BUFFER." (when octets ;; eol type unix is required to preserve "\r\n" - (decode-coding-string octets 'utf-8-unix t buffer))) + (decode-coding-string octets + sieve-manage--coding-system + t buffer))) (defun sieve-manage-make-process-buffer () - (with-current-buffer - (generate-new-buffer (format " *sieve %s:%s*" - sieve-manage-server - sieve-manage-port)) - (mapc #'make-local-variable sieve-manage-local-variables) - (set-buffer-multibyte nil) - (setq-local after-change-functions nil) - (buffer-disable-undo) - (current-buffer))) + (let ((buffer (sieve-manage--set-buffer-maybe-append-text + (format " *sieve %s:%s*" + sieve-manage-server + sieve-manage-port)))) + (with-current-buffer buffer + (mapc #'make-local-variable sieve-manage-local-variables)) + buffer)) (defun sieve-manage-erase (&optional p buffer) (with-current-buffer (or buffer (current-buffer)) @@ -244,8 +276,7 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream - ;; eol type unix is required to preserve "\r\n" - :coding 'raw-text-unix + :coding 'binary :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" @@ -480,9 +511,7 @@ sieve-manage-havespace (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name - (length (sieve-manage-encode content)) - sieve-manage-client-eol content)) + (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) (sieve-manage-parse-okno))) (defun sieve-manage-deletescript (name &optional buffer) @@ -609,11 +638,23 @@ sieve-manage-parse-listscripts data rsp))) -(defun sieve-manage-send (cmdstr) - (setq cmdstr (sieve-manage-encode - (concat cmdstr sieve-manage-client-eol))) - (sieve-manage--append-to-log cmdstr) - (process-send-string sieve-manage-process cmdstr)) +(defun sieve-manage-send (command &optional payload-str) + "Send COMMAND with optional PAYLOAD-STR. + +If non-nil, PAYLOAD-STR will be appended to COMMAND using the +\\='literal-s2c\\' representation according to RFC5804." + (let ((encoded (when payload-str (sieve-manage-encode payload-str))) + literal-c2s cmdstr) + (when encoded + (setq literal-c2s (format " {%d+}%s%s" + (length encoded) + sieve-manage-client-eol + encoded))) + (setq cmdstr (concat (sieve-manage-encode command) + literal-c2s + sieve-manage-client-eol)) + (sieve-manage--append-to-log cmdstr) + (process-send-string sieve-manage-process cmdstr))) (provide 'sieve-manage) -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-Handle-BYE-in-sieve-manage-server-responses.patch >From c35bbe4a8ece09d0755386009a97ba4a91839753 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:52:43 +0100 Subject: [PATCH 2/4] Handle BYE in sieve-manage server responses * lisp/net/sieve-manage.el (sieve-manage-regex-oknobye): New function. (sieve-manage-parse-oknobye): Renamed from `sieve-manage-parse-okno'. (sieve-manage-open-server) (sieve-sasl-auth) (sieve-manage-listscripts) (sieve-manage-putscript) (sieve-manage-deletescript) (sieve-manage-getscript) (sieve-manage-setactive) (sieve-manage-is-okno) (sieve-manage-parse-oknobye) (sieve-manage-parse-parse-string) (sieve-manage-parse-parse-crlf): Use `sieve-manage-regex-oknobye' to handle BYE in addition to OK and NO. --- lisp/net/sieve-manage.el | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 70d100ead36..b8f91962194 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -278,8 +278,8 @@ sieve-manage-open-server :type stream :coding 'binary :capability-command "CAPABILITY\r\n" - :end-of-command "^\\(OK\\|NO\\).*\n" - :success "^OK.*\n" + :end-of-command "^\\(OK\\|NO\\|BYE\\).*\r\n" + :success "^OK.*\r\n" :return-list t :starttls-function (lambda (capabilities) @@ -338,7 +338,7 @@ sieve-sasl-auth (setq rsp nil) (goto-char (match-end 0)) rsp)) - (setq rsp (sieve-manage-is-okno)))) + (setq rsp (sieve-manage-is-oknobye)))) (accept-process-output sieve-manage-process 1) (goto-char (point-min))) (sieve-manage-erase) @@ -507,17 +507,17 @@ sieve-manage-listscripts (defun sieve-manage-havespace (name size &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "DELETESCRIPT \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -525,17 +525,22 @@ sieve-manage-getscript (sieve-manage-decode (sieve-manage-parse-string) output-buffer) (sieve-manage-parse-crlf) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "SETACTIVE \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) ;; Protocol parsing routines +(defun sieve-manage-regexp-oknobye () + "Return regexp for managesieve 'response-oknobye'." + (concat + "^\\(OK\\|NO\\|BYE\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" + sieve-manage-server-eol)) (defun sieve-manage-wait-for-answer () - (let ((pattern "^\\(OK\\|NO\\).*\n") + (let ((pattern (sieve-manage-regexp-oknobye)) pos) (while (not pos) (setq pos (search-forward-regexp pattern nil t)) @@ -553,10 +558,8 @@ sieve-manage-ok-p (defun sieve-manage-no-p (rsp) (string= (downcase (or (car-safe rsp) "")) "no")) -(defun sieve-manage-is-okno () - (when (looking-at (concat - "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" - sieve-manage-server-eol)) +(defun sieve-manage-is-oknobye () + (when (looking-at (sieve-manage-regexp-oknobye)) (let ((status (match-string 1)) (resp-code (match-string 3)) (response (match-string 5))) @@ -565,12 +568,12 @@ sieve-manage-is-okno (setq response (sieve-manage-is-string))) (list status resp-code response)))) -(defun sieve-manage-parse-okno () +(defun sieve-manage-parse-oknobye () (let (rsp) (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-okno))) + (setq rsp (sieve-manage-is-oknobye))) (sieve-manage-erase) rsp)) @@ -604,7 +607,7 @@ sieve-manage-parse-string (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) (unless (setq rsp (sieve-manage-is-string)) - (when (sieve-manage-no-p (sieve-manage-is-okno)) + (when (sieve-manage-no-p (sieve-manage-is-oknobye)) ;; simple `error' is enough since `sieve-manage-erase' ;; already adds the server response to the log (error (sieve-manage-erase))))) @@ -618,7 +621,7 @@ sieve-manage-parse-crlf (defun sieve-manage-parse-listscripts () (let (tmp rsp data) (while (null rsp) - (while (null (or (setq rsp (sieve-manage-is-okno)) + (while (null (or (setq rsp (sieve-manage-is-oknobye)) (setq tmp (sieve-manage-decode (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) -- 2.39.0 --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0003-Add-test-lisp-net-sieve-manage-tests.el.patch Content-Transfer-Encoding: quoted-printable >From 8a04e8aaaf3abac0fb9d42bb535633c05be909a2 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Tue, 22 Mar 2022 20:48:09 +0100 Subject: [PATCH 3/4] Add test/lisp/net/sieve-manage-tests.el * test/lisp/net/sieve-manage-tests.el: (mk-literal-s2c) (mk-rsp-oknobye) (mk-rsp-getscript-ok) (managesieve-getscript): Test helper functions. (ert/managesieve-getscript-multibyte): Test `sieve-manage-getscript' with script names containing multibyte characters. --- test/lisp/net/sieve-manage-tests.el | 104 ++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 test/lisp/net/sieve-manage-tests.el diff --git a/test/lisp/net/sieve-manage-tests.el b/test/lisp/net/sieve-mana= ge-tests.el new file mode 100644 index 00000000000..010c9071608 --- /dev/null +++ b/test/lisp/net/sieve-manage-tests.el @@ -0,0 +1,104 @@ +;;; sieve-manage-tests.el --- tests for sieve-manage.el -*- lexical-= binding: t; -*- + +;; Copyright (C) 2022 Free Software Foundation, Inc. + +;; Author: Kai Tetzlaff + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see . + +;;; Commentary: + +;;; Code: + +(require 'ert) +(require 'sieve-manage) + +(defvar sieve-script-multibyte-unix + "if body :matches \"=C3=A4\" { stop; }\n" + "Simple multibyte sieve script.") + +(defun mk-literal-s2c (string) + "Encode STRING to managesieve 'literal-s2c'." + (let ((data (sieve-manage-encode string))) + (concat (format "{%d}\r\n" (length data)) + data))) + +(defun mk-rsp-oknobye (type &optional resp-code string) + "Encode TYPE, RESP-CODE and STRING to managesieve 'response-oknobye'." + (when (memq type '(OK NO BYE)) + (concat + (mapconcat #'identity + (delq nil + (list (symbol-name type) + (when resp-code + (format "(%s)" resp-code)) + (when string + (format "\"%s\"" + (sieve-manage-encode string))))) + " ") + "\r\n"))) +;; (mk-rsp-oknobye 'OK nil "Getscript completed.") +;; (mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") + +(defun mk-rsp-getscript-ok (script) + "Encode SCRIPT to managesieve 'response-getscript'." + (concat (mk-literal-s2c script) + "\r\n" + (mk-rsp-oknobye 'OK "Getscript completed."))) +;; (mk-rsp-getscript-ok sieve-script-multibyte-unix) + +(defun managesieve-getscript (script) + "Simulate managesieve getscript response to test +`sieve-manage-getscript' function." + (let* ((script-name "test.sieve") + ;; `sieve-manage-server' and `sieve-manage-port' are used in + ;; `sieve-manage-make-process-buffer' + (sieve-manage-server) + (sieve-manage-port "sieve") + (sieve-buffer (sieve-manage-make-process-buffer)) + (output-buffer (generate-new-buffer script-name))) + ;; use cl-letf to mock some functions in call chain of + ;; sieve-manage-getscript + (cl-letf (((symbol-function 'sieve-manage-send) + ;; simulate sieve server response with a single + ;; multibyte character `=C3=A4` + (lambda (_) + (with-current-buffer sieve-buffer + (goto-char (point-min)) + (insert (mk-rsp-getscript-ok script))))) + ((symbol-function 'accept-process-output) + (lambda (&optional _ _ _ _) nil)) + ((symbol-function 'get-buffer-process) (lambda (_) nil))) + ;; extract sieve script from sieve-buffer and put it into + ;; output-buffer + (sieve-manage-getscript script-name output-buffer sieve-buffer) + ;; extract script from output-buffer and return it as a string + (let ((script (with-current-buffer output-buffer + (set-buffer-modified-p nil) + (buffer-string)))) + ;; cleanup + (kill-buffer sieve-buffer) + (kill-buffer output-buffer) + (when (get-buffer sieve-manage-log) + (kill-buffer sieve-manage-log)) + ;; return script + script)))) + +(ert-deftest ert/managesieve-getscript-multibyte () + (should (string=3D sieve-script-multibyte-unix + (managesieve-getscript sieve-script-multibyte-unix)))) + +;;; sieve-manage-tests.el ends here --=20 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0004-Some-minor-fixes-in-lisp-net-sieve.el.patch >From 24cef4c762f5c5f5de9799b57bf6789ff0724b1d Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 04:06:22 +0100 Subject: [PATCH 4/4] Some minor fixes in lisp/net/sieve.el * lisp/net/sieve.el (sieve-next-line) (sieve-prev-line): Handle situations where point in `sieve-buffer' is either before or after the list of server side scripts. (sieve-server-script-list): New variable. (sieve-refresh-scriptlist) (sieve-upload): Use `sieve-server-script-list' to make sure that local list of server side scripts in`sieve-buffer' is up-to-date after uploading a (new) script. (sieve-edit-script): Make sure that the newly created user facing sieve-manage buffer is using 'utf-8-unix encoding. --- lisp/net/sieve.el | 40 +++++++++++++++++++++++++++------------- 1 file changed, 27 insertions(+), 13 deletions(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index fbd07dee27c..1e5aae7825c 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -95,6 +95,8 @@ sieve-template (defvar sieve-manage-buffer nil) (defvar sieve-buffer-header-end nil) +(defvar sieve-server-script-list nil + "Current list of server-side sieve scripts.") (defvar sieve-buffer-script-name nil "The real script name of the buffer.") (make-local-variable 'sieve-buffer-script-name) @@ -204,6 +206,8 @@ sieve-edit-script (if (not (string-equal name sieve-new-script)) (let ((newbuf (generate-new-buffer name)) err) + (with-current-buffer newbuf + (set-buffer-file-coding-system sieve-manage--coding-system)) (setq err (sieve-manage-getscript name newbuf sieve-manage-buffer)) (switch-to-buffer newbuf) (if (sieve-manage-ok-p err) @@ -235,23 +239,28 @@ sieve-next-line (interactive) (unless arg (setq arg 1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "End of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "End of list")) + (goto-char (next-overlay-change sieve-buffer-header-end)))) (defun sieve-prev-line (&optional arg) (interactive) (unless arg (setq arg -1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "Beginning of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "Beginning of list")) + (goto-char (previous-overlay-change (point-max))) + (beginning-of-line))) (defun sieve-help () "Display help for various sieve commands." @@ -319,6 +328,9 @@ sieve-refresh-scriptlist (let* ((scripts (sieve-manage-listscripts sieve-manage-buffer)) (count (length scripts)) (keys (substitute-command-keys "\\[sieve-edit-script]"))) + (setq sieve-server-script-list + (mapcar (lambda (elt) (if (consp elt) (cdr elt) elt)) + scripts)) (insert (if (null scripts) (format @@ -361,7 +373,9 @@ sieve-upload (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." - (substitute-command-keys "\\[sieve-manage]")))) + (substitute-command-keys "\\[sieve-manage]")) + (when (not (member script-name sieve-server-script-list)) + (sieve-refresh-scriptlist)))) (set-buffer-modified-p nil)))) ;;;###autoload -- 2.39.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sat Jan 21 21:12:20 2023 Received: (at 54154) by debbugs.gnu.org; 22 Jan 2023 02:12:20 +0000 Received: from localhost ([127.0.0.1]:50158 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJPqE-0002gP-Qz for submit@debbugs.gnu.org; Sat, 21 Jan 2023 21:12:20 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:58423) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJPqB-0002g9-TB for 54154@debbugs.gnu.org; Sat, 21 Jan 2023 21:12:17 -0500 Received: from mail.tetzco.de (unknown [IPv6:2a02:810d:1380:7900::9]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id 021EABD078; Sun, 22 Jan 2023 03:12:10 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id C92C16C00B2; Sun, 22 Jan 2023 03:31:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674354718; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=4XLyLlEapIvB4sw12IR8eRdsF8Yv8DKsgk6CwmoPANQ=; b=OsJlvoNayFwdXuw3o8taPqtxjIbVUGQheWLWRWWxY+DRtCq9OQ7USCswasrl2mYmK1Vl4/ JtauwfppZew60wNhMMEY8fPEhSuTekXAmDluyZUOPxZ4ABuJc0sLd+wQVeF/BymuUAziDX jirnzdDAqMjxx2k2iwQXr9TxGVvYELM2PnreIf55z/Werq8xbXx2PVN9SSju45/c9TC6hA 4O8Ju5Zt+pl/2ZZvLv6bS84DihNRiUqLFXWGb0m3JDFXv01OYcw9L3eXD6ug5yDlMdisTa JCDrHJegq6H8XmVaeVclv6z4vL+RqWpWy5/cijmJZ9g1JMS0O26tjHKeWRUndjv/xs18VU 3WY5RJxpkkDnQYMUFUQyX2zq1Lz/sLVqSdCum4PfLVvcFOX1BUTOa2OsbNLUlldNDEroR1 00VYlJDH9CQSxxFVX7x817G+dSGEaEogehaqen7oCXRUG8j84+ByQzSqOO0xX8vCnDVds9 g2adrGQ0kXJUAAY9kfUGIchzxBDThTv/r7k7oTcUpiFwwa2ZKZtRDHi3fxLdfuFdwla2R4 1Mu1q+PKajcVmFnv/KQDrWpv7H0W3OI68Yk1Hpo3HOObpFb7Dfzt39IpdgYhx26cTgf4nO 9Tr5fgmBDMeoZXXRz/WoANWt4turopiXTTj7q+lBC6SBi4m3uGUbQ= From: Kai Tetzlaff To: Eli Zaretskii Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: <83v8l2o212.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 19 Jan 2023 19:41:13 +0200") References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> Date: Sun, 22 Jan 2023 03:12:00 +0100 Message-ID: <87mt6buxlb.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: C92C16C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.83 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-3.00)[-1.000]; GENERIC_REPUTATION(-0.73)[-0.726638554566]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:+,6:+,7:+]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; ARC_NA(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_FROM(0.00)[bug]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: multipart/mixed; boundary="==-=-=" --==-=-= Content-Type: text/plain Content-Disposition: inline I've now added an additional patch which automatically handles unix/dos eol-types when downloading/uploading sieve scripts. So far, if a script downloaded from the server contained CRLF EOLs, the script buffer was full of '^M's. With the additional patch (0005-Autodetect-eol-type-of-sieve-manage-scripts), the EOL type is detected and used for decoding during script download (and subsequently also for encoding during upload). For that, I changed the interface between 'sieve-upload' (in sieve.el), and 'manage-sieve-putscript' (plus 'sieve-manage-decode' and 'sieve-manage-send' in sieve-manage.el). Instead of transferring the script data as a string, the functions are now using the actual script buffer. The eol-type detection is done in the new function 'sieve-manage--guess-buffer-coding-system'. But I would assume, that this functionality already exists somewhere else. E.g. 'find-file' must do a similar, much more detailed analysis. However, that seems to happen in the C implementation, so it's not directly usable in sieve-manage. Or am I missing something? --==-=-=-- --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Fix-bug-in-sieve-manage-append-to-log-and-do-some-re.patch >From 097edd5192164578e96db75c0b7f76dc340121ca Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:08:07 +0100 Subject: [PATCH 1/5] Fix bug in sieve-manage--append-to-log and do some refactoring - simplify buffer creation and setup for process and log buffers - handle all encoding tasks in `sieve-manage-send' (and thus simplify `sieve-manage-putscript') - add/update doc-strings - update some comments * lisp/net/sieve-manage.el (sieve-manage-coding-system-for-read) (sieve-manage-coding-system-for-write): Remove unused constants. (sieve-manage-encode): Change misleading argument name (utf8-string -> str). (sieve-manage--coding-system): New constant. (sieve-manage-encode): Use `sieve-manage--coding-system', rename arguments, update doc string. (sieve-manage--set-buffer-maybe-append-text): New function. (sieve-manage-open-server): Change :coding property of `open-network-stream' from 'raw-text-unix to 'binary. (sieve-manage--append-to-log): Use `sieve-manage--set-buffer-maybe-append-text' to fix log buffer creation. (sieve-manage-decode) (sieve-manage-make-process-buffer): Use `sieve-manage--set-buffer-maybe-append-text'. (sieve-manage-send): Handle all encoding steps (including the 'literal-c2s' encoding previously done in `sieve-manage-putscript'). (sieve-manage-putscript): Move all encoding steps to `sieve-manage-send'. --- lisp/net/sieve-manage.el | 117 ++++++++++++++++++++++++++------------- 1 file changed, 79 insertions(+), 38 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..bc8ba25f400 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -58,7 +58,7 @@ ;; ;; References: ;; -;; draft-martin-managesieve-02.txt, +;; RFC5804, ;; "A Protocol for Remotely Managing Sieve Scripts", ;; by Tim Martin. ;; @@ -145,6 +145,15 @@ sieve-manage-ignore-starttls :type 'boolean) ;; Internal variables: +(defconst sieve-manage--coding-system 'utf-8-unix + "Use \\='utf-8-unix coding system for (network) communication. + +Defines the coding system used for the internal (process, log) +buffers and the network stream created to communicate with the +managesieve server. Using \\='utf-8-unix encoding corresponds to +the use of UTF-8 in RFC5804 (managesieve). The explicit use of +\\='-unix\\=' avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") +sequences intact).") (defconst sieve-manage-local-variables '(sieve-manage-server sieve-manage-port @@ -154,8 +163,6 @@ sieve-manage-local-variables sieve-manage-client-eol sieve-manage-server-eol sieve-manage-capability)) -(defconst sieve-manage-coding-system-for-read 'binary) -(defconst sieve-manage-coding-system-for-write 'binary) (defvar sieve-manage-stream nil) (defvar sieve-manage-auth nil) (defvar sieve-manage-server nil) @@ -167,20 +174,45 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions +(defun sieve-manage--set-buffer-maybe-append-text (buffer-name + &rest args) + "Append ARGS to buffer named BUFFER-NAME and return buffer. + +To be used with process and log buffers. When the buffer doesn't +exist, it gets created and configure as follows: +- set coding system to 'sieve-manage--coding-system', +- set buffer to single-byte mode, +- set `after-change-functions' to nil to avoid those + functions messing with buffer content, +- disable undo (to save a bit of memory and improve + performance). + +ARGS can be a nil, a string or a list of strings. If no +ARGS are provided, the content of buffer will not be +modified." + (let* ((existing-buffer (get-buffer buffer-name)) + (buffer (or existing-buffer + (get-buffer-create buffer-name)))) + (with-current-buffer buffer + (unless existing-buffer + (set-buffer-file-coding-system sieve-manage--coding-system) + (set-buffer-multibyte nil) + (setq-local after-change-functions nil) + (buffer-disable-undo)) + (when args + (goto-char (point-max)) + (apply #'insert args))) + buffer)) + (defun sieve-manage--append-to-log (&rest args) - "Append ARGS to sieve-manage log buffer. + "Append ARGS to `sieve-manage-log' buffer. -ARGS can be a string or a list of strings. -The buffer to use for logging is specifified via -`sieve-manage-log'. If it is nil, logging is disabled." +If `sieve-manage-log' is nil, logging is disabled. See also +`sieve-manage--set-buffer-maybe-append-text'." (when sieve-manage-log - (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) - (set-buffer-multibyte nil) - (buffer-disable-undo))) - (goto-char (point-max)) - (apply #'insert args)))) + (apply #'sieve-manage--set-buffer-maybe-append-text + sieve-manage-log + args))) (defun sieve-manage--message (format-string &rest args) "Wrapper around `message' which also logs to sieve manage log. @@ -202,28 +234,28 @@ sieve-manage--error (sieve-manage--append-to-log msg "\n") (error msg))) -(defun sieve-manage-encode (utf8-string) - "Convert UTF8-STRING to managesieve protocol octets." - (encode-coding-string utf8-string 'raw-text t)) +(defun sieve-manage-encode (str) + "Convert STR to managesieve protocol octets." + (encode-coding-string str sieve-manage--coding-system t)) (defun sieve-manage-decode (octets &optional buffer) - "Convert managesieve protocol OCTETS to utf-8 string. + "Convert managesieve protocol OCTETS to UTF-8 string. If optional BUFFER is non-nil, insert decoded string into BUFFER." (when octets ;; eol type unix is required to preserve "\r\n" - (decode-coding-string octets 'utf-8-unix t buffer))) + (decode-coding-string octets + sieve-manage--coding-system + t buffer))) (defun sieve-manage-make-process-buffer () - (with-current-buffer - (generate-new-buffer (format " *sieve %s:%s*" - sieve-manage-server - sieve-manage-port)) - (mapc #'make-local-variable sieve-manage-local-variables) - (set-buffer-multibyte nil) - (setq-local after-change-functions nil) - (buffer-disable-undo) - (current-buffer))) + (let ((buffer (sieve-manage--set-buffer-maybe-append-text + (format " *sieve %s:%s*" + sieve-manage-server + sieve-manage-port)))) + (with-current-buffer buffer + (mapc #'make-local-variable sieve-manage-local-variables)) + buffer)) (defun sieve-manage-erase (&optional p buffer) (with-current-buffer (or buffer (current-buffer)) @@ -244,8 +276,7 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream - ;; eol type unix is required to preserve "\r\n" - :coding 'raw-text-unix + :coding 'binary :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" @@ -480,9 +511,7 @@ sieve-manage-havespace (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name - (length (sieve-manage-encode content)) - sieve-manage-client-eol content)) + (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) (sieve-manage-parse-okno))) (defun sieve-manage-deletescript (name &optional buffer) @@ -609,11 +638,23 @@ sieve-manage-parse-listscripts data rsp))) -(defun sieve-manage-send (cmdstr) - (setq cmdstr (sieve-manage-encode - (concat cmdstr sieve-manage-client-eol))) - (sieve-manage--append-to-log cmdstr) - (process-send-string sieve-manage-process cmdstr)) +(defun sieve-manage-send (command &optional payload-str) + "Send COMMAND with optional PAYLOAD-STR. + +If non-nil, PAYLOAD-STR will be appended to COMMAND using the +\\='literal-s2c\\' representation according to RFC5804." + (let ((encoded (when payload-str (sieve-manage-encode payload-str))) + literal-c2s cmdstr) + (when encoded + (setq literal-c2s (format " {%d+}%s%s" + (length encoded) + sieve-manage-client-eol + encoded))) + (setq cmdstr (concat (sieve-manage-encode command) + literal-c2s + sieve-manage-client-eol)) + (sieve-manage--append-to-log cmdstr) + (process-send-string sieve-manage-process cmdstr))) (provide 'sieve-manage) -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-Handle-BYE-in-sieve-manage-server-responses.patch >From 5e9c5c14bc2c115b0d36d06bbdd39f6f90687e3b Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:52:43 +0100 Subject: [PATCH 2/5] Handle BYE in sieve-manage server responses * etc/NEWS: Mention the support for BYE. * lisp/net/sieve-manage.el (sieve-manage-regex-oknobye): New function. (sieve-manage-parse-oknobye): Renamed from `sieve-manage-parse-okno'. (sieve-manage-open-server) (sieve-sasl-auth) (sieve-manage-listscripts) (sieve-manage-putscript) (sieve-manage-deletescript) (sieve-manage-getscript) (sieve-manage-setactive) (sieve-manage-is-okno) (sieve-manage-parse-oknobye) (sieve-manage-parse-parse-string) (sieve-manage-parse-parse-crlf): Use `sieve-manage-regex-oknobye' to handle BYE in addition to OK and NO. --- etc/NEWS | 12 ++++++++++++ lisp/net/sieve-manage.el | 37 ++++++++++++++++++++----------------- 2 files changed, 32 insertions(+), 17 deletions(-) diff --git a/etc/NEWS b/etc/NEWS index 4851802716a..f8e4aed6703 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -181,6 +181,18 @@ the new argument NEW-BUFFER non-nil, it will use a new buffer instead. Interactively, invoke 'eww-open-file' with a prefix argument to activate this behavior. +** sieve-manage + +--- +*** Support handling of BYE responses from managesieve servers. +The managesieve client in sieve-manage now handles BYE responses sent +be the server (in addition to OK and NO). This makes the +implementation more robust in case of e.g. timeouts and authentication +failures. + +Note: The special case of a REFERRAL/BYE responses is still not +handled by the client (s. RFC5804 for more details). + * New Modes and Packages in Emacs 30.1 diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index bc8ba25f400..de5c3cd1386 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -278,8 +278,8 @@ sieve-manage-open-server :type stream :coding 'binary :capability-command "CAPABILITY\r\n" - :end-of-command "^\\(OK\\|NO\\).*\n" - :success "^OK.*\n" + :end-of-command "^\\(OK\\|NO\\|BYE\\).*\r\n" + :success "^OK.*\r\n" :return-list t :starttls-function (lambda (capabilities) @@ -338,7 +338,7 @@ sieve-sasl-auth (setq rsp nil) (goto-char (match-end 0)) rsp)) - (setq rsp (sieve-manage-is-okno)))) + (setq rsp (sieve-manage-is-oknobye)))) (accept-process-output sieve-manage-process 1) (goto-char (point-min))) (sieve-manage-erase) @@ -507,17 +507,17 @@ sieve-manage-listscripts (defun sieve-manage-havespace (name size &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "DELETESCRIPT \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -525,17 +525,22 @@ sieve-manage-getscript (sieve-manage-decode (sieve-manage-parse-string) output-buffer) (sieve-manage-parse-crlf) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "SETACTIVE \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) ;; Protocol parsing routines +(defun sieve-manage-regexp-oknobye () + "Return regexp for managesieve 'response-oknobye'." + (concat + "^\\(OK\\|NO\\|BYE\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" + sieve-manage-server-eol)) (defun sieve-manage-wait-for-answer () - (let ((pattern "^\\(OK\\|NO\\).*\n") + (let ((pattern (sieve-manage-regexp-oknobye)) pos) (while (not pos) (setq pos (search-forward-regexp pattern nil t)) @@ -553,10 +558,8 @@ sieve-manage-ok-p (defun sieve-manage-no-p (rsp) (string= (downcase (or (car-safe rsp) "")) "no")) -(defun sieve-manage-is-okno () - (when (looking-at (concat - "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" - sieve-manage-server-eol)) +(defun sieve-manage-is-oknobye () + (when (looking-at (sieve-manage-regexp-oknobye)) (let ((status (match-string 1)) (resp-code (match-string 3)) (response (match-string 5))) @@ -565,12 +568,12 @@ sieve-manage-is-okno (setq response (sieve-manage-is-string))) (list status resp-code response)))) -(defun sieve-manage-parse-okno () +(defun sieve-manage-parse-oknobye () (let (rsp) (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-okno))) + (setq rsp (sieve-manage-is-oknobye))) (sieve-manage-erase) rsp)) @@ -604,7 +607,7 @@ sieve-manage-parse-string (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) (unless (setq rsp (sieve-manage-is-string)) - (when (sieve-manage-no-p (sieve-manage-is-okno)) + (when (sieve-manage-no-p (sieve-manage-is-oknobye)) ;; simple `error' is enough since `sieve-manage-erase' ;; already adds the server response to the log (error (sieve-manage-erase))))) @@ -618,7 +621,7 @@ sieve-manage-parse-crlf (defun sieve-manage-parse-listscripts () (let (tmp rsp data) (while (null rsp) - (while (null (or (setq rsp (sieve-manage-is-okno)) + (while (null (or (setq rsp (sieve-manage-is-oknobye)) (setq tmp (sieve-manage-decode (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) -- 2.39.0 --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0003-Add-test-lisp-net-sieve-manage-tests.el.patch Content-Transfer-Encoding: quoted-printable >From 9e84b0a7206ba6796e25ebd1045646a48907cbe7 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Tue, 22 Mar 2022 20:48:09 +0100 Subject: [PATCH 3/5] Add test/lisp/net/sieve-manage-tests.el * test/lisp/net/sieve-manage-tests.el: (mk-literal-s2c) (mk-rsp-oknobye) (mk-rsp-getscript-ok) (managesieve-getscript): Test helper functions. (ert/managesieve-getscript-multibyte): Test `sieve-manage-getscript' with script names containing multibyte characters. --- test/lisp/net/sieve-manage-tests.el | 104 ++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 test/lisp/net/sieve-manage-tests.el diff --git a/test/lisp/net/sieve-manage-tests.el b/test/lisp/net/sieve-mana= ge-tests.el new file mode 100644 index 00000000000..010c9071608 --- /dev/null +++ b/test/lisp/net/sieve-manage-tests.el @@ -0,0 +1,104 @@ +;;; sieve-manage-tests.el --- tests for sieve-manage.el -*- lexical-= binding: t; -*- + +;; Copyright (C) 2022 Free Software Foundation, Inc. + +;; Author: Kai Tetzlaff + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see . + +;;; Commentary: + +;;; Code: + +(require 'ert) +(require 'sieve-manage) + +(defvar sieve-script-multibyte-unix + "if body :matches \"=C3=A4\" { stop; }\n" + "Simple multibyte sieve script.") + +(defun mk-literal-s2c (string) + "Encode STRING to managesieve 'literal-s2c'." + (let ((data (sieve-manage-encode string))) + (concat (format "{%d}\r\n" (length data)) + data))) + +(defun mk-rsp-oknobye (type &optional resp-code string) + "Encode TYPE, RESP-CODE and STRING to managesieve 'response-oknobye'." + (when (memq type '(OK NO BYE)) + (concat + (mapconcat #'identity + (delq nil + (list (symbol-name type) + (when resp-code + (format "(%s)" resp-code)) + (when string + (format "\"%s\"" + (sieve-manage-encode string))))) + " ") + "\r\n"))) +;; (mk-rsp-oknobye 'OK nil "Getscript completed.") +;; (mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") + +(defun mk-rsp-getscript-ok (script) + "Encode SCRIPT to managesieve 'response-getscript'." + (concat (mk-literal-s2c script) + "\r\n" + (mk-rsp-oknobye 'OK "Getscript completed."))) +;; (mk-rsp-getscript-ok sieve-script-multibyte-unix) + +(defun managesieve-getscript (script) + "Simulate managesieve getscript response to test +`sieve-manage-getscript' function." + (let* ((script-name "test.sieve") + ;; `sieve-manage-server' and `sieve-manage-port' are used in + ;; `sieve-manage-make-process-buffer' + (sieve-manage-server) + (sieve-manage-port "sieve") + (sieve-buffer (sieve-manage-make-process-buffer)) + (output-buffer (generate-new-buffer script-name))) + ;; use cl-letf to mock some functions in call chain of + ;; sieve-manage-getscript + (cl-letf (((symbol-function 'sieve-manage-send) + ;; simulate sieve server response with a single + ;; multibyte character `=C3=A4` + (lambda (_) + (with-current-buffer sieve-buffer + (goto-char (point-min)) + (insert (mk-rsp-getscript-ok script))))) + ((symbol-function 'accept-process-output) + (lambda (&optional _ _ _ _) nil)) + ((symbol-function 'get-buffer-process) (lambda (_) nil))) + ;; extract sieve script from sieve-buffer and put it into + ;; output-buffer + (sieve-manage-getscript script-name output-buffer sieve-buffer) + ;; extract script from output-buffer and return it as a string + (let ((script (with-current-buffer output-buffer + (set-buffer-modified-p nil) + (buffer-string)))) + ;; cleanup + (kill-buffer sieve-buffer) + (kill-buffer output-buffer) + (when (get-buffer sieve-manage-log) + (kill-buffer sieve-manage-log)) + ;; return script + script)))) + +(ert-deftest ert/managesieve-getscript-multibyte () + (should (string=3D sieve-script-multibyte-unix + (managesieve-getscript sieve-script-multibyte-unix)))) + +;;; sieve-manage-tests.el ends here --=20 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0004-Some-minor-fixes-in-lisp-net-sieve.el.patch >From 4889ffcd509f08e6fabb073dd7764e2cd5ffc916 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 04:06:22 +0100 Subject: [PATCH 4/5] Some minor fixes in lisp/net/sieve.el * lisp/net/sieve.el (sieve-next-line) (sieve-prev-line): Handle situations where point in `sieve-buffer' is either before or after the list of server side scripts. (sieve-server-script-list): New variable. (sieve-refresh-scriptlist) (sieve-upload): Use `sieve-server-script-list' to make sure that local list of server side scripts in`sieve-buffer' is up-to-date after uploading a (new) script. --- lisp/net/sieve.el | 38 +++++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 13 deletions(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index fbd07dee27c..12a85e89d7e 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -95,6 +95,8 @@ sieve-template (defvar sieve-manage-buffer nil) (defvar sieve-buffer-header-end nil) +(defvar sieve-server-script-list nil + "Current list of server-side sieve scripts.") (defvar sieve-buffer-script-name nil "The real script name of the buffer.") (make-local-variable 'sieve-buffer-script-name) @@ -235,23 +237,28 @@ sieve-next-line (interactive) (unless arg (setq arg 1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "End of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "End of list")) + (goto-char (next-overlay-change sieve-buffer-header-end)))) (defun sieve-prev-line (&optional arg) (interactive) (unless arg (setq arg -1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "Beginning of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "Beginning of list")) + (goto-char (previous-overlay-change (point-max))) + (beginning-of-line))) (defun sieve-help () "Display help for various sieve commands." @@ -319,6 +326,9 @@ sieve-refresh-scriptlist (let* ((scripts (sieve-manage-listscripts sieve-manage-buffer)) (count (length scripts)) (keys (substitute-command-keys "\\[sieve-edit-script]"))) + (setq sieve-server-script-list + (mapcar (lambda (elt) (if (consp elt) (cdr elt) elt)) + scripts)) (insert (if (null scripts) (format @@ -361,7 +371,9 @@ sieve-upload (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." - (substitute-command-keys "\\[sieve-manage]")))) + (substitute-command-keys "\\[sieve-manage]")) + (when (not (member script-name sieve-server-script-list)) + (sieve-refresh-scriptlist)))) (set-buffer-modified-p nil)))) ;;;###autoload -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0005-Autodetect-eol-type-of-sieve-manage-scripts.patch >From cde0b4bdcc159edc07ed367f3682f80e5f834725 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Sun, 22 Jan 2023 01:06:57 +0100 Subject: [PATCH 5/5] Autodetect eol-type of sieve(-manage) scripts When using `sieve-manage-getscript' to download a sieve script, sieve-manage now automatically detects the eol-type (either 'utf-8-unix or 'utf-8-dos), uses it to decode the script data and sets `buffer-file-coding-system' accordingly. This gets rid of '^M' in sieve script buffers (for scripts which use CRLF type EOLs). The same eol-type is then used to encode the script during upload with `sieve-manage-putscript'. * lisp/net/sieve-manage.el (sieve-manage--guess-buffer-coding-system): New function which analyzes the eol-type of the first couple of lines of a downloaded script to make a best guess and returns either 'utf-8-unix or 'utf-8-dos. (sieve-manage-decode): Use `sieve-manage--guess-buffer-coding-system' to decode downloaded script data with the correct coding-system and sets the `buffer-file-coding-system' of the resulting sieve script buffer. (sieve-manage-putscript): Now takes a sieve script buffer (instead of a string) argument and forwards it to `sieve-manage-send'. (sieve-manage-send): Now also uses a (payload-)buffer instead of a string. The `buffer-file-coding-system' of the buffer is then used when encoding the payload in order to use the correct eol-type. * lisp/net/sieve.el: (sieve-upload): Adapt to changed argument type of `sieve-manage-putscript'. * etc/NEWS: Add a short description of the changes. --- etc/NEWS | 10 +++++++ lisp/net/sieve-manage.el | 63 +++++++++++++++++++++++++++++++--------- lisp/net/sieve.el | 6 ++-- 3 files changed, 62 insertions(+), 17 deletions(-) diff --git a/etc/NEWS b/etc/NEWS index f8e4aed6703..e3791171220 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -193,6 +193,16 @@ failures. Note: The special case of a REFERRAL/BYE responses is still not handled by the client (s. RFC5804 for more details). +--- +*** Autodetect eol-type of downloaded sieve scripts. +When a downloaded script contained CRLF type EOLs, they caused '^M's +to appear in the sieve script edit buffer. To avoid that, the +eol-type of sieve scripts is now detected during download via +'sieve-manage-getscript', used when decoding the data and stored in +'buffer-file-coding-system' of the script buffer. The +'buffer-file-coding-system' is then also used for encoding during +upload by 'sieve-manage-putscript'. + * New Modes and Packages in Emacs 30.1 diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index de5c3cd1386..7c680007042 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -238,15 +238,43 @@ sieve-manage-encode "Convert STR to managesieve protocol octets." (encode-coding-string str sieve-manage--coding-system t)) +(defun sieve-manage--guess-buffer-coding-system (&optional buffer) + "Return the coding system to be use for (sieve script) BUFFER. + +Since RFC5804 requires scripts to be encoded as UTF-8, the +returned coding system is either \\='utf-8-unix or \\='utf-8-dos." + (with-current-buffer (or buffer (current-buffer)) + (let ((pos (point)) + (max-lines 10) + (line 0) + (crlf-count 0)) + (goto-char (point-min)) + (while (and (> max-lines line) (not (eobp))) + (when (= #x0d (char-before (pos-eol))) + (cl-incf crlf-count)) + (let ((eol (pos-eol))) + (when (> (goto-char (+ eol 1)) eol) + (cl-incf line)))) + (goto-char pos) + (if (> crlf-count (/ line 2)) + 'utf-8-dos + 'utf-8-unix)))) + (defun sieve-manage-decode (octets &optional buffer) "Convert managesieve protocol OCTETS to UTF-8 string. If optional BUFFER is non-nil, insert decoded string into BUFFER." (when octets - ;; eol type unix is required to preserve "\r\n" - (decode-coding-string octets - sieve-manage--coding-system - t buffer))) + (if buffer + (with-current-buffer buffer + (insert octets) + (let ((coding-system + (sieve-manage--guess-buffer-coding-system))) + (set-buffer-file-coding-system coding-system) + (decode-coding-region (point-min) (point-max) + coding-system))) + (decode-coding-string + octets sieve-manage--coding-system t)))) (defun sieve-manage-make-process-buffer () (let ((buffer (sieve-manage--set-buffer-maybe-append-text @@ -509,9 +537,9 @@ sieve-manage-havespace (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) (sieve-manage-parse-oknobye))) -(defun sieve-manage-putscript (name content &optional buffer) - (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) +(defun sieve-manage-putscript (name script-buffer &optional process-buffer) + (with-current-buffer (or process-buffer (current-buffer)) + (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) script-buffer) (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) @@ -641,13 +669,20 @@ sieve-manage-parse-listscripts data rsp))) -(defun sieve-manage-send (command &optional payload-str) - "Send COMMAND with optional PAYLOAD-STR. - -If non-nil, PAYLOAD-STR will be appended to COMMAND using the -\\='literal-s2c\\' representation according to RFC5804." - (let ((encoded (when payload-str (sieve-manage-encode payload-str))) - literal-c2s cmdstr) +(defun sieve-manage-send (command &optional payload-buffer) + "Send COMMAND with optional string from PAYLOAD-BUFFER. + +If non-nil, the content of PAYLOAD-BUFFER will be appended to +COMMAND using the \\='literal-s2c\\=' representation according to RFC5804." + (let* ((encoded (when (and payload-buffer + (> (buffer-size payload-buffer) 0)) + (with-current-buffer payload-buffer + (encode-coding-region + (point-min) (point-max) + (buffer-local-value 'buffer-file-coding-system + payload-buffer) + t)))) + cmdstr literal-c2s) (when encoded (setq literal-c2s (format " {%d+}%s%s" (length encoded) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index 12a85e89d7e..2108732c5dd 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -361,13 +361,13 @@ sieve-upload (interactive) (when (or (get-buffer sieve-buffer) (save-current-buffer (call-interactively 'sieve-manage))) - (let ((script (buffer-string)) + (let ((script-buffer (current-buffer)) (script-name (file-name-sans-extension (buffer-name))) err) (with-current-buffer (get-buffer sieve-buffer) - (setq err (sieve-manage-putscript + (setq err (sieve-manage-putscript (or name sieve-buffer-script-name script-name) - script sieve-manage-buffer)) + script-buffer sieve-manage-buffer)) (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." -- 2.39.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Sun Jan 22 19:59:28 2023 Received: (at 54154) by debbugs.gnu.org; 23 Jan 2023 00:59:28 +0000 Received: from localhost ([127.0.0.1]:52854 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJlBG-00079x-0g for submit@debbugs.gnu.org; Sun, 22 Jan 2023 19:59:28 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:40205) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJlBB-00079f-CY for 54154@debbugs.gnu.org; Sun, 22 Jan 2023 19:59:25 -0500 Received: from mail.tetzco.de (unknown [IPv6:2a02:810d:1380:7900::9]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id 1C98BBD078; Mon, 23 Jan 2023 01:59:13 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id D7B276C00B2; Mon, 23 Jan 2023 02:19:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674436748; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6lm1MnSYrlltuBMPA2gnQTpxs9LlD7DCQp2nyc7gzmo=; b=Qy63e8dzecfQKhvB41qmSKlibuUIo7VLqkc5vUkd7S4KPcxQGrjKrGZS9BTZpoM2UYeeuz 5cmuHMsjmk+T6e3BxpGOya8EWKHwetJCDMqYPnpXNJ/EGBts+j7LUd6zP9RsiPcNfbGTXm mEvn0ccg4weLikwgYri5mfyTUIdWhrxaBfd1vqzwoNCmpsxrbqv9jNfr8u0sD4AuymlymU 1zXsTexXx+CC61jLTwbd5h9zNI11MlZHbFPfNiAK7z0HXvCr0/g269wzQGM6xtDCxWnkL/ d8JPy1LG1LVDgUTBGgn8KB3HOwXePle44e6G+NGeIjlFWHYYm5oOpTsEZZLgOBXBDXRvp7 KMLVv384na1SHAaPlfx7ZXFtBl/XbWDS4Bd+0uTY+JAjQ3oOLQz9LaIuBYJJwz/rlcMe69 Lr11qAKyjeCXKgRIQ7gTa1gO6ASkLDh1cMck1zc/L7fo09GeiU48VDHzzKl5zvUkq+sHOM 0kg7fjTj92FuA5jw27AkDQXiI+xJ23wkggOzp79ddHjP7oqg409InX8kYEdBBryoWTuXPP O/6Sg+eRNgRpRkK8YtlVbWKtFXSnp8lz4Yv27o3jqS6gDj8vvJeL1xFnd10IEm6pHxSV0L W+3N8CYfTWcFwqFsNdWEuj4ySG0wfUkpNI1OGEAYBcFpA426tkNwg= From: Kai Tetzlaff To: Eli Zaretskii Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: <83v8l2o212.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 19 Jan 2023 19:41:13 +0200") References: <87wnhj5nbk.fsf@tetzco.de> <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> Date: Mon, 23 Jan 2023 01:59:09 +0100 Message-ID: <878rhut6aq.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: D7B276C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.83 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-3.00)[-0.999]; GENERIC_REPUTATION(-0.73)[-0.72664186213518]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:+,6:+,7:+]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; ARC_NA(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_FROM(0.00)[bug]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Just a quick update (also answering my own question(s) below): Regarding the status of this bug report: The fix for emacs-29 which was initially intended to be emacs-29 only has (inadvertently?) found its way to master. So I think we can close the report. The discussion about the additional changes I made (I'm still working on that s. below) could be moved elsewhere (emacs-devel?). Even though, from my side there is no urgent need for that. I really appreciate the feedback I got from Eli and hope to get some more. > I've now added an additional patch which automatically handles unix/dos > eol-types when downloading/uploading sieve scripts. So far, if a script > downloaded from the server contained CRLF EOLs, the script buffer was > full of '^M's. With the additional patch > (0005-Autodetect-eol-type-of-sieve-manage-scripts), the EOL type is > detected and used for decoding during script download (and subsequently > also for encoding during upload). > > For that, I changed the interface between 'sieve-upload' (in sieve.el), > and 'manage-sieve-putscript' (plus 'sieve-manage-decode' and > 'sieve-manage-send' in sieve-manage.el). Instead of transferring the > script data as a string, the functions are now using the actual script > buffer. > > The eol-type detection is done in the new function > 'sieve-manage--guess-buffer-coding-system'. But I would assume, that > this functionality already exists somewhere else. E.g. 'find-file' must > do a similar, much more detailed analysis. However, that seems to happen > in the C implementation, so it's not directly usable in sieve-manage. Or > am I missing something? In the meantime, I found `detect-coding-region'/`detect-coding-string' which in combination with `coding-system-eol-type' do exactly what I was missing. I've now started to refactor the encoding and sending functions in sieve-manage.el with the intent to improve the readability and testability of the code. I'm also adding some more tests. These additional changes are in yet another patch (0006-WiP-new-encode-tested-OK.patch). I also added NEWS entries to patches 0002-Handle-BYE... and 0005-Autodetect-eol-type... (these were already present in the patches of the previous mail). --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Fix-bug-in-sieve-manage-append-to-log-and-do-some-re.patch >From 8b659e704a6b39b586168a6851923fcfd6035d8e Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 28 Feb 2022 11:08:07 +0100 Subject: [PATCH 1/6] Fix bug in sieve-manage--append-to-log and do some refactoring - simplify buffer creation and setup for process and log buffers - handle all encoding tasks in `sieve-manage-send' (and thus simplify `sieve-manage-putscript') - add/update doc-strings - update some comments * lisp/net/sieve-manage.el (sieve-manage-coding-system-for-read) (sieve-manage-coding-system-for-write): Remove unused constants. (sieve-manage-encode): Change misleading argument name (utf8-string -> str). (sieve-manage--coding-system): New constant. (sieve-manage-encode): Use `sieve-manage--coding-system', rename arguments, update doc string. (sieve-manage--set-buffer-maybe-append-text): New function. (sieve-manage-open-server): Change :coding property of `open-network-stream' from 'raw-text-unix to 'binary. (sieve-manage--append-to-log): Use `sieve-manage--set-buffer-maybe-append-text' to fix log buffer creation. (sieve-manage-decode) (sieve-manage-make-process-buffer): Use `sieve-manage--set-buffer-maybe-append-text'. (sieve-manage-send): Handle all encoding steps (including the 'literal-c2s' encoding previously done in `sieve-manage-putscript'). (sieve-manage-putscript): Move all encoding steps to `sieve-manage-send'. --- lisp/net/sieve-manage.el | 117 ++++++++++++++++++++++++++------------- 1 file changed, 79 insertions(+), 38 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..bc8ba25f400 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -58,7 +58,7 @@ ;; ;; References: ;; -;; draft-martin-managesieve-02.txt, +;; RFC5804, ;; "A Protocol for Remotely Managing Sieve Scripts", ;; by Tim Martin. ;; @@ -145,6 +145,15 @@ sieve-manage-ignore-starttls :type 'boolean) ;; Internal variables: +(defconst sieve-manage--coding-system 'utf-8-unix + "Use \\='utf-8-unix coding system for (network) communication. + +Defines the coding system used for the internal (process, log) +buffers and the network stream created to communicate with the +managesieve server. Using \\='utf-8-unix encoding corresponds to +the use of UTF-8 in RFC5804 (managesieve). The explicit use of +\\='-unix\\=' avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") +sequences intact).") (defconst sieve-manage-local-variables '(sieve-manage-server sieve-manage-port @@ -154,8 +163,6 @@ sieve-manage-local-variables sieve-manage-client-eol sieve-manage-server-eol sieve-manage-capability)) -(defconst sieve-manage-coding-system-for-read 'binary) -(defconst sieve-manage-coding-system-for-write 'binary) (defvar sieve-manage-stream nil) (defvar sieve-manage-auth nil) (defvar sieve-manage-server nil) @@ -167,20 +174,45 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions +(defun sieve-manage--set-buffer-maybe-append-text (buffer-name + &rest args) + "Append ARGS to buffer named BUFFER-NAME and return buffer. + +To be used with process and log buffers. When the buffer doesn't +exist, it gets created and configure as follows: +- set coding system to 'sieve-manage--coding-system', +- set buffer to single-byte mode, +- set `after-change-functions' to nil to avoid those + functions messing with buffer content, +- disable undo (to save a bit of memory and improve + performance). + +ARGS can be a nil, a string or a list of strings. If no +ARGS are provided, the content of buffer will not be +modified." + (let* ((existing-buffer (get-buffer buffer-name)) + (buffer (or existing-buffer + (get-buffer-create buffer-name)))) + (with-current-buffer buffer + (unless existing-buffer + (set-buffer-file-coding-system sieve-manage--coding-system) + (set-buffer-multibyte nil) + (setq-local after-change-functions nil) + (buffer-disable-undo)) + (when args + (goto-char (point-max)) + (apply #'insert args))) + buffer)) + (defun sieve-manage--append-to-log (&rest args) - "Append ARGS to sieve-manage log buffer. + "Append ARGS to `sieve-manage-log' buffer. -ARGS can be a string or a list of strings. -The buffer to use for logging is specifified via -`sieve-manage-log'. If it is nil, logging is disabled." +If `sieve-manage-log' is nil, logging is disabled. See also +`sieve-manage--set-buffer-maybe-append-text'." (when sieve-manage-log - (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) - (set-buffer-multibyte nil) - (buffer-disable-undo))) - (goto-char (point-max)) - (apply #'insert args)))) + (apply #'sieve-manage--set-buffer-maybe-append-text + sieve-manage-log + args))) (defun sieve-manage--message (format-string &rest args) "Wrapper around `message' which also logs to sieve manage log. @@ -202,28 +234,28 @@ sieve-manage--error (sieve-manage--append-to-log msg "\n") (error msg))) -(defun sieve-manage-encode (utf8-string) - "Convert UTF8-STRING to managesieve protocol octets." - (encode-coding-string utf8-string 'raw-text t)) +(defun sieve-manage-encode (str) + "Convert STR to managesieve protocol octets." + (encode-coding-string str sieve-manage--coding-system t)) (defun sieve-manage-decode (octets &optional buffer) - "Convert managesieve protocol OCTETS to utf-8 string. + "Convert managesieve protocol OCTETS to UTF-8 string. If optional BUFFER is non-nil, insert decoded string into BUFFER." (when octets ;; eol type unix is required to preserve "\r\n" - (decode-coding-string octets 'utf-8-unix t buffer))) + (decode-coding-string octets + sieve-manage--coding-system + t buffer))) (defun sieve-manage-make-process-buffer () - (with-current-buffer - (generate-new-buffer (format " *sieve %s:%s*" - sieve-manage-server - sieve-manage-port)) - (mapc #'make-local-variable sieve-manage-local-variables) - (set-buffer-multibyte nil) - (setq-local after-change-functions nil) - (buffer-disable-undo) - (current-buffer))) + (let ((buffer (sieve-manage--set-buffer-maybe-append-text + (format " *sieve %s:%s*" + sieve-manage-server + sieve-manage-port)))) + (with-current-buffer buffer + (mapc #'make-local-variable sieve-manage-local-variables)) + buffer)) (defun sieve-manage-erase (&optional p buffer) (with-current-buffer (or buffer (current-buffer)) @@ -244,8 +276,7 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream - ;; eol type unix is required to preserve "\r\n" - :coding 'raw-text-unix + :coding 'binary :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" @@ -480,9 +511,7 @@ sieve-manage-havespace (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name - (length (sieve-manage-encode content)) - sieve-manage-client-eol content)) + (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) (sieve-manage-parse-okno))) (defun sieve-manage-deletescript (name &optional buffer) @@ -609,11 +638,23 @@ sieve-manage-parse-listscripts data rsp))) -(defun sieve-manage-send (cmdstr) - (setq cmdstr (sieve-manage-encode - (concat cmdstr sieve-manage-client-eol))) - (sieve-manage--append-to-log cmdstr) - (process-send-string sieve-manage-process cmdstr)) +(defun sieve-manage-send (command &optional payload-str) + "Send COMMAND with optional PAYLOAD-STR. + +If non-nil, PAYLOAD-STR will be appended to COMMAND using the +\\='literal-s2c\\' representation according to RFC5804." + (let ((encoded (when payload-str (sieve-manage-encode payload-str))) + literal-c2s cmdstr) + (when encoded + (setq literal-c2s (format " {%d+}%s%s" + (length encoded) + sieve-manage-client-eol + encoded))) + (setq cmdstr (concat (sieve-manage-encode command) + literal-c2s + sieve-manage-client-eol)) + (sieve-manage--append-to-log cmdstr) + (process-send-string sieve-manage-process cmdstr))) (provide 'sieve-manage) -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-Handle-BYE-in-sieve-manage-server-responses.patch >From 196aaf2d7f7ebea1f5a8999970092fd80dfc8f4e Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 03:52:43 +0100 Subject: [PATCH 2/6] Handle BYE in sieve-manage server responses * etc/NEWS: Mention the support for BYE. * lisp/net/sieve-manage.el (sieve-manage-regex-oknobye): New function. (sieve-manage-parse-oknobye): Renamed from `sieve-manage-parse-okno'. (sieve-manage-open-server) (sieve-sasl-auth) (sieve-manage-listscripts) (sieve-manage-putscript) (sieve-manage-deletescript) (sieve-manage-getscript) (sieve-manage-setactive) (sieve-manage-is-okno) (sieve-manage-parse-oknobye) (sieve-manage-parse-parse-string) (sieve-manage-parse-parse-crlf): Use `sieve-manage-regex-oknobye' to handle BYE in addition to OK and NO. --- etc/NEWS | 12 ++++++++++++ lisp/net/sieve-manage.el | 37 ++++++++++++++++++++----------------- 2 files changed, 32 insertions(+), 17 deletions(-) diff --git a/etc/NEWS b/etc/NEWS index 4851802716a..f8e4aed6703 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -181,6 +181,18 @@ the new argument NEW-BUFFER non-nil, it will use a new buffer instead. Interactively, invoke 'eww-open-file' with a prefix argument to activate this behavior. +** sieve-manage + +--- +*** Support handling of BYE responses from managesieve servers. +The managesieve client in sieve-manage now handles BYE responses sent +be the server (in addition to OK and NO). This makes the +implementation more robust in case of e.g. timeouts and authentication +failures. + +Note: The special case of a REFERRAL/BYE responses is still not +handled by the client (s. RFC5804 for more details). + * New Modes and Packages in Emacs 30.1 diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index bc8ba25f400..de5c3cd1386 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -278,8 +278,8 @@ sieve-manage-open-server :type stream :coding 'binary :capability-command "CAPABILITY\r\n" - :end-of-command "^\\(OK\\|NO\\).*\n" - :success "^OK.*\n" + :end-of-command "^\\(OK\\|NO\\|BYE\\).*\r\n" + :success "^OK.*\r\n" :return-list t :starttls-function (lambda (capabilities) @@ -338,7 +338,7 @@ sieve-sasl-auth (setq rsp nil) (goto-char (match-end 0)) rsp)) - (setq rsp (sieve-manage-is-okno)))) + (setq rsp (sieve-manage-is-oknobye)))) (accept-process-output sieve-manage-process 1) (goto-char (point-min))) (sieve-manage-erase) @@ -507,17 +507,17 @@ sieve-manage-listscripts (defun sieve-manage-havespace (name size &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "DELETESCRIPT \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -525,17 +525,22 @@ sieve-manage-getscript (sieve-manage-decode (sieve-manage-parse-string) output-buffer) (sieve-manage-parse-crlf) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "SETACTIVE \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) ;; Protocol parsing routines +(defun sieve-manage-regexp-oknobye () + "Return regexp for managesieve 'response-oknobye'." + (concat + "^\\(OK\\|NO\\|BYE\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" + sieve-manage-server-eol)) (defun sieve-manage-wait-for-answer () - (let ((pattern "^\\(OK\\|NO\\).*\n") + (let ((pattern (sieve-manage-regexp-oknobye)) pos) (while (not pos) (setq pos (search-forward-regexp pattern nil t)) @@ -553,10 +558,8 @@ sieve-manage-ok-p (defun sieve-manage-no-p (rsp) (string= (downcase (or (car-safe rsp) "")) "no")) -(defun sieve-manage-is-okno () - (when (looking-at (concat - "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" - sieve-manage-server-eol)) +(defun sieve-manage-is-oknobye () + (when (looking-at (sieve-manage-regexp-oknobye)) (let ((status (match-string 1)) (resp-code (match-string 3)) (response (match-string 5))) @@ -565,12 +568,12 @@ sieve-manage-is-okno (setq response (sieve-manage-is-string))) (list status resp-code response)))) -(defun sieve-manage-parse-okno () +(defun sieve-manage-parse-oknobye () (let (rsp) (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-okno))) + (setq rsp (sieve-manage-is-oknobye))) (sieve-manage-erase) rsp)) @@ -604,7 +607,7 @@ sieve-manage-parse-string (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) (unless (setq rsp (sieve-manage-is-string)) - (when (sieve-manage-no-p (sieve-manage-is-okno)) + (when (sieve-manage-no-p (sieve-manage-is-oknobye)) ;; simple `error' is enough since `sieve-manage-erase' ;; already adds the server response to the log (error (sieve-manage-erase))))) @@ -618,7 +621,7 @@ sieve-manage-parse-crlf (defun sieve-manage-parse-listscripts () (let (tmp rsp data) (while (null rsp) - (while (null (or (setq rsp (sieve-manage-is-okno)) + (while (null (or (setq rsp (sieve-manage-is-oknobye)) (setq tmp (sieve-manage-decode (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) -- 2.39.0 --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0003-Add-test-lisp-net-sieve-manage-tests.el.patch Content-Transfer-Encoding: quoted-printable >From 86ba9f91e4c78fee72eab0c752cd9c5e78fab402 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Tue, 22 Mar 2022 20:48:09 +0100 Subject: [PATCH 3/6] Add test/lisp/net/sieve-manage-tests.el * test/lisp/net/sieve-manage-tests.el: (mk-literal-s2c) (mk-rsp-oknobye) (mk-rsp-getscript-ok) (managesieve-getscript): Test helper functions. (ert/managesieve-getscript-multibyte): Test `sieve-manage-getscript' with script names containing multibyte characters. --- test/lisp/net/sieve-manage-tests.el | 104 ++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 test/lisp/net/sieve-manage-tests.el diff --git a/test/lisp/net/sieve-manage-tests.el b/test/lisp/net/sieve-mana= ge-tests.el new file mode 100644 index 00000000000..010c9071608 --- /dev/null +++ b/test/lisp/net/sieve-manage-tests.el @@ -0,0 +1,104 @@ +;;; sieve-manage-tests.el --- tests for sieve-manage.el -*- lexical-= binding: t; -*- + +;; Copyright (C) 2022 Free Software Foundation, Inc. + +;; Author: Kai Tetzlaff + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see . + +;;; Commentary: + +;;; Code: + +(require 'ert) +(require 'sieve-manage) + +(defvar sieve-script-multibyte-unix + "if body :matches \"=C3=A4\" { stop; }\n" + "Simple multibyte sieve script.") + +(defun mk-literal-s2c (string) + "Encode STRING to managesieve 'literal-s2c'." + (let ((data (sieve-manage-encode string))) + (concat (format "{%d}\r\n" (length data)) + data))) + +(defun mk-rsp-oknobye (type &optional resp-code string) + "Encode TYPE, RESP-CODE and STRING to managesieve 'response-oknobye'." + (when (memq type '(OK NO BYE)) + (concat + (mapconcat #'identity + (delq nil + (list (symbol-name type) + (when resp-code + (format "(%s)" resp-code)) + (when string + (format "\"%s\"" + (sieve-manage-encode string))))) + " ") + "\r\n"))) +;; (mk-rsp-oknobye 'OK nil "Getscript completed.") +;; (mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") + +(defun mk-rsp-getscript-ok (script) + "Encode SCRIPT to managesieve 'response-getscript'." + (concat (mk-literal-s2c script) + "\r\n" + (mk-rsp-oknobye 'OK "Getscript completed."))) +;; (mk-rsp-getscript-ok sieve-script-multibyte-unix) + +(defun managesieve-getscript (script) + "Simulate managesieve getscript response to test +`sieve-manage-getscript' function." + (let* ((script-name "test.sieve") + ;; `sieve-manage-server' and `sieve-manage-port' are used in + ;; `sieve-manage-make-process-buffer' + (sieve-manage-server) + (sieve-manage-port "sieve") + (sieve-buffer (sieve-manage-make-process-buffer)) + (output-buffer (generate-new-buffer script-name))) + ;; use cl-letf to mock some functions in call chain of + ;; sieve-manage-getscript + (cl-letf (((symbol-function 'sieve-manage-send) + ;; simulate sieve server response with a single + ;; multibyte character `=C3=A4` + (lambda (_) + (with-current-buffer sieve-buffer + (goto-char (point-min)) + (insert (mk-rsp-getscript-ok script))))) + ((symbol-function 'accept-process-output) + (lambda (&optional _ _ _ _) nil)) + ((symbol-function 'get-buffer-process) (lambda (_) nil))) + ;; extract sieve script from sieve-buffer and put it into + ;; output-buffer + (sieve-manage-getscript script-name output-buffer sieve-buffer) + ;; extract script from output-buffer and return it as a string + (let ((script (with-current-buffer output-buffer + (set-buffer-modified-p nil) + (buffer-string)))) + ;; cleanup + (kill-buffer sieve-buffer) + (kill-buffer output-buffer) + (when (get-buffer sieve-manage-log) + (kill-buffer sieve-manage-log)) + ;; return script + script)))) + +(ert-deftest ert/managesieve-getscript-multibyte () + (should (string=3D sieve-script-multibyte-unix + (managesieve-getscript sieve-script-multibyte-unix)))) + +;;; sieve-manage-tests.el ends here --=20 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0004-Some-minor-fixes-in-lisp-net-sieve.el.patch >From aef415f651e59542fe7bb3a2ab76c2b27bb51a07 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Thu, 19 Jan 2023 04:06:22 +0100 Subject: [PATCH 4/6] Some minor fixes in lisp/net/sieve.el * lisp/net/sieve.el (sieve-next-line) (sieve-prev-line): Handle situations where point in `sieve-buffer' is either before or after the list of server side scripts. (sieve-server-script-list): New variable. (sieve-refresh-scriptlist) (sieve-upload): Use `sieve-server-script-list' to make sure that local list of server side scripts in`sieve-buffer' is up-to-date after uploading a (new) script. --- lisp/net/sieve.el | 38 +++++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 13 deletions(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index fbd07dee27c..12a85e89d7e 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -95,6 +95,8 @@ sieve-template (defvar sieve-manage-buffer nil) (defvar sieve-buffer-header-end nil) +(defvar sieve-server-script-list nil + "Current list of server-side sieve scripts.") (defvar sieve-buffer-script-name nil "The real script name of the buffer.") (make-local-variable 'sieve-buffer-script-name) @@ -235,23 +237,28 @@ sieve-next-line (interactive) (unless arg (setq arg 1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "End of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "End of list")) + (goto-char (next-overlay-change sieve-buffer-header-end)))) (defun sieve-prev-line (&optional arg) (interactive) (unless arg (setq arg -1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "Beginning of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "Beginning of list")) + (goto-char (previous-overlay-change (point-max))) + (beginning-of-line))) (defun sieve-help () "Display help for various sieve commands." @@ -319,6 +326,9 @@ sieve-refresh-scriptlist (let* ((scripts (sieve-manage-listscripts sieve-manage-buffer)) (count (length scripts)) (keys (substitute-command-keys "\\[sieve-edit-script]"))) + (setq sieve-server-script-list + (mapcar (lambda (elt) (if (consp elt) (cdr elt) elt)) + scripts)) (insert (if (null scripts) (format @@ -361,7 +371,9 @@ sieve-upload (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." - (substitute-command-keys "\\[sieve-manage]")))) + (substitute-command-keys "\\[sieve-manage]")) + (when (not (member script-name sieve-server-script-list)) + (sieve-refresh-scriptlist)))) (set-buffer-modified-p nil)))) ;;;###autoload -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0005-Autodetect-eol-type-of-sieve-manage-scripts.patch >From 4b144b0eff79cdcba1af4e46bd0a57836747d9ce Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Sun, 22 Jan 2023 01:06:57 +0100 Subject: [PATCH 5/6] Autodetect eol-type of sieve(-manage) scripts When using `sieve-manage-getscript' to download a sieve script, sieve-manage now automatically detects the eol-type (either 'utf-8-unix or 'utf-8-dos), uses it to decode the script data and sets `buffer-file-coding-system' accordingly. This gets rid of '^M' in sieve script buffers (for scripts which use CRLF type EOLs). The same eol-type is then used to encode the script during upload with `sieve-manage-putscript'. * lisp/net/sieve-manage.el (sieve-manage--guess-buffer-coding-system): New function which analyzes the eol-type of the first couple of lines of a downloaded script to make a best guess and returns either 'utf-8-unix or 'utf-8-dos. (sieve-manage-decode): Use `sieve-manage--guess-buffer-coding-system' to decode downloaded script data with the correct coding-system and sets the `buffer-file-coding-system' of the resulting sieve script buffer. (sieve-manage-putscript): Now takes a sieve script buffer (instead of a string) argument and forwards it to `sieve-manage-send'. (sieve-manage-send): Now also uses a (payload-)buffer instead of a string. The `buffer-file-coding-system' of the buffer is then used when encoding the payload in order to use the correct eol-type. * lisp/net/sieve.el: (sieve-upload): Adapt to changed argument type of `sieve-manage-putscript'. * etc/NEWS: Add a short description of the changes. --- etc/NEWS | 10 +++++++ lisp/net/sieve-manage.el | 63 +++++++++++++++++++++++++++++++--------- lisp/net/sieve.el | 6 ++-- 3 files changed, 62 insertions(+), 17 deletions(-) diff --git a/etc/NEWS b/etc/NEWS index f8e4aed6703..e3791171220 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -193,6 +193,16 @@ failures. Note: The special case of a REFERRAL/BYE responses is still not handled by the client (s. RFC5804 for more details). +--- +*** Autodetect eol-type of downloaded sieve scripts. +When a downloaded script contained CRLF type EOLs, they caused '^M's +to appear in the sieve script edit buffer. To avoid that, the +eol-type of sieve scripts is now detected during download via +'sieve-manage-getscript', used when decoding the data and stored in +'buffer-file-coding-system' of the script buffer. The +'buffer-file-coding-system' is then also used for encoding during +upload by 'sieve-manage-putscript'. + * New Modes and Packages in Emacs 30.1 diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index de5c3cd1386..7c680007042 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -238,15 +238,43 @@ sieve-manage-encode "Convert STR to managesieve protocol octets." (encode-coding-string str sieve-manage--coding-system t)) +(defun sieve-manage--guess-buffer-coding-system (&optional buffer) + "Return the coding system to be use for (sieve script) BUFFER. + +Since RFC5804 requires scripts to be encoded as UTF-8, the +returned coding system is either \\='utf-8-unix or \\='utf-8-dos." + (with-current-buffer (or buffer (current-buffer)) + (let ((pos (point)) + (max-lines 10) + (line 0) + (crlf-count 0)) + (goto-char (point-min)) + (while (and (> max-lines line) (not (eobp))) + (when (= #x0d (char-before (pos-eol))) + (cl-incf crlf-count)) + (let ((eol (pos-eol))) + (when (> (goto-char (+ eol 1)) eol) + (cl-incf line)))) + (goto-char pos) + (if (> crlf-count (/ line 2)) + 'utf-8-dos + 'utf-8-unix)))) + (defun sieve-manage-decode (octets &optional buffer) "Convert managesieve protocol OCTETS to UTF-8 string. If optional BUFFER is non-nil, insert decoded string into BUFFER." (when octets - ;; eol type unix is required to preserve "\r\n" - (decode-coding-string octets - sieve-manage--coding-system - t buffer))) + (if buffer + (with-current-buffer buffer + (insert octets) + (let ((coding-system + (sieve-manage--guess-buffer-coding-system))) + (set-buffer-file-coding-system coding-system) + (decode-coding-region (point-min) (point-max) + coding-system))) + (decode-coding-string + octets sieve-manage--coding-system t)))) (defun sieve-manage-make-process-buffer () (let ((buffer (sieve-manage--set-buffer-maybe-append-text @@ -509,9 +537,9 @@ sieve-manage-havespace (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) (sieve-manage-parse-oknobye))) -(defun sieve-manage-putscript (name content &optional buffer) - (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) +(defun sieve-manage-putscript (name script-buffer &optional process-buffer) + (with-current-buffer (or process-buffer (current-buffer)) + (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) script-buffer) (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) @@ -641,13 +669,20 @@ sieve-manage-parse-listscripts data rsp))) -(defun sieve-manage-send (command &optional payload-str) - "Send COMMAND with optional PAYLOAD-STR. - -If non-nil, PAYLOAD-STR will be appended to COMMAND using the -\\='literal-s2c\\' representation according to RFC5804." - (let ((encoded (when payload-str (sieve-manage-encode payload-str))) - literal-c2s cmdstr) +(defun sieve-manage-send (command &optional payload-buffer) + "Send COMMAND with optional string from PAYLOAD-BUFFER. + +If non-nil, the content of PAYLOAD-BUFFER will be appended to +COMMAND using the \\='literal-s2c\\=' representation according to RFC5804." + (let* ((encoded (when (and payload-buffer + (> (buffer-size payload-buffer) 0)) + (with-current-buffer payload-buffer + (encode-coding-region + (point-min) (point-max) + (buffer-local-value 'buffer-file-coding-system + payload-buffer) + t)))) + cmdstr literal-c2s) (when encoded (setq literal-c2s (format " {%d+}%s%s" (length encoded) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index 12a85e89d7e..2108732c5dd 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -361,13 +361,13 @@ sieve-upload (interactive) (when (or (get-buffer sieve-buffer) (save-current-buffer (call-interactively 'sieve-manage))) - (let ((script (buffer-string)) + (let ((script-buffer (current-buffer)) (script-name (file-name-sans-extension (buffer-name))) err) (with-current-buffer (get-buffer sieve-buffer) - (setq err (sieve-manage-putscript + (setq err (sieve-manage-putscript (or name sieve-buffer-script-name script-name) - script sieve-manage-buffer)) + script-buffer sieve-manage-buffer)) (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." -- 2.39.0 --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0006-WiP-new-encode-tested-OK.patch Content-Transfer-Encoding: quoted-printable >From 334792ee0072890800933f080d9ca86ac2aecf3f Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 23 Jan 2023 00:50:18 +0100 Subject: [PATCH 6/6] WiP: new encode, tested OK --- lisp/net/sieve-manage.el | 293 ++++++++++++++++++++-------- test/lisp/net/sieve-manage-tests.el | 216 +++++++++++++++----- 2 files changed, 381 insertions(+), 128 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 7c680007042..ea96dfd14ef 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -151,7 +151,7 @@ sieve-manage--coding-system Defines the coding system used for the internal (process, log) buffers and the network stream created to communicate with the managesieve server. Using \\=3D'utf-8-unix encoding corresponds to -the use of UTF-8 in RFC5804 (managesieve). The explicit use of +the use of UTF-8 in rfc5804 (managesieve). The explicit use of \\=3D'-unix\\=3D' avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") sequences intact).") =20 @@ -173,6 +173,13 @@ sieve-manage-state (defvar sieve-manage-process nil) (defvar sieve-manage-capability nil) =20 +(define-error 'sieve-manage-error + "Unknown sieve-manage error") +(define-error 'sieve-manage-encode-error + "sieve-manage encoding error" 'sieve-manage-error) +(define-error 'sieve-manage-server-error + "Managesieve server signaled an error" 'sieve-manage-error) + ;; Internal utility functions (defun sieve-manage--set-buffer-maybe-append-text (buffer-name &rest args) @@ -224,55 +231,144 @@ sieve-manage--message (sieve-manage--append-to-log ret "\n") ret)) =20 -(defun sieve-manage--error (format-string &rest args) - "Wrapper around `error' which also logs to sieve manage log. +(defun sieve-manage--error (type format-string &rest args) + "Generate a \\=3D'sieve-manage-error\\=3D' of TYPE. +FORMAT-STRONG and ARGS are used as arguments to `format'. +Errors are also logged to sieve manage log. =20 See `sieve-manage--append-to-log'." (let ((msg (apply #'format (concat "sieve-manage/ERROR: " format-string) args))) (sieve-manage--append-to-log msg "\n") - (error msg))) + (signal type (list msg)))) =20 -(defun sieve-manage-encode (str) - "Convert STR to managesieve protocol octets." +(defun sieve-manage--convert-multibyte (str) + "Convert multibyte STR to unibyte octet string. +The conversion is done using `sieve-manage--coding-system' +(\\=3D'utf-8-unix)." (encode-coding-string str sieve-manage--coding-system t)) =20 -(defun sieve-manage--guess-buffer-coding-system (&optional buffer) +(defun sieve-manage-mk-quoted (octet-str) + "Convert OCTET-STR to rfc5804 \\=3D'quoted\\=3D' string." + (or (stringp octet-str) + (sieve-manage--error 'sieve-manage-encode-error + "Argument must be a string")) + (format "\"%s\"" octet-str)) + +(defun sieve-manage-mk-literal-c2s (octet-str) + "Convert OCTET-STR to rfc5804 \\=3D'literal-c2s\\=3D' string." + (or (stringp octet-str) + (sieve-manage--error 'sieve-manage-encode-error + "Argument must be a string")) + (format "{%d+}%s%s" + (length octet-str) + sieve-manage-client-eol + octet-str)) + +(defun sieve-manage-mk-literal-s2c (octet-str) + "Convert OCTET-STR to rfc5804 \\=3D'literal-s2c\\=3D' string." + (or (stringp octet-str) + (sieve-manage--error 'sieve-manage-encode-error + "Argument must be a string")) + (format "{%d}%s%s" + (length octet-str) + sieve-manage-client-eol + octet-str)) + +(defun sieve-manage-encode (item) + "Convert ITEM to a rfc5804 protocol string. + +An ITEM can be a: +1. number (positive integer or 0), +2. string (will be converted to unibyte using \\=3D'utf-8-unix + coding system and encoded as \\=3D'quoted\\=3D' rfc5804 string), +3. buffer (buffer content will be converted to unibyte using + `buffer-file-coding-system and encoded as \\=3D'literal-c2s\\=3D' rfc58= 04 + string), +4. cons cell (TYPE . DATA) where the following combinations of TYPE + and DATA are supported: + - (number . ) where is an integer in the range of + [0..4294967296] + - (cmd-name . ) where is a unibyte string + - ( . ) where is one of \\=3D'quoted, + \\=3D'literal-c2s, \\=3D'literal-s2c and is a unibyte str= ing + - ( . ) where is one of \\=3D'mb-quoted, + \\=3D'mb-literal-c2s, \\=3D'mb-literal-s2c and is a (multibyte)= UTF-8 + string which will be converted to unibyte using coding system + \\=3D'utf-8-unix." + (cond + ((and (integerp item) (>=3D item 0)) + (sieve-manage-encode (cons 'number item))) + ((stringp item) + ;; TODO: + ;; - check if character set of item requires use of 'literal-c2s + ;; - check if length of item requires use of 'literal-c2s + (sieve-manage-encode (cons 'quoted + (sieve-manage--convert-multibyte item)))) + ((bufferp item) + (sieve-manage-encode + (cons 'literal-c2s (encode-coding-region + (point-min) (point-max) + (buffer-local-value 'buffer-file-coding-system + item) + t)))) + ((consp item) + (let ((type (car item)) + (data (cdr item))) + (pcase type + ('number (format "%d" data)) + ('cmd-name (format "%s" data)) + ('quoted (sieve-manage-mk-quoted data)) + ('literal-c2s (sieve-manage-mk-literal-c2s data)) + ('literal-s2c (sieve-manage-mk-literal-s2c data)) + ('mb-quoted (sieve-manage-mk-quoted + (sieve-manage--convert-multibyte data))) + ('mb-literal-c2s (sieve-manage-mk-literal-c2s + (sieve-manage--convert-multibyte data))) + ('mb-literal-s2c (sieve-manage-mk-literal-s2c + (sieve-manage--convert-multibyte data))) + (_ (sieve-manage--error 'sieve-manage-encode-error + "Unknown encoding type: '%s'" + type))))) + (t (sieve-manage--error 'sieve-manage-encode-error + "Don't know how to encode '%s'" item)))) + +(defun sieve-manage--detect-buffer-coding-system (&optional buffer) "Return the coding system to be use for (sieve script) BUFFER. =20 -Since RFC5804 requires scripts to be encoded as UTF-8, the -returned coding system is either \\=3D'utf-8-unix or \\=3D'utf-8-dos." +Since rfc5804 requires sieve scripts to use (a subset of) UTF-8, the +returned coding system is of type \\=3D'utf-8 with either \\=3D'-unix\\=3D= ', +\\=3D'-dos\\=3D' or \\=3D'-mac\\=3D' eol-type." (with-current-buffer (or buffer (current-buffer)) - (let ((pos (point)) - (max-lines 10) - (line 0) - (crlf-count 0)) - (goto-char (point-min)) - (while (and (> max-lines line) (not (eobp))) - (when (=3D #x0d (char-before (pos-eol))) - (cl-incf crlf-count)) - (let ((eol (pos-eol))) - (when (> (goto-char (+ eol 1)) eol) - (cl-incf line)))) - (goto-char pos) - (if (> crlf-count (/ line 2)) - 'utf-8-dos - 'utf-8-unix)))) + (let ((coding-system (detect-coding-region + (point-min) (point-max) t))) + (alist-get (coding-system-eol-type coding-system) + '((0 . utf-8-unix) + (1 . utf-8-dos) + (2 . utf-8-mac)) + 'utf-8-unix)))) =20 (defun sieve-manage-decode (octets &optional buffer) "Convert managesieve protocol OCTETS to UTF-8 string. =20 -If optional BUFFER is non-nil, insert decoded string into BUFFER." +If BUFFER is non-nil detect \\=3D'eol-type\\=3D' of OCTETS, use correspond= ing +\\=3D'utf-8- coding system to decode octets, set +`buffer-file-coding-system` of BUFFER, insert the decoded UTF-8 string +into BUFFER and return BUFFER. + +Otherwise, decode OCTETS using `sieve-manage--coding-system' (\\=3D'utf-8-= unix) +and return the resulting UTF-8 string." (when octets (if buffer (with-current-buffer buffer (insert octets) (let ((coding-system - (sieve-manage--guess-buffer-coding-system))) + (sieve-manage--detect-buffer-coding-system))) (set-buffer-file-coding-system coding-system) (decode-coding-region (point-min) (point-max) - coding-system))) + coding-system)) + buffer) (decode-coding-string octets sieve-manage--coding-system t)))) =20 @@ -342,18 +438,11 @@ sieve-sasl-auth ;; somehow. (lambda (_prompt) (copy-sequence user-password))) (step (sasl-next-step client nil)) - (_tag (sieve-manage-send - (concat - "AUTHENTICATE \"" - mech - "\"" - (and (sasl-step-data step) - (concat - " \"" - (base64-encode-string - (sasl-step-data step) - 'no-line-break) - "\""))))) + (_tag (sieve-manage-send-command + "AUTHENTICATE" mech (and (sasl-step-data step) + (base64-encode-string + (sasl-step-data step) + 'no-line-break)))) data rsp) (catch 'done (while t @@ -378,24 +467,23 @@ sieve-sasl-auth (if (and (setq step (sasl-next-step client step)) (setq data (sasl-step-data step))) ;; We got data for server but it's finished - (sieve-manage--error + (sieve-manage--error 'sieve-manage-server-error "Server not ready for SASL data: %s" data) ;; The authentication process is finished. (sieve-manage--message "Logged in as %s using %s" user-name mech) (throw 'done t))) (unless (stringp rsp) - (sieve-manage--error + (sieve-manage--error 'sieve-manage-server-error "Server aborted SASL authentication: %s" (caddr rsp))) (sasl-step-set-data step (base64-decode-string rsp)) (setq step (sasl-next-step client step)) (sieve-manage-send - (if (sasl-step-data step) - (concat "\"" - (base64-encode-string (sasl-step-data step) - 'no-line-break) - "\"") - ""))))))) + (cons 'quoted + (if (sasl-step-data step) + (base64-encode-string (sasl-step-data step) + 'no-line-break) + "")))))))) =20 (defun sieve-manage-cram-md5-p (buffer) (sieve-manage-capability "SASL" "CRAM-MD5" buffer)) @@ -474,7 +562,7 @@ sieve-manage-open (setq sieve-manage-auth auth) (cl-return))) (unless sieve-manage-auth - (sieve-manage--error + (sieve-manage--error 'sieve-manage-error "Couldn't figure out authenticator for server"))) (sieve-manage-erase) (current-buffer)))) @@ -504,7 +592,7 @@ sieve-manage-close If BUFFER is nil, the current buffer is used." (with-current-buffer (or buffer (current-buffer)) (when (sieve-manage-opened) - (sieve-manage-send "LOGOUT") + (sieve-manage-send-command "LOGOUT") (sit-for 1)) (when (and sieve-manage-process (memq (process-status sieve-manage-process) '(open run))) @@ -528,36 +616,66 @@ sieve-manage-capability server-value))))) =20 (defun sieve-manage-listscripts (&optional buffer) + "Send LISTSCRIPTS command to download list of available scripts. +BUFFER is the \\=3D'sieve-manage\\=3D' process buffer (default: \\=3D'(cur= rent-buffer)\\=3D')." (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send "LISTSCRIPTS") + (sieve-manage-send-command "LISTSCRIPTS") (sieve-manage-parse-listscripts))) =20 (defun sieve-manage-havespace (name size &optional buffer) + "Send HAVESPACE command for script NAME and SIZE. +BUFFER is the \\=3D'sieve-manage\\=3D' process buffer (default: \\=3D'(cur= rent-buffer)\\=3D')." + (when (string-empty-p name) + (sieve-manage-error "HAVESPACE script name cannot be empty")) (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) + (sieve-manage-send-command "HAVESPACE" name size) (sieve-manage-parse-oknobye))) =20 -(defun sieve-manage-putscript (name script-buffer &optional process-buffer) - (with-current-buffer (or process-buffer (current-buffer)) - (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) script-buffer) +(defun sieve-manage-putscript (name script-buffer &optional buffer) + "Send PUTSCRIPT command to upload script named NAME. +Uses the content of SCRIPT-BUFFER as the actual script to upload. +`buffer-file-coding-system' of SCRIPT-BUFFER is used to convert +the script to unibyte before the uploaded. It shall be set to either +\\=3D'utf-8-unix, \\=3D'utf-8-dos or \\=3D'utf-8-mac. +BUFFER is the \\=3D'sieve-manage\\=3D' process buffer (default: \\=3D'(cur= rent-buffer)\\=3D')." + (when (string-empty-p name) + (sieve-manage-error "PUTSCRIPT script name cannot be empty")) + (with-current-buffer (or buffer (current-buffer)) + (sieve-manage-send-command "PUTSCRIPT" name script-buffer) (sieve-manage-parse-oknobye))) =20 (defun sieve-manage-deletescript (name &optional buffer) + "Send DELETESCRIPT command to delete script named NAME. +BUFFER is the \\=3D'sieve-manage\\=3D' process buffer (default: \\=3D'(cur= rent-buffer)\\=3D')." + (when (string-empty-p name) + (sieve-manage-error "DELETESCRIPT script name cannot be empty")) (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "DELETESCRIPT \"%s\"" name)) + (sieve-manage-send-command "DELETESCRIPT" name) (sieve-manage-parse-oknobye))) =20 -(defun sieve-manage-getscript (name output-buffer &optional buffer) +(defun sieve-manage-getscript (name script-buffer &optional buffer) + "Send GETSCRIPT command to download script named NAME. +Inserts the downloaded script into SCRIPT-BUFFER. The \\=3D'eol-type\\=3D= ' of +the downloaded script will be detected automatically and gets +used to decode the script data before inserting it into +SCRIPT-BUFFER. Also sets `buffer-file-coding-system' of +SCRIPT-BUFFER to the \\=3D'utf-8\\=3D' variant with the detected \\=3D'eol= -type\\=3D'. +BUFFER is the \\=3D'sieve-manage\\=3D' process buffer (default: \\=3D'(cur= rent-buffer)\\=3D')." + (when (string-empty-p name) + (sieve-manage-error "GETSCRIPT script name cannot be empty")) (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "GETSCRIPT \"%s\"" name)) + (sieve-manage-send-command "GETSCRIPT" name) (sieve-manage-decode (sieve-manage-parse-string) - output-buffer) + script-buffer) (sieve-manage-parse-crlf) (sieve-manage-parse-oknobye))) =20 (defun sieve-manage-setactive (name &optional buffer) + "Send SETACTIVE command to activate script named NAME. +Use an empty NAME to deactivate/disable any active script. +BUFFER is the \\=3D'sieve-manage\\=3D' process buffer (default: \\=3D'(cur= rent-buffer)\\=3D')." (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "SETACTIVE \"%s\"" name)) + (sieve-manage-send-command "SETACTIVE" name) (sieve-manage-parse-oknobye))) =20 ;; Protocol parsing routines @@ -636,9 +754,10 @@ sieve-manage-parse-string (goto-char (point-min)) (unless (setq rsp (sieve-manage-is-string)) (when (sieve-manage-no-p (sieve-manage-is-oknobye)) - ;; simple `error' is enough since `sieve-manage-erase' + ;; no further details required since `sieve-manage-erase' ;; already adds the server response to the log - (error (sieve-manage-erase))))) + (sieve-manage--error 'sieve-manage-server-error + "%s" (sieve-manage-erase))))) (sieve-manage-erase (point)) rsp)) =20 @@ -669,28 +788,36 @@ sieve-manage-parse-listscripts data rsp))) =20 -(defun sieve-manage-send (command &optional payload-buffer) - "Send COMMAND with optional string from PAYLOAD-BUFFER. - -If non-nil, the content of PAYLOAD-BUFFER will be appended to -COMMAND using the \\=3D'literal-s2c\\=3D' representation according to RFC5= 804." - (let* ((encoded (when (and payload-buffer - (> (buffer-size payload-buffer) 0)) - (with-current-buffer payload-buffer - (encode-coding-region - (point-min) (point-max) - (buffer-local-value 'buffer-file-coding-system - payload-buffer) - t)))) - cmdstr literal-c2s) - (when encoded - (setq literal-c2s (format " {%d+}%s%s" - (length encoded) - sieve-manage-client-eol - encoded))) - (setq cmdstr (concat (sieve-manage-encode command) - literal-c2s - sieve-manage-client-eol)) +(defun sieve-manage-send-command (cmd-name &rest items) + "Assemble rfc5804 command from CMD-NAME and ITEMS and send it. +See `sieve-manage-send' for further details." + (when (string-empty-p cmd-name) + (sieve-manage--error 'sieve-manage-encode-error + "Command name cannot be empty")) + (when (multibyte-string-p cmd-name) + (sieve-manage--error 'sieve-manage-encode-error + "Command name '%s' cannot must be a unibyte string" cmd-name)) + (apply #'sieve-manage-send (cons 'cmd-name cmd-name) items)) + +(defun sieve-manage-send (&rest items) + "Encode and concatenate ITEMS, send the result to the sieve server. +ITEMs will be: + +1. skipped/ignored if nil +2. converted to unibyte (optional, depends on item type) +3. encoded according to rfc5804 +4. concatenated (using SPaces as separators) + +The result will then be sent to the managesieve server. + +See `sieve-manage-encode' for details regarding supported ITEMS and their +handling." + (let ((cmdstr (concat + (string-join + (remove nil (mapcar #'sieve-manage-encode + items)) + " ") + sieve-manage-client-eol))) (sieve-manage--append-to-log cmdstr) (process-send-string sieve-manage-process cmdstr))) =20 diff --git a/test/lisp/net/sieve-manage-tests.el b/test/lisp/net/sieve-mana= ge-tests.el index 010c9071608..60f47518f67 100644 --- a/test/lisp/net/sieve-manage-tests.el +++ b/test/lisp/net/sieve-manage-tests.el @@ -26,79 +26,205 @@ (require 'ert) (require 'sieve-manage) =20 -(defvar sieve-script-multibyte-unix +;;; test data + +(defvar smt/script-multibyte-unix "if body :matches \"=C3=A4\" { stop; }\n" - "Simple multibyte sieve script.") + "Simple multibyte sieve script with unix EOL.") + +(defvar smt/script-multibyte-dos + "if body :matches \"=C3=A4\" { stop; }\r\n" + "Simple multibyte sieve script with dos EOL.") + +(defvar smt/script-multibyte-mac + "if body :matches \"=C3=A4\" { stop; }\r" + "Simple multibyte sieve script with mac EOL.") + +(defvar smt/string-empty "" + "Empty sieve string.") + +(defvar smt/string-unibyte "Hello world!" + "Unibyte sieve string.") + +(defvar smt/string-multibyte "=C3=A4=C3=B6=C3=BC=C3=9F=C3=84=C3=96=C3=9C" + "Multibyte sieve string.") =20 -(defun mk-literal-s2c (string) - "Encode STRING to managesieve 'literal-s2c'." - (let ((data (sieve-manage-encode string))) - (concat (format "{%d}\r\n" (length data)) - data))) +(defvar smt/script-name-unibyte "abcdefg.sieve" + "Unibyte sieve script name.") =20 -(defun mk-rsp-oknobye (type &optional resp-code string) +(defvar smt/script-name-multibyte "=C3=A4=C3=B6=C3=BC=C3=9F=C3=84=C3=96=C3= =9C.sieve" + "Multibyte sieve script name.") + +;;; helper functions + +(defun smt/mk-literal-s2c (string) + "Encode multibyte STRING to managesieve 'literal-s2c'." + (sieve-manage-encode (cons 'mb-literal-s2c string))) +;; (smt/mk-literal-s2c smt/string-multibyte) + +(defun smt/mk-rsp-oknobye (type &optional resp-code string) "Encode TYPE, RESP-CODE and STRING to managesieve 'response-oknobye'." (when (memq type '(OK NO BYE)) (concat - (mapconcat #'identity - (delq nil - (list (symbol-name type) - (when resp-code - (format "(%s)" resp-code)) - (when string - (format "\"%s\"" - (sieve-manage-encode string))))) - " ") + (string-join + (delete nil + (list (symbol-name type) + (when resp-code + (format "(%s)" resp-code)) + (when string + (format "\"%s\"" + (sieve-manage--encode-multibyte string))))) + " ") "\r\n"))) -;; (mk-rsp-oknobye 'OK nil "Getscript completed.") -;; (mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") +;; (smt/mk-rsp-oknobye 'OK nil "Getscript completed.") +;; (smt/mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") =20 -(defun mk-rsp-getscript-ok (script) - "Encode SCRIPT to managesieve 'response-getscript'." - (concat (mk-literal-s2c script) +(defun smt/mk-rsp-getscript-ok (script) + "Encode SCRIPT to rfc5804 \\=3D'response-getscript\\=3D'." + (concat (smt/mk-literal-s2c script) "\r\n" - (mk-rsp-oknobye 'OK "Getscript completed."))) -;; (mk-rsp-getscript-ok sieve-script-multibyte-unix) - -(defun managesieve-getscript (script) - "Simulate managesieve getscript response to test -`sieve-manage-getscript' function." + (smt/mk-rsp-oknobye 'OK "Getscript completed."))) +;; (smt/mk-rsp-getscript-ok smt/script-multibyte-unix) + +(defun smt/managesieve-getscript (script &optional nocleanup) + "Simulate rfc5804 response for GETSCRIPT command. +The value of SCRIPT is used as the actual sieve script. +Use for testing `sieve-manage-getscript' function. +If NOCLEANUP is non-nil, keep all created buffers." (let* ((script-name "test.sieve") ;; `sieve-manage-server' and `sieve-manage-port' are used in ;; `sieve-manage-make-process-buffer' (sieve-manage-server) (sieve-manage-port "sieve") - (sieve-buffer (sieve-manage-make-process-buffer)) + ;; sieve-manage process buffer + (process-buffer (sieve-manage-make-process-buffer)) + ;; sieve-manage buffer to receive downloaded sieve script (output-buffer (generate-new-buffer script-name))) ;; use cl-letf to mock some functions in call chain of ;; sieve-manage-getscript (cl-letf (((symbol-function 'sieve-manage-send) - ;; simulate sieve server response with a single - ;; multibyte character `=C3=A4` - (lambda (_) - (with-current-buffer sieve-buffer + ;; simulate sieve server getscript response + ;; containing 'script' + (lambda (&rest _) + (with-current-buffer process-buffer (goto-char (point-min)) - (insert (mk-rsp-getscript-ok script))))) + (insert (smt/mk-rsp-getscript-ok script))))) ((symbol-function 'accept-process-output) (lambda (&optional _ _ _ _) nil)) ((symbol-function 'get-buffer-process) (lambda (_) nil))) - ;; extract sieve script from sieve-buffer and put it into + ;; extract sieve script from process-buffer and put it into ;; output-buffer - (sieve-manage-getscript script-name output-buffer sieve-buffer) + (sieve-manage-getscript script-name output-buffer process-buffer) ;; extract script from output-buffer and return it as a string (let ((script (with-current-buffer output-buffer (set-buffer-modified-p nil) - (buffer-string)))) + (buffer-string))) + (coding-system + (buffer-local-value 'buffer-file-coding-system + output-buffer))) ;; cleanup - (kill-buffer sieve-buffer) - (kill-buffer output-buffer) - (when (get-buffer sieve-manage-log) - (kill-buffer sieve-manage-log)) - ;; return script - script)))) + (unless nocleanup + (kill-buffer process-buffer) + (kill-buffer output-buffer) + (when (get-buffer sieve-manage-log) + (kill-buffer sieve-manage-log))) + ;; return (script . coding-system) + (cons script coding-system))))) + +;;; tests + +(ert-deftest ert/manage-sieve-mk-quoted () + (should-error (sieve-manage-mk-quoted 42) + :type 'sieve-manage-encode-error) + (should-error (sieve-manage-mk-quoted nil) + :type 'sieve-manage-encode-error) + (should (equal (format "\"%s\"" smt/script-name-unibyte) + (sieve-manage-mk-quoted smt/script-name-unibyte))) + (should (equal "\"\"" + (sieve-manage-mk-quoted "")))) + +(ert-deftest ert/manage-sieve-mk-literal-c2s () + (should-error (sieve-manage-mk-literal-c2s 42) + :type 'sieve-manage-encode-error) + (should-error (sieve-manage-mk-literal-c2s nil) + :type 'sieve-manage-encode-error) + (should (equal (format "{%d+}\r\n%s" + (length smt/script-name-unibyte) + smt/script-name-unibyte) + (sieve-manage-mk-literal-c2s smt/script-name-unibyte))) + (should (equal "{0+}\r\n" + (sieve-manage-mk-literal-c2s "")))) + +(ert-deftest ert/manage-sieve-mk-literal-s2c () + (should-error (sieve-manage-mk-literal-s2c 42) + :type 'sieve-manage-encode-error) + (should-error (sieve-manage-mk-literal-s2c nil) + :type 'sieve-manage-encode-error) + (should (equal (format "{%d}\r\n%s" + (length smt/script-name-unibyte) + smt/script-name-unibyte) + (sieve-manage-mk-literal-s2c smt/script-name-unibyte))) + (should (equal "{0}\r\n" + (sieve-manage-mk-literal-s2c "")))) + +(ert-deftest ert/sieve-manage-encode () + ;; unsupported data types + (should-error (sieve-manage-encode nil) + :type 'sieve-manage-encode-error) + (should-error + ;; RFC5804 doesn't support negative numbers + (sieve-manage-encode -1) :type 'sieve-manage-encode-error) + + ;; number [0..4294967296] + (should (equal (format "%d" 0) (sieve-manage-encode 0))) + (should (equal (format "%d" 255) (sieve-manage-encode 255))) + (should (equal (format "%d" 4294967296) + (sieve-manage-encode 4294967296))) + + ;; simple string + (should (equal (format "\"%s\"" smt/script-name-unibyte) + (sieve-manage-encode smt/script-name-unibyte))) + (should (equal (format "\"%s\"" (encode-coding-string + smt/script-name-multibyte + 'utf-8-unix)) + (sieve-manage-encode smt/script-name-multibyte))) + + ;; request explicit quoted encoding + (should (equal (format "{%d+}\r\n%s" + (length smt/script-name-unibyte) + smt/script-name-unibyte) + (sieve-manage-encode (cons 'literal-c2s smt/script-name-u= nibyte)))) + (should (equal "{0+}\r\n" + (sieve-manage-encode (cons 'literal-c2s "")))) + + ;; request explicit literal-c2s encoding + (should (equal (format "{%d+}\r\n%s" + (length smt/script-name-unibyte) + smt/script-name-unibyte) + (sieve-manage-encode (cons 'literal-c2s smt/script-name-u= nibyte)))) + (should (equal "{0+}\r\n" + (sieve-manage-encode (cons 'literal-c2s "")))) + + ;; request explicit literal-s2c encoding + (should (equal (format "{%d}\r\n%s" + (length smt/script-name-unibyte) + smt/script-name-unibyte) + (sieve-manage-encode (cons 'literal-s2c smt/script-name-u= nibyte)))) + (should (equal "{0}\r\n" + (sieve-manage-encode (cons 'literal-s2c "")))) + ) + =20 (ert-deftest ert/managesieve-getscript-multibyte () - (should (string=3D sieve-script-multibyte-unix - (managesieve-getscript sieve-script-multibyte-unix)))) + (let ((ret (smt/managesieve-getscript smt/script-multibyte-unix))) + (should (string=3D smt/script-multibyte-unix (car ret))) + (should (eq 'utf-8-unix (cdr ret)))) + (let ((ret (smt/managesieve-getscript smt/script-multibyte-dos))) + (should (string=3D smt/script-multibyte-unix (car ret))) + (should (eq 'utf-8-dos (cdr ret)))) + (let ((ret (smt/managesieve-getscript smt/script-multibyte-mac))) + (should (string=3D smt/script-multibyte-unix (car ret))) + (should (eq 'utf-8-mac (cdr ret)))) + ) =20 ;;; sieve-manage-tests.el ends here --=20 2.39.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 23 07:47:27 2023 Received: (at 54154) by debbugs.gnu.org; 23 Jan 2023 12:47:27 +0000 Received: from localhost ([127.0.0.1]:53566 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJwER-0005iA-4c for submit@debbugs.gnu.org; Mon, 23 Jan 2023 07:47:27 -0500 Received: from mail.bsd4all.net ([94.130.200.20]:43743) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJwEP-0005hw-Dy for 54154@debbugs.gnu.org; Mon, 23 Jan 2023 07:47:26 -0500 Date: Mon, 23 Jan 2023 13:47:19 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1674478039; bh=UsJzp8mWo9hXMEC5DuICDiNI3FruZh+RJICa5/WNe+A=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type; b=JIbMUKTLbkAOzOiAVB9hSCvxXDv219FtQohNx71R13dZbGpfEfudSv/Vssiv9kas/ 6zjPQsqPy1lVNFr+loeHeZndSXHZlp9B72xafqlks6nnRWx66e/+dQ0kJBokO36kmG ArQpwBjLOw+Ij6rdb6PLK3ecXwsoLMcbP53sE01DHqr1UQZCgbFuiCwBSY71RwZ7rl yAXz7RyNC98H1e5DOZzg1thCtCxOFCn45CBUowoeb50Hswxf5dFtmleXXC6B2zSB1W 9XEW436LrtdV8nYGUOYWJt7oE86g9i2uvH6qKg1oPZTlDRVt7up8S3o9RfL0w4hDQN ZKyLlKXR/TbyA== From: "Herbert J. Skuhra" To: Kai Tetzlaff Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) Message-ID: References: <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <878rhut6aq.fsf@tetzco.de> X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: Eli Zaretskii , 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Mon, Jan 23, 2023 at 01:59:09AM +0100, Kai Tetzlaff wrote: > Just a quick update (also answering my own question(s) below): > > Regarding the status of this bug report: The fix for emacs-29 which was > initially intended to be emacs-29 only has (inadvertently?) found its way > to master. No, the change was not merged to master. Thanks. -- Herbert From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 23 08:01:15 2023 Received: (at 54154) by debbugs.gnu.org; 23 Jan 2023 13:01:15 +0000 Received: from localhost ([127.0.0.1]:53576 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJwRn-00066i-83 for submit@debbugs.gnu.org; Mon, 23 Jan 2023 08:01:15 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:35653) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJwRj-00066T-C6 for 54154@debbugs.gnu.org; Mon, 23 Jan 2023 08:01:13 -0500 Received: from mail.tetzco.de (unknown [IPv6:2a02:810d:1380:7900::9]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id 7A391BD078; Mon, 23 Jan 2023 14:01:05 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id 6695D6C00B2; Mon, 23 Jan 2023 14:21:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674480062; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=rcVF4NxLPahWWvDX6bm5UXRtUrbZgzQLBhJfqFOOJXo=; b=bmxTIBoV37jQ988mj9q8Vo7HQwt4HIQr7ysyuVHP9GAmUyitTlFkYSTbGypsNx9qkKZYr8 V9fzFBX+LICCFy8ftIff3CHDklKO/IA1J5A3zFh9q6n5wmTNXI3NUlS9wWNA7Q4o8EJi7A AIUn/AJxC6vf3zMNuq1bJ0t4RVmKG+Z9qQzR9mRExLmUn1MO4TXbJBC0psIamvajjNnyVQ skNGztvAmQ65ruWrZ/4ih9vLIvJWgfDLC/xLE3K4HRMjXQoD3DUEn5Yt7N7SoW35JtQ2bT Gws0OnFfYg3Vyu8u9vvxo6cyFvgvK5NdUyAnwAlvZ/hFSQ62WyPXt61bx9PjMQ2CCUJWh5 3v+/HW7yiYWim+m/ce6yylOGrfr8OwTu08iAL9EoTEFklyBC55KT8Ebxmez/e+JrA4asZP Yeh2fDiKCLpBVTFvK9iygP9ZvK7c26/izDxqb8EtUNfZJ8dbpv5ishhnS2PycXycywjKvZ wwgF7YjwbtZvca2lgb3w9Q/ER/XNMjANsZPp75pszvVWc6A5x/os7AOw1fjwDYqNhf8k3/ Bjz681zaqkGmHT8D36RraOYwFSagVbdFZTHGhfGsvwhyowEiMUi5GPrSWPkRptW/Hp6zcl +KZznbl6twS5DIjfCHskCg4tEaxLImTHF30aEa3SZlwVNqcLzGw6k= From: Kai Tetzlaff To: "Herbert J. Skuhra" Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: (Herbert J. Skuhra's message of "Mon, 23 Jan 2023 13:47:19 +0100") References: <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> Date: Mon, 23 Jan 2023 14:01:02 +0100 Message-ID: <87ilgxs8vl.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: 6695D6C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.83 / 30.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM(-3.00)[-1.000]; GENERIC_REPUTATION(-0.73)[-0.72664323811806]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ZERO(0.00)[0]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_FROM(0.00)[bug]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: Eli Zaretskii , 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) "Herbert J. Skuhra" writes: > On Mon, Jan 23, 2023 at 01:59:09AM +0100, Kai Tetzlaff wrote: >> Just a quick update (also answering my own question(s) below): >> >> Regarding the status of this bug report: The fix for emacs-29 which was >> initially intended to be emacs-29 only has (inadvertently?) found its way >> to master. > > No, the change was not merged to master. Hmm, what about: $ git log --grep "^Fix bug in 'sieve-manage--append-to-log" origin/master commit 12d7670b90f66f1d45a8c69d9acfc25238a65b02 Author: Kai Tetzlaff Date: 2023-01-19 03:16:14 +0100 Fix bug in 'sieve-manage--append-to-log' * lisp/net/sieve-manage.el (sieve-manage--append-to-log): Fix log buffer creation. (Bug#54154) Do not merge to master. BR, Kai From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 23 08:36:52 2023 Received: (at 54154) by debbugs.gnu.org; 23 Jan 2023 13:36:52 +0000 Received: from localhost ([127.0.0.1]:53681 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJx0G-0000wj-IA for submit@debbugs.gnu.org; Mon, 23 Jan 2023 08:36:52 -0500 Received: from mail.bsd4all.net ([94.130.200.20]:44539) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJx0E-0000wU-VE for 54154@debbugs.gnu.org; Mon, 23 Jan 2023 08:36:51 -0500 Date: Mon, 23 Jan 2023 14:36:44 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1674481004; bh=02Sr7yxBBpmiMRzYUmA3Jm/VnxY6dgNckyrPQpH1IkE=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type; b=P1A7/+gX1nMqr9O/SHpDZn/orJyp/1Loj67fxsAVS/7yhelaMf3jcwo2zYxj/wq9d 2CBe4hSPpNj+EL3tTFcM+jTEhZ2MUV3IZMaqu2lH02ipAPGVDjxEFW5NIPjC9MvXPH 64ipPqf0qaO6gfFZsRaZWjCJq9+QRZHPnxdTaca+kGk8DVPW7oFr3bwcQfMfZM8Q2/ U1dKRlM6Go7+nSVtfK2gMtCP7RNRk7d17KEgjMPpGBM322pYw4lIB4O9lBNenDznoU BpkryaCG3mLvAN8pVd5PRyyywgvz2/o93zzx9wFmB7Du5/5gQBVtefuRqD5e4+6+wT Y1jZrNpcT6Neg== From: "Herbert J. Skuhra" To: Kai Tetzlaff Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) Message-ID: References: <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87ilgxs8vl.fsf@tetzco.de> X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: Eli Zaretskii , 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Mon, Jan 23, 2023 at 02:01:02PM +0100, Kai Tetzlaff wrote: > > Hmm, what about: > > $ git log --grep "^Fix bug in 'sieve-manage--append-to-log" origin/master > commit 12d7670b90f66f1d45a8c69d9acfc25238a65b02 > Author: Kai Tetzlaff > Date: 2023-01-19 03:16:14 +0100 > > Fix bug in 'sieve-manage--append-to-log' > > * lisp/net/sieve-manage.el (sieve-manage--append-to-log): Fix > log buffer creation. (Bug#54154) Do not merge to master. Well, I don't see this change in my checkout (and even in a fresh clone) and sieve-manage still produces the reported error. Last entry of 'git log lisp/net/sieve-manage.el' is: commit cae528457cb862dc886a34240c9d4c73035b6659 Author: Eli Zaretskii Date: Sun Jan 1 05:31:12 2023 -0500 ; Add 2023 to copyright years. -- Herbert From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 23 08:40:26 2023 Received: (at 54154) by debbugs.gnu.org; 23 Jan 2023 13:40:27 +0000 Received: from localhost ([127.0.0.1]:53690 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJx3i-00012V-IJ for submit@debbugs.gnu.org; Mon, 23 Jan 2023 08:40:26 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59646) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJx3g-00012H-8L for 54154@debbugs.gnu.org; Mon, 23 Jan 2023 08:40:24 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pJx3a-0003xg-Ix; Mon, 23 Jan 2023 08:40:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=2OCfkWaP+zCZ2Hx7yACB70EhMnMbJqGDxhhtQ0+r98Y=; b=EdGJw7xrUIMT PMiUDxBV4SWICjtvPCvnxs5cNCwcUCwVDEXzAqjjOZLnDd5XZofVy6DgFEZr+IASNhVQ8qtPLR20P iQPItJL0XAs/kF5DFnVQ1m4auz0zD4m98qdAOnj7mvg0UHzadlW93jJ1iIxDgXHcSXuk1do0vIl1/ FGfiV74WMBr5Jk6BzrJe+GXR2qcYd/jJKvq6jvBm93ZCZGwENBxei87XJYc7av2uWKCi4VzKLSiaH 7aXkaGlLDme7hUvoo+j/pjl++Pnk7l6UW/fQpn/tTGIOTY4NRcutNxyXhIlpri+P9gevTtPIQpwkv MChnBjRwKZyKb2NjkA/Zdg==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pJx3a-0002NP-1o; Mon, 23 Jan 2023 08:40:18 -0500 Date: Mon, 23 Jan 2023 15:40:26 +0200 Message-Id: <83sfg1id2t.fsf@gnu.org> From: Eli Zaretskii To: Kai Tetzlaff In-Reply-To: <87ilgxs8vl.fsf@tetzco.de> (message from Kai Tetzlaff on Mon, 23 Jan 2023 14:01:02 +0100) Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) References: <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Kai Tetzlaff > Cc: Eli Zaretskii , larsi@gnus.org, 54154@debbugs.gnu.org > Date: Mon, 23 Jan 2023 14:01:02 +0100 > > "Herbert J. Skuhra" writes: > > > On Mon, Jan 23, 2023 at 01:59:09AM +0100, Kai Tetzlaff wrote: > >> Just a quick update (also answering my own question(s) below): > >> > >> Regarding the status of this bug report: The fix for emacs-29 which was > >> initially intended to be emacs-29 only has (inadvertently?) found its way > >> to master. > > > > No, the change was not merged to master. > Hmm, what about: > > $ git log --grep "^Fix bug in 'sieve-manage--append-to-log" origin/master > commit 12d7670b90f66f1d45a8c69d9acfc25238a65b02 > Author: Kai Tetzlaff > Date: 2023-01-19 03:16:14 +0100 > > Fix bug in 'sieve-manage--append-to-log' > > * lisp/net/sieve-manage.el (sieve-manage--append-to-log): Fix > log buffer creation. (Bug#54154) Do not merge to master. The "Do not merge to master" part prevents its merging to the master branch. This was according to your request: you intended (and still do, AFAIU) to fix this differently on the master branch, for Emacs 30. From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 23 08:57:36 2023 Received: (at 54154) by debbugs.gnu.org; 23 Jan 2023 13:57:36 +0000 Received: from localhost ([127.0.0.1]:53712 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJxKK-0001V6-7F for submit@debbugs.gnu.org; Mon, 23 Jan 2023 08:57:36 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:47259) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJxKH-0001Up-SG for 54154@debbugs.gnu.org; Mon, 23 Jan 2023 08:57:34 -0500 Received: from mail.tetzco.de (ipb21a9472.dynamic.kabel-deutschland.de [178.26.148.114]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id E00CABD078; Mon, 23 Jan 2023 14:57:27 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id 7605F6C00B2; Mon, 23 Jan 2023 15:17:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674483438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=+qviNin+PM3AhriCV8mqAtB3lWG5ODX9bHpTZMqMxSs=; b=rxFuUm7fsYpDBAklk9HuI3EVmmRDSyBZ9ss/1g1E3/3DB8nyG5uZB6jG5RrLgXT2ZKLBU8 ohRQ6IULBVcVq4QN9HknCAnAgLGMLDOw5zF1ntduICWj7hO4R+ZB7wowzRriaahjDy0VVv Vbe8dcLhGvoRLn3HgVXptZIE+WlHQWrBmh3hZ5hafhx+wpkbIfvSy8828u9V46zXddQS+f FqlmpuU0t5Q/l1wLgN0yMkJrnS4j0604TEJi2MDxajGKrZslMOFTVtQ8+LgZ2quOQLf4D7 lr5gjPIwqgq3OGobFgbYRD+UcfngGOnevp5pi1pCYuIoJYkqcSmKLvwTaZpk5nfh6F4g7m uA0bSV/otEB+j/xWEjX4dJCtbi/ahnEdgB9Vfm5mck+CLFyAUargpkkBTZwU2eWqQ+B0xQ gxRHjYIlD1fArJGLaS+ykFq6PqFQTpHjN3mCBWoGHKdIPXJ9txK7O+g3+HkMYUwvfhNX6N 65ZJXRLvnfBgrLIZ+CNHo+UCwkKtxD2Sg6AnTF0L/LTByxQReK2UOUb+qTZOfkKSync7I+ NOXJJT3FZlWPigQq05iMuf4nuHTSd+ox6ZCqq8nwtI0jFLKfriyVZivQFsCIie5sSgNx0o QO3ioiB6xvAhnQ9B3iWaBYjXz9ofpySLXNQDAM5w2BYCVyj5/rvLc= From: Kai Tetzlaff To: "Herbert J. Skuhra" Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: (Herbert J. Skuhra's message of "Mon, 23 Jan 2023 14:36:44 +0100") References: <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> Date: Mon, 23 Jan 2023 14:57:19 +0100 Message-ID: <878rhts69s.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: 7605F6C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.83 / 30.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM(-3.00)[-1.000]; GENERIC_REPUTATION(-0.73)[-0.72664466200185]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ZERO(0.00)[0]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_FROM(0.00)[bug]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: Eli Zaretskii , 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) "Herbert J. Skuhra" writes: > On Mon, Jan 23, 2023 at 02:01:02PM +0100, Kai Tetzlaff wrote: >> >> Hmm, what about: >> >> $ git log --grep "^Fix bug in 'sieve-manage--append-to-log" origin/master >> commit 12d7670b90f66f1d45a8c69d9acfc25238a65b02 >> Author: Kai Tetzlaff >> Date: 2023-01-19 03:16:14 +0100 >> >> Fix bug in 'sieve-manage--append-to-log' >> >> * lisp/net/sieve-manage.el (sieve-manage--append-to-log): Fix >> log buffer creation. (Bug#54154) Do not merge to master. > > > Well, I don't see this change in my checkout (and even in a fresh clone) > and sieve-manage still produces the reported error. > > Last entry of 'git log lisp/net/sieve-manage.el' is: > > commit cae528457cb862dc886a34240c9d4c73035b6659 > Author: Eli Zaretskii > Date: Sun Jan 1 05:31:12 2023 -0500 > > ; Add 2023 to copyright years. You're right! And I learned something about git again. `git log --grep ...` apparently silently ignores my origin/master ref and greps through all available refs. From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 23 09:27:57 2023 Received: (at 54154) by debbugs.gnu.org; 23 Jan 2023 14:27:58 +0000 Received: from localhost ([127.0.0.1]:53749 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJxnh-0002JJ-Km for submit@debbugs.gnu.org; Mon, 23 Jan 2023 09:27:57 -0500 Received: from smtp-out1.suse.de ([195.135.220.28]:34136) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJxnf-0002J5-Lw for 54154@debbugs.gnu.org; Mon, 23 Jan 2023 09:27:56 -0500 Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id C4A5E339C7; Mon, 23 Jan 2023 14:27:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1674484069; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jI6hJkUM4kDgR+7itRRJstCbPfrhvibVaWGqw/ESIj8=; b=ehkeeaGgezGya4JDW3IAdl/BSk76/XQCPK5Nsw03RyTmY4xHZflIMnWBJtZPtnYRrjdiQP pageJGkyncSJGpzUlC9UKHL8KxVPgoHoR9QnNc7tO//SPzJXOxBhb/0ZbniI7wB8rTC2Mb QQUA0ZE7kUXov1aetwxf+FNn0WKGQ84= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1674484069; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jI6hJkUM4kDgR+7itRRJstCbPfrhvibVaWGqw/ESIj8=; b=2oDYfsaU3xMQYHs2xBXL1TXu69kQOfa36MZOtCMdLwKYu0cB978DuT0DsF1phcRD0v0VVv QuIINbvawX23vuCA== Received: from hawking.suse.de (unknown [10.168.4.11]) by relay2.suse.de (Postfix) with ESMTP id 9912C2C141; Mon, 23 Jan 2023 14:27:49 +0000 (UTC) Received: by hawking.suse.de (Postfix, from userid 17005) id 671CB44384A; Mon, 23 Jan 2023 15:27:49 +0100 (CET) From: Andreas Schwab To: Kai Tetzlaff Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) References: <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> <878rhts69s.fsf@tetzco.de> X-Yow: I haven't been married in over six years, but we had sexual counseling every day from Oral Roberts!! Date: Mon, 23 Jan 2023 15:27:49 +0100 In-Reply-To: <878rhts69s.fsf@tetzco.de> (Kai Tetzlaff's message of "Mon, 23 Jan 2023 14:57:19 +0100") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54154 Cc: larsi@gnus.org, "Herbert J. Skuhra" , 54154@debbugs.gnu.org, Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On Jan 23 2023, Kai Tetzlaff wrote: > And I learned something about git again. `git log --grep ...` apparently > silently ignores my origin/master ref and greps through all available > refs. It searches through all commit messages that you would see without the --grep option. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 23 11:22:36 2023 Received: (at 54154) by debbugs.gnu.org; 23 Jan 2023 16:22:36 +0000 Received: from localhost ([127.0.0.1]:55173 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJzab-0005j2-85 for submit@debbugs.gnu.org; Mon, 23 Jan 2023 11:22:36 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:54865) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pJzaV-0005ig-O7 for 54154@debbugs.gnu.org; Mon, 23 Jan 2023 11:22:31 -0500 Received: from mail.tetzco.de (ipb21a9472.dynamic.kabel-deutschland.de [178.26.148.114]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id 5AB03BD078; Mon, 23 Jan 2023 17:22:20 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id 10E946C00B2; Mon, 23 Jan 2023 17:42:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674492134; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=hjnFaLNpSK/BW0c6a+eDDWH013DERZ1VTdr68etUxpQ=; b=t1c2PfOO7R9KnXXgBk8eN4bsxM+pn+JvVC/V1VULkE8S6zsBxSTOTTHfeoiL0FM7tthUjj Ho+7NTakhzFUOg8m1EnziJzfvYLDIuJHs/cp8JHMk0hm0i3VRA72kgbrlSavu/QSJE+cAS AzBBbMjlJsmRgxR/ImTR/jufe8kAL49UBU7BXZMZlXLgK+ndln1U9SHhQU1b620ACueGgn TrVWgkp1by0/Mt/JiKkxjXneifNZ3K8YAhXpbnCUpcG3/QSwrmOYTvezEbUa43u796PBtP 62eejEJpvFUaZ39Bn5+11ZeDrMe6c9+3icbNpYLcSzFQQAWHjCN37HH/3HeL/xzN3pKwrU JV4b/XU5Z6ccFXQ6gLKl7AGKMwnIakE2OAN0/mAQLQzpx5LwFAW+EkY72DWZy8p92iKLQX Bjx5HlYNpyd0t12BJGx6JFzmd7ijb8wZTEUQlDtmGRwnmdAKqELPNwP9e7XWCnKu28ETu1 t4Pb60/OeD7GJJWt9ilwWSAyqfSbyXk6+3QC6aMqGz742PeaTN96H5U6PCtn9lVJ3gzAR2 D33dNjebiCycobrKjoL9xTStHYW4c5b5VnUBkY+pEOI+mipx64jvar78kmFKo/TGzTibVL X43ClfyiNgasMteuHqOlDhCllWX6tTMoN2OFUY7bXYcEZHTDr8q2o= From: Kai Tetzlaff To: Eli Zaretskii Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: <83sfg1id2t.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 23 Jan 2023 15:40:26 +0200") References: <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> <83sfg1id2t.fsf@gnu.org> Date: Mon, 23 Jan 2023 17:22:13 +0100 Message-ID: <871qnlrzka.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: 10E946C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.83 / 30.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM(-3.00)[-1.000]; GENERIC_REPUTATION(-0.73)[-0.72664477219286]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-diff]; RCVD_COUNT_ZERO(0.00)[0]; RCPT_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_SOME(0.00)[]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:+,6:+,7:+]; TAGGED_FROM(0.00)[bug]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Eli Zaretskii writes: >> From: Kai Tetzlaff >> Cc: Eli Zaretskii , larsi@gnus.org, 54154@debbugs.gnu.org >> Date: Mon, 23 Jan 2023 14:01:02 +0100 >> >> "Herbert J. Skuhra" writes: >> >> > On Mon, Jan 23, 2023 at 01:59:09AM +0100, Kai Tetzlaff wrote: >> >> Just a quick update (also answering my own question(s) below): >> >> >> >> Regarding the status of this bug report: The fix for emacs-29 which was >> >> initially intended to be emacs-29 only has (inadvertently?) found its way >> >> to master. >> > >> > No, the change was not merged to master. >> Hmm, what about: >> >> $ git log --grep "^Fix bug in 'sieve-manage--append-to-log" origin/master >> commit 12d7670b90f66f1d45a8c69d9acfc25238a65b02 >> Author: Kai Tetzlaff >> Date: 2023-01-19 03:16:14 +0100 >> >> Fix bug in 'sieve-manage--append-to-log' >> >> * lisp/net/sieve-manage.el (sieve-manage--append-to-log): Fix >> log buffer creation. (Bug#54154) Do not merge to master. > > The "Do not merge to master" part prevents its merging to the master > branch. Yes, I just was under the wrong assumption that this didn't work. > This was according to your request: you intended (and still do, AFAIU) > to fix this differently on the master branch, for Emacs 30. Correct, I'm still working on additional sieve-manage changes for master. However, I've now updated/reordered the patches. And from my POV, the following are ready to be applied to master: 1. 0001-Fix-bug-in-sieve-manage-append-to-log-and-do-some-re.patch: This is the master version of the patch which has been added to emacs-29. 2. 0002-Handle-BYE-in-sieve-manage-server-responses.patch: Adds support for BYE in server responses. 3. 0003-Some-minor-fixes-in-lisp-net-sieve.el.patch: Fixes some minor usability issues in sieve.el. 4. 0004-Autodetect-eol-type-of-sieve-manage-scripts.patch: Adds eol-type autodetection. Avoids showing '^M's in sieve script buffers for scripts which use CRLF EOLs. For the other two: 5. 0005-Add-test-lisp-net-sieve-manage-tests.el.patch: This adds a first test which verifies that multibyte script names and scripts can be downloaded without issues. I wrote this already last year, but then Lars committed what was on debbugs before I got to add it there. However, since I've already added additional tests in patch 0006-*, it might not be worth adding this one right now. Up to you... 6. 0006-WiP-new-encode-tested-OK.patch: This is contains my current WiP status. It works but is not ready/finished. It will probably take me one or two weeks to finish it. Some of the changes affect code which got updated/added in patches 0001/0002/0004. And there are a lot of new tests. Eli, how do you want to handle this? Apply the first patches now? Wait until I'm done with all all of them? Or, ...? Feedback appreciated! Thanks, Kai --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Fix-bug-in-sieve-manage-append-to-log-and-do-some-re.patch >From 3a3fb7d4a44d00d8450261a9bc5eca08014c3594 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 23 Jan 2023 15:28:57 +0100 Subject: [PATCH 1/6] Fix bug in sieve-manage--append-to-log and do some refactoring - simplify buffer creation and setup for process and log buffers - handle all encoding tasks in `sieve-manage-send' (and thus simplify `sieve-manage-putscript') - add/update doc-strings - update some comments * lisp/net/sieve-manage.el (sieve-manage-coding-system-for-read) (sieve-manage-coding-system-for-write): Remove unused constants. (sieve-manage--convert-multibyte): Renamed from `sieve-manage-encode', change misleading argument name (utf8-string -> str), use `sieve-manage--coding-system'. (sieve-manage--coding-system): New constant. (sieve-manage--set-buffer-maybe-append-text): New function. (sieve-manage-open-server): Change :coding property of `open-network-stream' from 'raw-text-unix to 'binary. (sieve-manage--append-to-log): Use `sieve-manage--set-buffer-maybe-append-text' to fix log buffer creation. (sieve-manage-decode) (sieve-manage-make-process-buffer): Use `sieve-manage--set-buffer-maybe-append-text'. (sieve-manage-send): Handle all encoding steps (including the 'literal-c2s' encoding previously done in `sieve-manage-putscript'). (sieve-manage-putscript): Move all encoding steps to `sieve-manage-send'. --- lisp/net/sieve-manage.el | 119 ++++++++++++++++++++++++++------------- 1 file changed, 81 insertions(+), 38 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 5bee4f4c4ad..b6d3fa573e8 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -58,7 +58,7 @@ ;; ;; References: ;; -;; draft-martin-managesieve-02.txt, +;; RFC5804, ;; "A Protocol for Remotely Managing Sieve Scripts", ;; by Tim Martin. ;; @@ -145,6 +145,15 @@ sieve-manage-ignore-starttls :type 'boolean) ;; Internal variables: +(defconst sieve-manage--coding-system 'utf-8-unix + "Use \\='utf-8-unix coding system for (network) communication. + +Defines the coding system used for the internal (process, log) +buffers and the network stream created to communicate with the +managesieve server. Using \\='utf-8-unix encoding corresponds to +the use of UTF-8 in rfc5804 (managesieve). The explicit use of +\\='-unix\\=' avoids EOL conversions (and thus keeps CRLF (\"\\r\\n\") +sequences intact).") (defconst sieve-manage-local-variables '(sieve-manage-server sieve-manage-port @@ -154,8 +163,6 @@ sieve-manage-local-variables sieve-manage-client-eol sieve-manage-server-eol sieve-manage-capability)) -(defconst sieve-manage-coding-system-for-read 'binary) -(defconst sieve-manage-coding-system-for-write 'binary) (defvar sieve-manage-stream nil) (defvar sieve-manage-auth nil) (defvar sieve-manage-server nil) @@ -167,20 +174,45 @@ sieve-manage-process (defvar sieve-manage-capability nil) ;; Internal utility functions +(defun sieve-manage--set-buffer-maybe-append-text (buffer-name + &rest args) + "Append ARGS to buffer named BUFFER-NAME and return buffer. + +To be used with process and log buffers. When the buffer doesn't +exist, it gets created and configure as follows: +- set coding system to the value of `sieve-manage--coding-system', +- set buffer to single-byte mode, +- set `after-change-functions' to nil to avoid those + functions messing with buffer content, +- disable undo (to save a bit of memory and improve + performance). + +ARGS can be a nil, a string or a list of strings. If no +ARGS are provided, the content of buffer will not be +modified." + (let* ((existing-buffer (get-buffer buffer-name)) + (buffer (or existing-buffer + (get-buffer-create buffer-name)))) + (with-current-buffer buffer + (unless existing-buffer + (set-buffer-file-coding-system sieve-manage--coding-system) + (set-buffer-multibyte nil) + (setq-local after-change-functions nil) + (buffer-disable-undo)) + (when args + (goto-char (point-max)) + (apply #'insert args))) + buffer)) + (defun sieve-manage--append-to-log (&rest args) - "Append ARGS to sieve-manage log buffer. + "Append ARGS to `sieve-manage-log' buffer. -ARGS can be a string or a list of strings. -The buffer to use for logging is specifified via -`sieve-manage-log'. If it is nil, logging is disabled." +If `sieve-manage-log' is nil, logging is disabled. See also +`sieve-manage--set-buffer-maybe-append-text'." (when sieve-manage-log - (with-current-buffer (or (get-buffer sieve-manage-log) - (with-current-buffer - (get-buffer-create sieve-manage-log) - (set-buffer-multibyte nil) - (buffer-disable-undo))) - (goto-char (point-max)) - (apply #'insert args)))) + (apply #'sieve-manage--set-buffer-maybe-append-text + sieve-manage-log + args))) (defun sieve-manage--message (format-string &rest args) "Wrapper around `message' which also logs to sieve manage log. @@ -202,28 +234,29 @@ sieve-manage--error (sieve-manage--append-to-log msg "\n") (error msg))) -(defun sieve-manage-encode (utf8-string) - "Convert UTF8-STRING to managesieve protocol octets." - (encode-coding-string utf8-string 'raw-text t)) +(defun sieve-manage--convert-multibyte (str) + "Convert multibyte STR to unibyte octet string. +The conversion is done using `sieve-manage--coding-system'." + (encode-coding-string str sieve-manage--coding-system t)) (defun sieve-manage-decode (octets &optional buffer) - "Convert managesieve protocol OCTETS to utf-8 string. + "Convert managesieve protocol OCTETS to UTF-8 string. If optional BUFFER is non-nil, insert decoded string into BUFFER." (when octets ;; eol type unix is required to preserve "\r\n" - (decode-coding-string octets 'utf-8-unix t buffer))) + (decode-coding-string octets + sieve-manage--coding-system + t buffer))) (defun sieve-manage-make-process-buffer () - (with-current-buffer - (generate-new-buffer (format " *sieve %s:%s*" - sieve-manage-server - sieve-manage-port)) - (mapc #'make-local-variable sieve-manage-local-variables) - (set-buffer-multibyte nil) - (setq-local after-change-functions nil) - (buffer-disable-undo) - (current-buffer))) + (let ((buffer (sieve-manage--set-buffer-maybe-append-text + (format " *sieve %s:%s*" + sieve-manage-server + sieve-manage-port)))) + (with-current-buffer buffer + (mapc #'make-local-variable sieve-manage-local-variables)) + buffer)) (defun sieve-manage-erase (&optional p buffer) (with-current-buffer (or buffer (current-buffer)) @@ -244,8 +277,7 @@ sieve-manage-open-server (open-network-stream "SIEVE" buffer server port :type stream - ;; eol type unix is required to preserve "\r\n" - :coding 'raw-text-unix + :coding 'binary :capability-command "CAPABILITY\r\n" :end-of-command "^\\(OK\\|NO\\).*\n" :success "^OK.*\n" @@ -480,9 +512,7 @@ sieve-manage-havespace (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "PUTSCRIPT \"%s\" {%d+}%s%s" name - (length (sieve-manage-encode content)) - sieve-manage-client-eol content)) + (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) (sieve-manage-parse-okno))) (defun sieve-manage-deletescript (name &optional buffer) @@ -609,11 +639,24 @@ sieve-manage-parse-listscripts data rsp))) -(defun sieve-manage-send (cmdstr) - (setq cmdstr (sieve-manage-encode - (concat cmdstr sieve-manage-client-eol))) - (sieve-manage--append-to-log cmdstr) - (process-send-string sieve-manage-process cmdstr)) +(defun sieve-manage-send (command &optional payload-str) + "Send COMMAND with optional PAYLOAD-STR. + +If non-nil, PAYLOAD-STR will be appended to COMMAND using the +\\='literal-s2c\\' representation according to rfc5804." + (let ((encoded (when payload-str (sieve-manage--convert-multibyte + payload-str))) + literal-c2s cmdstr) + (when encoded + (setq literal-c2s (format " {%d+}%s%s" + (length encoded) + sieve-manage-client-eol + encoded))) + (setq cmdstr (concat (sieve-manage--convert-multibyte command) + literal-c2s + sieve-manage-client-eol)) + (sieve-manage--append-to-log cmdstr) + (process-send-string sieve-manage-process cmdstr))) (provide 'sieve-manage) -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-Handle-BYE-in-sieve-manage-server-responses.patch >From 30695a712ac7fdecf276c2bb3806b8cbba20b576 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 23 Jan 2023 16:53:10 +0100 Subject: [PATCH 2/6] Handle BYE in sieve-manage server responses * lisp/net/sieve-manage.el (sieve-manage-regex-oknobye): New function. (sieve-manage-parse-oknobye): Renamed from `sieve-manage-parse-okno'. (sieve-manage-open-server) (sieve-sasl-auth) (sieve-manage-listscripts) (sieve-manage-putscript) (sieve-manage-deletescript) (sieve-manage-getscript) (sieve-manage-setactive) (sieve-manage-is-okno) (sieve-manage-parse-oknobye) (sieve-manage-parse-parse-string) (sieve-manage-parse-parse-crlf): Use `sieve-manage-regex-oknobye' to handle BYE in addition to OK and NO. (sieve-manage-open-server): Use `sieve-manager-(client|server)-eol' for consistency. * etc/NEWS: Mention the support for BYE. --- etc/NEWS | 12 +++++++++++ lisp/net/sieve-manage.el | 43 ++++++++++++++++++++++------------------ 2 files changed, 36 insertions(+), 19 deletions(-) diff --git a/etc/NEWS b/etc/NEWS index 5b8ab06086c..bebea918b11 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -195,6 +195,18 @@ This command adds a docstring comment to the current defun. If a comment already exists, point is only moved to the comment. It is bound to 'C-c C-d' in 'go-ts-mode'. +** sieve-manage + +--- +*** Support handling of BYE responses from managesieve servers. +The managesieve client in sieve-manage now handles BYE responses sent +be the server (in addition to OK and NO). This makes the +implementation more robust in case of e.g. timeouts and authentication +failures. + +Note: The special case of a REFERRAL/BYE responses is still not +handled by the client (s. RFC5804 for more details). + * New Modes and Packages in Emacs 30.1 diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index b6d3fa573e8..79f97f371e0 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -278,15 +278,17 @@ sieve-manage-open-server "SIEVE" buffer server port :type stream :coding 'binary - :capability-command "CAPABILITY\r\n" - :end-of-command "^\\(OK\\|NO\\).*\n" - :success "^OK.*\n" + :capability-command (concat "CAPABILITY" + sieve-manage-client-eol) + :end-of-command (concat "^\\(OK\\|NO\\|BYE\\).*" + sieve-manage-server-eol) + :success (concat "^OK.*" sieve-manage-server-eol) :return-list t :starttls-function (lambda (capabilities) (when (and (not sieve-manage-ignore-starttls) (string-match "\\bSTARTTLS\\b" capabilities)) - "STARTTLS\r\n"))))) + (concat "STARTTLS" sieve-manage-client-eol)))))) (setq sieve-manage-process proc) (setq sieve-manage-capability (sieve-manage-parse-capability (plist-get props :capabilities))) @@ -339,7 +341,7 @@ sieve-sasl-auth (setq rsp nil) (goto-char (match-end 0)) rsp)) - (setq rsp (sieve-manage-is-okno)))) + (setq rsp (sieve-manage-is-oknobye)))) (accept-process-output sieve-manage-process 1) (goto-char (point-min))) (sieve-manage-erase) @@ -508,17 +510,17 @@ sieve-manage-listscripts (defun sieve-manage-havespace (name size &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-putscript (name content &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "DELETESCRIPT \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-getscript (name output-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) @@ -526,17 +528,22 @@ sieve-manage-getscript (sieve-manage-decode (sieve-manage-parse-string) output-buffer) (sieve-manage-parse-crlf) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) (defun sieve-manage-setactive (name &optional buffer) (with-current-buffer (or buffer (current-buffer)) (sieve-manage-send (format "SETACTIVE \"%s\"" name)) - (sieve-manage-parse-okno))) + (sieve-manage-parse-oknobye))) ;; Protocol parsing routines +(defun sieve-manage-regexp-oknobye () + "Return regexp for managesieve 'response-oknobye'." + (concat + "^\\(OK\\|NO\\|BYE\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" + sieve-manage-server-eol)) (defun sieve-manage-wait-for-answer () - (let ((pattern "^\\(OK\\|NO\\).*\n") + (let ((pattern (sieve-manage-regexp-oknobye)) pos) (while (not pos) (setq pos (search-forward-regexp pattern nil t)) @@ -554,10 +561,8 @@ sieve-manage-ok-p (defun sieve-manage-no-p (rsp) (string= (downcase (or (car-safe rsp) "")) "no")) -(defun sieve-manage-is-okno () - (when (looking-at (concat - "^\\(OK\\|NO\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" - sieve-manage-server-eol)) +(defun sieve-manage-is-oknobye () + (when (looking-at (sieve-manage-regexp-oknobye)) (let ((status (match-string 1)) (resp-code (match-string 3)) (response (match-string 5))) @@ -566,12 +571,12 @@ sieve-manage-is-okno (setq response (sieve-manage-is-string))) (list status resp-code response)))) -(defun sieve-manage-parse-okno () +(defun sieve-manage-parse-oknobye () (let (rsp) (while (null rsp) (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) - (setq rsp (sieve-manage-is-okno))) + (setq rsp (sieve-manage-is-oknobye))) (sieve-manage-erase) rsp)) @@ -605,7 +610,7 @@ sieve-manage-parse-string (accept-process-output (get-buffer-process (current-buffer)) 1) (goto-char (point-min)) (unless (setq rsp (sieve-manage-is-string)) - (when (sieve-manage-no-p (sieve-manage-is-okno)) + (when (sieve-manage-no-p (sieve-manage-is-oknobye)) ;; simple `error' is enough since `sieve-manage-erase' ;; already adds the server response to the log (error (sieve-manage-erase))))) @@ -619,7 +624,7 @@ sieve-manage-parse-crlf (defun sieve-manage-parse-listscripts () (let (tmp rsp data) (while (null rsp) - (while (null (or (setq rsp (sieve-manage-is-okno)) + (while (null (or (setq rsp (sieve-manage-is-oknobye)) (setq tmp (sieve-manage-decode (sieve-manage-is-string))))) (accept-process-output (get-buffer-process (current-buffer)) 1) -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0003-Some-minor-fixes-in-lisp-net-sieve.el.patch >From 75348dd4bcd40ed2ce732ecd91a887fb1ce0be1d Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 23 Jan 2023 16:53:24 +0100 Subject: [PATCH 3/6] Some minor fixes in lisp/net/sieve.el * lisp/net/sieve.el (sieve-next-line) (sieve-prev-line): Handle situations where point in `sieve-buffer' is either before or after the list of server side scripts. (sieve-server-script-list): New variable. (sieve-refresh-scriptlist) (sieve-upload): Use `sieve-server-script-list' to make sure that local list of server side scripts in`sieve-buffer' is up-to-date after uploading a (new) script. (sieve-edit-script): Improve upload hint in the message area of the created sieve script buffer. --- lisp/net/sieve.el | 49 +++++++++++++++++++++++++++++++---------------- 1 file changed, 33 insertions(+), 16 deletions(-) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index fbd07dee27c..a73584f203c 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -95,6 +95,8 @@ sieve-template (defvar sieve-manage-buffer nil) (defvar sieve-buffer-header-end nil) +(defvar sieve-server-script-list nil + "Current list of server-side sieve scripts.") (defvar sieve-buffer-script-name nil "The real script name of the buffer.") (make-local-variable 'sieve-buffer-script-name) @@ -219,8 +221,13 @@ sieve-edit-script (setq sieve-buffer-script-name name) (goto-char (point-min)) (set-buffer-modified-p nil) - (message "Press %s to upload script to server." - (substitute-command-keys "\\[sieve-upload]")))) + (message (string-join '("Type %s to upload script" + "%s to upload and kill buffer" + "%s to return to *sieve* buffer") + ", ") + (substitute-command-keys "\\[sieve-upload]") + (substitute-command-keys "\\[sieve-upload-and-kill]") + (substitute-command-keys "\\[sieve-manage]")))) (defmacro sieve-change-region (&rest body) "Turn off sieve-region before executing BODY, then re-enables it after. @@ -235,23 +242,28 @@ sieve-next-line (interactive) (unless arg (setq arg 1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "End of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "End of list")) + (goto-char (next-overlay-change sieve-buffer-header-end)))) (defun sieve-prev-line (&optional arg) (interactive) (unless arg (setq arg -1)) - (if (save-excursion - (forward-line arg) - (sieve-script-at-point)) - (sieve-change-region - (forward-line arg)) - (message "Beginning of list"))) + (if (sieve-script-at-point) + (if (save-excursion + (forward-line arg) + (sieve-script-at-point)) + (sieve-change-region + (forward-line arg)) + (message "Beginning of list")) + (goto-char (previous-overlay-change (point-max))) + (beginning-of-line))) (defun sieve-help () "Display help for various sieve commands." @@ -285,7 +297,7 @@ sieve-highlight (overlay-put (car (overlays-at (point))) 'face (if on 'highlight 'default))) (defun sieve-insert-scripts (scripts) - "Format and insert LANGUAGE-LIST strings into current buffer at point." + "Format and insert SCRIPTS strings into current buffer at point." (while scripts (let ((p (point)) (ext nil) @@ -319,6 +331,9 @@ sieve-refresh-scriptlist (let* ((scripts (sieve-manage-listscripts sieve-manage-buffer)) (count (length scripts)) (keys (substitute-command-keys "\\[sieve-edit-script]"))) + (setq sieve-server-script-list + (mapcar (lambda (elt) (if (consp elt) (cdr elt) elt)) + scripts)) (insert (if (null scripts) (format @@ -361,7 +376,9 @@ sieve-upload (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." - (substitute-command-keys "\\[sieve-manage]")))) + (substitute-command-keys "\\[sieve-manage]")) + (when (not (member script-name sieve-server-script-list)) + (sieve-refresh-scriptlist)))) (set-buffer-modified-p nil)))) ;;;###autoload -- 2.39.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0004-Autodetect-eol-type-of-sieve-manage-scripts.patch >From ffcfac99012f9a34b4bd3e735ad2337df988b5a8 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 23 Jan 2023 16:53:39 +0100 Subject: [PATCH 4/6] Autodetect eol-type of sieve(-manage) scripts When using `sieve-manage-getscript' to download a sieve script, sieve-manage now automatically detects the eol-type (either 'utf-8-unix or 'utf-8-dos), uses it to decode the script data and sets `buffer-file-coding-system' accordingly. This gets rid of '^M' in sieve script buffers (for scripts which use CRLF type EOLs). The same eol-type is then used to encode the script during upload with `sieve-manage-putscript'. * lisp/net/sieve-manage.el (sieve-manage--detect-buffer-coding-system): New function which detects the eol-type of a downloaded script and returns either 'utf-8-unix, 'utf-8-dos or 'utf-8-mac (if the latter is actually supported by any existing server or even makes sense ...). (sieve-manage-decode): Use `sieve-manage--guess-buffer-coding-system' to decode downloaded script data with the correct coding-system and sets the `buffer-file-coding-system' of the resulting sieve script buffer. (sieve-manage-putscript): Now takes a sieve script buffer (instead of a string) argument and forwards it to `sieve-manage-send'. (sieve-manage-send): Now also uses a (payload-)buffer instead of a string. The `buffer-file-coding-system' of the buffer is then used when encoding the payload in order to use the correct eol-type. * lisp/net/sieve.el: (sieve-upload): Adapt to changed argument type of `sieve-manage-putscript'. * etc/NEWS: Add a short description of the changes. --- etc/NEWS | 10 +++++++ lisp/net/sieve-manage.el | 56 ++++++++++++++++++++++++++++++---------- lisp/net/sieve.el | 6 ++--- 3 files changed, 55 insertions(+), 17 deletions(-) diff --git a/etc/NEWS b/etc/NEWS index bebea918b11..769aed9dfa8 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -207,6 +207,16 @@ failures. Note: The special case of a REFERRAL/BYE responses is still not handled by the client (s. RFC5804 for more details). +--- +*** Autodetect eol-type of downloaded sieve scripts. +When a downloaded script contained CRLF type EOLs, they caused '^M's +to appear in the sieve script edit buffer. To avoid that, the +eol-type of sieve scripts is now detected during download via +'sieve-manage-getscript', used when decoding the data and stored in +'buffer-file-coding-system' of the script buffer. The +'buffer-file-coding-system' is then also used for encoding during +upload by 'sieve-manage-putscript'. + * New Modes and Packages in Emacs 30.1 diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 79f97f371e0..cecdeb6f8e7 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -239,15 +239,37 @@ sieve-manage--convert-multibyte The conversion is done using `sieve-manage--coding-system'." (encode-coding-string str sieve-manage--coding-system t)) +(defun sieve-manage--detect-buffer-coding-system (&optional buffer) + "Return the coding system to be use for (sieve script) BUFFER. + +Since rfc5804 requires sieve scripts to use (a subset of) UTF-8, the +returned coding system is of type \\='utf-8 with either \\='-unix\\=', +\\='-dos\\=' or \\='-mac\\=' eol-type." + (with-current-buffer (or buffer (current-buffer)) + (let ((coding-system (detect-coding-region + (point-min) (point-max) t))) + (alist-get (coding-system-eol-type coding-system) + '((0 . utf-8-unix) + (1 . utf-8-dos) + (2 . utf-8-mac)) + 'utf-8-unix)))) + (defun sieve-manage-decode (octets &optional buffer) "Convert managesieve protocol OCTETS to UTF-8 string. If optional BUFFER is non-nil, insert decoded string into BUFFER." (when octets - ;; eol type unix is required to preserve "\r\n" - (decode-coding-string octets - sieve-manage--coding-system - t buffer))) + (if buffer + (with-current-buffer buffer + (insert octets) + (let ((coding-system + (sieve-manage--detect-buffer-coding-system))) + (set-buffer-file-coding-system coding-system) + (decode-coding-region (point-min) (point-max) + coding-system)) + buffer) + (decode-coding-string + octets sieve-manage--coding-system t)))) (defun sieve-manage-make-process-buffer () (let ((buffer (sieve-manage--set-buffer-maybe-append-text @@ -512,9 +534,9 @@ sieve-manage-havespace (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) (sieve-manage-parse-oknobye))) -(defun sieve-manage-putscript (name content &optional buffer) +(defun sieve-manage-putscript (name script-buffer &optional buffer) (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) content) + (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) script-buffer) (sieve-manage-parse-oknobye))) (defun sieve-manage-deletescript (name &optional buffer) @@ -644,14 +666,20 @@ sieve-manage-parse-listscripts data rsp))) -(defun sieve-manage-send (command &optional payload-str) - "Send COMMAND with optional PAYLOAD-STR. - -If non-nil, PAYLOAD-STR will be appended to COMMAND using the -\\='literal-s2c\\' representation according to rfc5804." - (let ((encoded (when payload-str (sieve-manage--convert-multibyte - payload-str))) - literal-c2s cmdstr) +(defun sieve-manage-send (command &optional payload-buffer) + "Send COMMAND with optional string from PAYLOAD-BUFFER. + +If non-nil, the content of PAYLOAD-BUFFER will be appended to +COMMAND using the \\='literal-s2c\\=' representation according to rfc5804." + (let* ((encoded (when (and payload-buffer + (> (buffer-size payload-buffer) 0)) + (with-current-buffer payload-buffer + (encode-coding-region + (point-min) (point-max) + (buffer-local-value 'buffer-file-coding-system + payload-buffer) + t)))) + cmdstr literal-c2s) (when encoded (setq literal-c2s (format " {%d+}%s%s" (length encoded) diff --git a/lisp/net/sieve.el b/lisp/net/sieve.el index a73584f203c..437486c8a13 100644 --- a/lisp/net/sieve.el +++ b/lisp/net/sieve.el @@ -366,13 +366,13 @@ sieve-upload (interactive) (when (or (get-buffer sieve-buffer) (save-current-buffer (call-interactively 'sieve-manage))) - (let ((script (buffer-string)) + (let ((script-buffer (current-buffer)) (script-name (file-name-sans-extension (buffer-name))) err) (with-current-buffer (get-buffer sieve-buffer) - (setq err (sieve-manage-putscript + (setq err (sieve-manage-putscript (or name sieve-buffer-script-name script-name) - script sieve-manage-buffer)) + script-buffer sieve-manage-buffer)) (if (not (sieve-manage-ok-p err)) (message "Sieve upload failed: %s" (nth 2 err)) (message "Sieve upload done. Use %s to manage scripts." -- 2.39.0 --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0005-Add-test-lisp-net-sieve-manage-tests.el.patch Content-Transfer-Encoding: quoted-printable >From 216571ea17e3e1d4aa670ba6c554d434f5c762db Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 23 Jan 2023 15:32:40 +0100 Subject: [PATCH 5/6] Add test/lisp/net/sieve-manage-tests.el * test/lisp/net/sieve-manage-tests.el: (mk-literal-s2c) (mk-rsp-oknobye) (mk-rsp-getscript-ok) (managesieve-getscript): Test helper functions. (ert/managesieve-getscript-multibyte): Test `sieve-manage-getscript' with script names containing multibyte characters. --- test/lisp/net/sieve-manage-tests.el | 104 ++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 test/lisp/net/sieve-manage-tests.el diff --git a/test/lisp/net/sieve-manage-tests.el b/test/lisp/net/sieve-mana= ge-tests.el new file mode 100644 index 00000000000..010c9071608 --- /dev/null +++ b/test/lisp/net/sieve-manage-tests.el @@ -0,0 +1,104 @@ +;;; sieve-manage-tests.el --- tests for sieve-manage.el -*- lexical-= binding: t; -*- + +;; Copyright (C) 2022 Free Software Foundation, Inc. + +;; Author: Kai Tetzlaff + +;; This file is part of GNU Emacs. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see . + +;;; Commentary: + +;;; Code: + +(require 'ert) +(require 'sieve-manage) + +(defvar sieve-script-multibyte-unix + "if body :matches \"=C3=A4\" { stop; }\n" + "Simple multibyte sieve script.") + +(defun mk-literal-s2c (string) + "Encode STRING to managesieve 'literal-s2c'." + (let ((data (sieve-manage-encode string))) + (concat (format "{%d}\r\n" (length data)) + data))) + +(defun mk-rsp-oknobye (type &optional resp-code string) + "Encode TYPE, RESP-CODE and STRING to managesieve 'response-oknobye'." + (when (memq type '(OK NO BYE)) + (concat + (mapconcat #'identity + (delq nil + (list (symbol-name type) + (when resp-code + (format "(%s)" resp-code)) + (when string + (format "\"%s\"" + (sieve-manage-encode string))))) + " ") + "\r\n"))) +;; (mk-rsp-oknobye 'OK nil "Getscript completed.") +;; (mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") + +(defun mk-rsp-getscript-ok (script) + "Encode SCRIPT to managesieve 'response-getscript'." + (concat (mk-literal-s2c script) + "\r\n" + (mk-rsp-oknobye 'OK "Getscript completed."))) +;; (mk-rsp-getscript-ok sieve-script-multibyte-unix) + +(defun managesieve-getscript (script) + "Simulate managesieve getscript response to test +`sieve-manage-getscript' function." + (let* ((script-name "test.sieve") + ;; `sieve-manage-server' and `sieve-manage-port' are used in + ;; `sieve-manage-make-process-buffer' + (sieve-manage-server) + (sieve-manage-port "sieve") + (sieve-buffer (sieve-manage-make-process-buffer)) + (output-buffer (generate-new-buffer script-name))) + ;; use cl-letf to mock some functions in call chain of + ;; sieve-manage-getscript + (cl-letf (((symbol-function 'sieve-manage-send) + ;; simulate sieve server response with a single + ;; multibyte character `=C3=A4` + (lambda (_) + (with-current-buffer sieve-buffer + (goto-char (point-min)) + (insert (mk-rsp-getscript-ok script))))) + ((symbol-function 'accept-process-output) + (lambda (&optional _ _ _ _) nil)) + ((symbol-function 'get-buffer-process) (lambda (_) nil))) + ;; extract sieve script from sieve-buffer and put it into + ;; output-buffer + (sieve-manage-getscript script-name output-buffer sieve-buffer) + ;; extract script from output-buffer and return it as a string + (let ((script (with-current-buffer output-buffer + (set-buffer-modified-p nil) + (buffer-string)))) + ;; cleanup + (kill-buffer sieve-buffer) + (kill-buffer output-buffer) + (when (get-buffer sieve-manage-log) + (kill-buffer sieve-manage-log)) + ;; return script + script)))) + +(ert-deftest ert/managesieve-getscript-multibyte () + (should (string=3D sieve-script-multibyte-unix + (managesieve-getscript sieve-script-multibyte-unix)))) + +;;; sieve-manage-tests.el ends here --=20 2.39.0 --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename=0006-WiP-new-encode-tested-OK.patch Content-Transfer-Encoding: quoted-printable >From 599501989322142f751de171ff4c138c0ec05956 Mon Sep 17 00:00:00 2001 From: Kai Tetzlaff Date: Mon, 23 Jan 2023 04:05:54 +0100 Subject: [PATCH 6/6] WiP: new encode, tested OK --- lisp/net/sieve-manage.el | 328 +++++++++++++++++++++------- test/lisp/net/sieve-manage-tests.el | 296 +++++++++++++++++++++---- 2 files changed, 504 insertions(+), 120 deletions(-) diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index cecdeb6f8e7..6a343874de4 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -144,6 +144,25 @@ sieve-manage-ignore-starttls :version "26.1" :type 'boolean) =20 +(defconst sieve-manage-supported-commands + '(CAPABILITY ;; send automatically when connecting + STARTTLS ;; supported implicilty by sasl/networking + AUTHENTICATE + LISTSCRIPTS + SETACTIVE + GETSCRIPT + DELETESCRIPT + HAVESPACE + PUTSCRIPT + LOGOUT) + "Commands supported by the \\=3D'sieve-manage\\=3D' client. + +The following commands from rfc5804 are not implemented: +- NOOP +- RENAMESCRIPT +- CHECKSCRIPT +- UNAUTHENTICATE") + ;; Internal variables: (defconst sieve-manage--coding-system 'utf-8-unix "Use \\=3D'utf-8-unix coding system for (network) communication. @@ -173,6 +192,18 @@ sieve-manage-state (defvar sieve-manage-process nil) (defvar sieve-manage-capability nil) =20 +(define-error 'sieve-manage-error + "Unknown sieve-manage error") +(define-error 'sieve-manage-encode-error + "sieve-manage encoding error" 'sieve-manage-error) +(define-error 'sieve-manage-server-error + "Managesieve server signaled an error" 'sieve-manage-error) +(define-error 'sieve-manage-client-error + "sieve-manage client error" 'sieve-manage-error) +(define-error 'sieve-manage-unsupported-cmd + "Command not supported by sieve-manage client." + 'sieve-manage-client-error) + ;; Internal utility functions (defun sieve-manage--set-buffer-maybe-append-text (buffer-name &rest args) @@ -216,7 +247,7 @@ sieve-manage--append-to-log =20 (defun sieve-manage--message (format-string &rest args) "Wrapper around `message' which also logs to sieve manage log. - +See `message' for the use of FORMAT-STRING and ARGS. See `sieve-manage--append-to-log'." (let ((ret (apply #'message (concat "sieve-manage: " format-string) @@ -224,21 +255,120 @@ sieve-manage--message (sieve-manage--append-to-log ret "\n") ret)) =20 -(defun sieve-manage--error (format-string &rest args) - "Wrapper around `error' which also logs to sieve manage log. +(defun sieve-manage--error (err format-string &rest args) + "Signal an error of type ERR. +ERR should be an error type derived from +\\=3D'sieve-manage-error\\=3D'. See `format' for the use of +FORMAT-STRING and ARGS. =20 -See `sieve-manage--append-to-log'." +Errors are also logged to sieve manage log (see +`sieve-manage--append-to-log')." (let ((msg (apply #'format - (concat "sieve-manage/ERROR: " format-string) + (concat "Error: " format-string) args))) (sieve-manage--append-to-log msg "\n") - (error msg))) + (signal err (list msg)))) =20 (defun sieve-manage--convert-multibyte (str) "Convert multibyte STR to unibyte octet string. The conversion is done using `sieve-manage--coding-system'." (encode-coding-string str sieve-manage--coding-system t)) =20 +(defun sieve-manage--mk-quoted (octet-str) + "Convert OCTET-STR to rfc5804 \\=3D'quoted\\=3D' string." + (or (stringp octet-str) + (sieve-manage--error 'sieve-manage-encode-error + "Argument must be a string")) + (format "\"%s\"" octet-str)) + +(defun sieve-manage--mk-literal-c2s (octet-str) + "Convert OCTET-STR to rfc5804 \\=3D'literal-c2s\\=3D' string." + (or (stringp octet-str) + (sieve-manage--error 'sieve-manage-encode-error + "Argument must be a string")) + (format "{%d+}%s%s" + (length octet-str) + sieve-manage-client-eol + octet-str)) + +(defun sieve-manage--mk-literal-s2c (octet-str) + "Convert OCTET-STR to rfc5804 \\=3D'literal-s2c\\=3D' string." + (or (stringp octet-str) + (sieve-manage--error 'sieve-manage-encode-error + "Argument must be a string")) + (format "{%d}%s%s" + (length octet-str) + sieve-manage-client-eol + octet-str)) + +(defun sieve-manage-encode (item) + "Convert ITEM to a rfc5804 protocol string. + +An ITEM can be a: +1. symbol (will be interpreted as sieve command name), +2. number (positive integer or 0), +3. string (will be converted to unibyte using \\=3D'utf-8-unix + coding system and encoded as \\=3D'quoted\\=3D' rfc5804 string), +4. buffer (buffer content will be converted to unibyte using + `buffer-file-coding-system and encoded as \\=3D'literal-c2s\\=3D' rfc58= 04 + string), +5. cons cell (TYPE . DATA) where the following combinations of TYPE + and DATA are supported: + - (number . ) where is an integer in the range of + [0..4294967296] + - (cmd-name . ) where is either a unibyte string or one of + the symbols from `sieve-manage-supported-commands' + - ( . ) where is one of \\=3D'quoted, + \\=3D'literal-c2s, \\=3D'literal-s2c and is a unibyte str= ing + - ( . ) where is one of \\=3D'mb-quoted, + \\=3D'mb-literal-c2s, \\=3D'mb-literal-s2c and is a (multibyte)= UTF-8 + string which will be converted to unibyte using coding system + \\=3D'utf-8-unix." + (cond + ((symbolp item) + (sieve-manage-encode (cons 'cmd-name item))) + ((and (integerp item) (>=3D item 0)) + (sieve-manage-encode (cons 'number item))) + ((stringp item) + ;; TODO: + ;; - check if character set of item requires use of 'literal-c2s + ;; - check if length of item requires use of 'literal-c2s + (sieve-manage-encode (cons 'quoted + (sieve-manage--convert-multibyte item)))) + ((bufferp item) + (sieve-manage-encode + (cons 'literal-c2s (with-current-buffer item + (encode-coding-region + (point-min) (point-max) + (buffer-local-value 'buffer-file-coding-system + item) + t))))) + ((consp item) + (let ((type (car item)) + (data (cdr item))) + (pcase type + ('number (format "%d" data)) + ('cmd-name + (when (and (symbolp data) + (not (memq data sieve-manage-supported-commands))) + (sieve-manage--error 'sieve-manage-unsupported-cmd + "Command '%s' is not supported" data)) + (format "%s" data)) + ('quoted (sieve-manage--mk-quoted data)) + ('literal-c2s (sieve-manage--mk-literal-c2s data)) + ('literal-s2c (sieve-manage--mk-literal-s2c data)) + ('mb-quoted (sieve-manage--mk-quoted + (sieve-manage--convert-multibyte data))) + ('mb-literal-c2s (sieve-manage--mk-literal-c2s + (sieve-manage--convert-multibyte data))) + ('mb-literal-s2c (sieve-manage--mk-literal-s2c + (sieve-manage--convert-multibyte data))) + (_ (sieve-manage--error 'sieve-manage-encode-error + "Unknown encoding type: '%s'" + type))))) + (t (sieve-manage--error 'sieve-manage-encode-error + "Don't know how to encode '%s'" item)))) + (defun sieve-manage--detect-buffer-coding-system (&optional buffer) "Return the coding system to be use for (sieve script) BUFFER. =20 @@ -257,7 +387,13 @@ sieve-manage--detect-buffer-coding-system (defun sieve-manage-decode (octets &optional buffer) "Convert managesieve protocol OCTETS to UTF-8 string. =20 -If optional BUFFER is non-nil, insert decoded string into BUFFER." +If BUFFER is non-nil detect \\=3D'eol-type\\=3D' of OCTETS, use correspond= ing +\\=3D'utf-8- coding system to decode octets, set +`buffer-file-coding-system` of BUFFER, insert the decoded UTF-8 string +into BUFFER and return BUFFER. + +Otherwise, decode OCTETS using `sieve-manage--coding-system' (\\=3D'utf-8-= unix) +and return the resulting UTF-8 string." (when octets (if buffer (with-current-buffer buffer @@ -339,18 +475,13 @@ sieve-sasl-auth ;; somehow. (lambda (_prompt) (copy-sequence user-password))) (step (sasl-next-step client nil)) - (_tag (sieve-manage-send - (concat - "AUTHENTICATE \"" - mech - "\"" + (_tag (sieve-manage-send-command + 'AUTHENTICATE + (cons 'quoted mech) (and (sasl-step-data step) - (concat - " \"" - (base64-encode-string - (sasl-step-data step) - 'no-line-break) - "\""))))) + (cons 'quoted (base64-encode-string + (sasl-step-data step) + 'no-line-break))))) data rsp) (catch 'done (while t @@ -375,24 +506,23 @@ sieve-sasl-auth (if (and (setq step (sasl-next-step client step)) (setq data (sasl-step-data step))) ;; We got data for server but it's finished - (sieve-manage--error + (sieve-manage--error 'sieve-manage-server-error "Server not ready for SASL data: %s" data) ;; The authentication process is finished. (sieve-manage--message "Logged in as %s using %s" user-name mech) (throw 'done t))) (unless (stringp rsp) - (sieve-manage--error + (sieve-manage--error 'sieve-manage-server-error "Server aborted SASL authentication: %s" (caddr rsp))) (sasl-step-set-data step (base64-decode-string rsp)) (setq step (sasl-next-step client step)) (sieve-manage-send - (if (sasl-step-data step) - (concat "\"" - (base64-encode-string (sasl-step-data step) - 'no-line-break) - "\"") - ""))))))) + (cons 'quoted + (if (sasl-step-data step) + (base64-encode-string (sasl-step-data step) + 'no-line-break) + "")))))))) =20 (defun sieve-manage-cram-md5-p (buffer) (sieve-manage-capability "SASL" "CRAM-MD5" buffer)) @@ -471,7 +601,7 @@ sieve-manage-open (setq sieve-manage-auth auth) (cl-return))) (unless sieve-manage-auth - (sieve-manage--error + (sieve-manage--error 'sieve-manage-client-error "Couldn't figure out authenticator for server"))) (sieve-manage-erase) (current-buffer)))) @@ -496,12 +626,21 @@ sieve-manage-opened (and sieve-manage-process (memq (process-status sieve-manage-process) '(open run)))))) =20 +(defun sieve-manage--check-script-name (cmd name) + "Check for script NAME for CMD. +Signals an error for unsupported names." + (when (and (not (eq 'SETACTIVE cmd)) + (string-empty-p name)) + (sieve-manage--error 'sieve-manage-client-error + "%s script name cannot be empty" cmd)) + t) + (defun sieve-manage-close (&optional buffer) "Close connection to managesieve server in BUFFER. If BUFFER is nil, the current buffer is used." (with-current-buffer (or buffer (current-buffer)) (when (sieve-manage-opened) - (sieve-manage-send "LOGOUT") + (sieve-manage-send-command 'LOGOUT) (sit-for 1)) (when (and sieve-manage-process (memq (process-status sieve-manage-process) '(open run))) @@ -525,41 +664,73 @@ sieve-manage-capability server-value))))) =20 (defun sieve-manage-listscripts (&optional buffer) + "Send LISTSCRIPTS command to download list of available scripts. +BUFFER is the \\=3D'sieve-manage\\=3D' process buffer (default: \\=3D'(cur= rent-buffer)\\=3D')." (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send "LISTSCRIPTS") + (sieve-manage-send-command 'LISTSCRIPTS) (sieve-manage-parse-listscripts))) =20 (defun sieve-manage-havespace (name size &optional buffer) - (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "HAVESPACE \"%s\" %s" name size)) - (sieve-manage-parse-oknobye))) + "Send HAVESPACE command for script NAME and SIZE. +BUFFER is the \\=3D'sieve-manage\\=3D' process buffer (default: \\=3D'(cur= rent-buffer)\\=3D')." + (let ((cmd 'HAVESPACE)) + (sieve-manage--check-script-name cmd name) + (with-current-buffer (or buffer (current-buffer)) + (sieve-manage-send-command cmd name size) + (sieve-manage-parse-oknobye)))) =20 (defun sieve-manage-putscript (name script-buffer &optional buffer) - (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "PUTSCRIPT \"%s\"" name) script-buffer) - (sieve-manage-parse-oknobye))) + "Send PUTSCRIPT command to upload script named NAME. +Uses the content of SCRIPT-BUFFER as the actual script to upload. +`buffer-file-coding-system' of SCRIPT-BUFFER is used to convert +the script to unibyte before the uploaded. It shall be set to either +\\=3D'utf-8-unix, \\=3D'utf-8-dos or \\=3D'utf-8-mac. +BUFFER is the \\=3D'sieve-manage\\=3D' process buffer (default: \\=3D'(cur= rent-buffer)\\=3D')." + (let ((cmd 'PUTSCRIPT)) + (sieve-manage--check-script-name cmd name) + (with-current-buffer (or buffer (current-buffer)) + (sieve-manage-send-command cmd name script-buffer) + (sieve-manage-parse-oknobye)))) =20 (defun sieve-manage-deletescript (name &optional buffer) - (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "DELETESCRIPT \"%s\"" name)) - (sieve-manage-parse-oknobye))) - -(defun sieve-manage-getscript (name output-buffer &optional buffer) - (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "GETSCRIPT \"%s\"" name)) - (sieve-manage-decode (sieve-manage-parse-string) - output-buffer) - (sieve-manage-parse-crlf) - (sieve-manage-parse-oknobye))) + "Send DELETESCRIPT command to delete script named NAME. +BUFFER is the \\=3D'sieve-manage\\=3D' process buffer (default: \\=3D'(cur= rent-buffer)\\=3D')." + (let ((cmd 'DELETESCRIPT)) + (sieve-manage--check-script-name cmd name) + (with-current-buffer (or buffer (current-buffer)) + (sieve-manage-send-command cmd name) + (sieve-manage-parse-oknobye)))) + +(defun sieve-manage-getscript (name script-buffer &optional buffer) + "Send GETSCRIPT command to download script named NAME. +Inserts the downloaded script into SCRIPT-BUFFER. The \\=3D'eol-type\\=3D= ' of +the downloaded script will be detected automatically and gets +used to decode the script data before inserting it into +SCRIPT-BUFFER. Also sets `buffer-file-coding-system' of +SCRIPT-BUFFER to the \\=3D'utf-8\\=3D' variant with the detected \\=3D'eol= -type\\=3D'. +BUFFER is the \\=3D'sieve-manage\\=3D' process buffer (default: \\=3D'(cur= rent-buffer)\\=3D')." + (let ((cmd 'GETSCRIPT)) + (sieve-manage--check-script-name cmd name) + (with-current-buffer (or buffer (current-buffer)) + (sieve-manage-send-command cmd name) + (sieve-manage-decode (sieve-manage-parse-string) + script-buffer) + (sieve-manage-parse-crlf) + (sieve-manage-parse-oknobye)))) =20 (defun sieve-manage-setactive (name &optional buffer) - (with-current-buffer (or buffer (current-buffer)) - (sieve-manage-send (format "SETACTIVE \"%s\"" name)) - (sieve-manage-parse-oknobye))) + "Send SETACTIVE command to activate script named NAME. +Use an empty NAME to deactivate/disable any active script. +BUFFER is the \\=3D'sieve-manage\\=3D' process buffer (default: \\=3D'(cur= rent-buffer)\\=3D')." + (let ((cmd 'SETACTIVE)) + (sieve-manage--check-script-name cmd name) + (with-current-buffer (or buffer (current-buffer)) + (sieve-manage-send-command cmd name) + (sieve-manage-parse-oknobye)))) =20 ;; Protocol parsing routines (defun sieve-manage-regexp-oknobye () - "Return regexp for managesieve 'response-oknobye'." + "Return regexp for managesieve \\=3D'response-oknobye\\'." (concat "^\\(OK\\|NO\\|BYE\\)\\( (\\([^)]+\\))\\)?\\( \\(.*\\)\\)?" sieve-manage-server-eol)) @@ -633,9 +804,10 @@ sieve-manage-parse-string (goto-char (point-min)) (unless (setq rsp (sieve-manage-is-string)) (when (sieve-manage-no-p (sieve-manage-is-oknobye)) - ;; simple `error' is enough since `sieve-manage-erase' + ;; no further details required since `sieve-manage-erase' ;; already adds the server response to the log - (error (sieve-manage-erase))))) + (sieve-manage--error 'sieve-manage-server-error + "%s" (sieve-manage-erase))))) (sieve-manage-erase (point)) rsp)) =20 @@ -666,28 +838,36 @@ sieve-manage-parse-listscripts data rsp))) =20 -(defun sieve-manage-send (command &optional payload-buffer) - "Send COMMAND with optional string from PAYLOAD-BUFFER. - -If non-nil, the content of PAYLOAD-BUFFER will be appended to -COMMAND using the \\=3D'literal-s2c\\=3D' representation according to rfc5= 804." - (let* ((encoded (when (and payload-buffer - (> (buffer-size payload-buffer) 0)) - (with-current-buffer payload-buffer - (encode-coding-region - (point-min) (point-max) - (buffer-local-value 'buffer-file-coding-system - payload-buffer) - t)))) - cmdstr literal-c2s) - (when encoded - (setq literal-c2s (format " {%d+}%s%s" - (length encoded) - sieve-manage-client-eol - encoded))) - (setq cmdstr (concat (sieve-manage--convert-multibyte command) - literal-c2s - sieve-manage-client-eol)) +(defun sieve-manage-send-command (cmd-name &rest items) + "Assemble rfc5804 command from CMD-NAME and ITEMS and send it. +See `sieve-manage-send' for further details." + (when (string-empty-p cmd-name) + (sieve-manage--error 'sieve-manage-encode-error + "Command name cannot be empty")) + (when (multibyte-string-p cmd-name) + (sieve-manage--error 'sieve-manage-encode-error + "Command name '%s' cannot must be a unibyte string" cmd-name)) + (apply #'sieve-manage-send (cons 'cmd-name cmd-name) items)) + +(defun sieve-manage-send (&rest items) + "Encode and concatenate ITEMS, send the result to the sieve server. +ITEMs will be: + +1. skipped/ignored if nil +2. converted to unibyte (optional, depends on item type) +3. encoded according to rfc5804 +4. concatenated (using SPaces as separators) + +The result will then be sent to the managesieve server. + +See `sieve-manage-encode' for details regarding supported ITEMS and their +handling." + (let ((cmdstr (concat + (string-join + (remove nil (mapcar #'sieve-manage-encode + items)) + " ") + sieve-manage-client-eol))) (sieve-manage--append-to-log cmdstr) (process-send-string sieve-manage-process cmdstr))) =20 diff --git a/test/lisp/net/sieve-manage-tests.el b/test/lisp/net/sieve-mana= ge-tests.el index 010c9071608..e58854ff6b0 100644 --- a/test/lisp/net/sieve-manage-tests.el +++ b/test/lisp/net/sieve-manage-tests.el @@ -26,79 +26,283 @@ (require 'ert) (require 'sieve-manage) =20 -(defvar sieve-script-multibyte-unix +;;; test data + +(defvar smt/script-multibyte-unix "if body :matches \"=C3=A4\" { stop; }\n" - "Simple multibyte sieve script.") + "Simple multibyte sieve script with unix EOL.") + +(defvar smt/script-multibyte-dos + "if body :matches \"=C3=A4\" { stop; }\r\n" + "Simple multibyte sieve script with dos EOL.") + +(defvar smt/script-multibyte-mac + "if body :matches \"=C3=A4\" { stop; }\r" + "Simple multibyte sieve script with mac EOL.") + +(defvar smt/string-empty "" + "Empty sieve string.") + +(defvar smt/string-unibyte "Hello world!" + "Unibyte sieve string.") + +(defvar smt/string-multibyte "=C3=A4=C3=B6=C3=BC=C3=9F=C3=84=C3=96=C3=9C" + "Multibyte sieve string.") + +(defvar smt/script-name-unibyte "abcdefg.sieve" + "Unibyte sieve script name.") =20 -(defun mk-literal-s2c (string) - "Encode STRING to managesieve 'literal-s2c'." - (let ((data (sieve-manage-encode string))) - (concat (format "{%d}\r\n" (length data)) - data))) +(defvar smt/script-name-multibyte "=C3=A4=C3=B6=C3=BC=C3=9F=C3=84=C3=96=C3= =9C.sieve" + "Multibyte sieve script name.") =20 -(defun mk-rsp-oknobye (type &optional resp-code string) +;;; helper functions + +(defun smt/mk-literal-s2c (string) + "Encode multibyte STRING to managesieve 'literal-s2c'." + (sieve-manage-encode (cons 'mb-literal-s2c string))) +;; (smt/mk-literal-s2c smt/string-multibyte) + +(defun smt/mk-rsp-oknobye (type &optional resp-code string) "Encode TYPE, RESP-CODE and STRING to managesieve 'response-oknobye'." (when (memq type '(OK NO BYE)) (concat - (mapconcat #'identity - (delq nil - (list (symbol-name type) - (when resp-code - (format "(%s)" resp-code)) - (when string - (format "\"%s\"" - (sieve-manage-encode string))))) - " ") + (string-join + (delete nil + (list (symbol-name type) + (when resp-code + (format "(%s)" resp-code)) + (when string + (format "\"%s\"" + (sieve-manage--encode-multibyte string))))) + " ") "\r\n"))) -;; (mk-rsp-oknobye 'OK nil "Getscript completed.") -;; (mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") +;; (smt/mk-rsp-oknobye 'OK nil "Getscript completed.") +;; (smt/mk-rsp-oknobye 'NO "TRYLATER" "Server is busy.") =20 -(defun mk-rsp-getscript-ok (script) - "Encode SCRIPT to managesieve 'response-getscript'." - (concat (mk-literal-s2c script) +(defun smt/mk-rsp-getscript-ok (script) + "Encode SCRIPT to rfc5804 \\=3D'response-getscript\\=3D'." + (concat (smt/mk-literal-s2c script) "\r\n" - (mk-rsp-oknobye 'OK "Getscript completed."))) -;; (mk-rsp-getscript-ok sieve-script-multibyte-unix) + (smt/mk-rsp-oknobye 'OK "Getscript completed."))) +;; (smt/mk-rsp-getscript-ok smt/script-multibyte-unix) =20 -(defun managesieve-getscript (script) - "Simulate managesieve getscript response to test -`sieve-manage-getscript' function." +(defun smt/managesieve-getscript (script &optional nocleanup) + "Simulate rfc5804 response for GETSCRIPT command. +The value of SCRIPT is used as the actual sieve script. +Use for testing `sieve-manage-getscript' function. +If NOCLEANUP is non-nil, keep all created buffers." (let* ((script-name "test.sieve") ;; `sieve-manage-server' and `sieve-manage-port' are used in ;; `sieve-manage-make-process-buffer' (sieve-manage-server) (sieve-manage-port "sieve") - (sieve-buffer (sieve-manage-make-process-buffer)) + ;; sieve-manage process buffer + (process-buffer (sieve-manage-make-process-buffer)) + ;; sieve-manage buffer to receive downloaded sieve script (output-buffer (generate-new-buffer script-name))) ;; use cl-letf to mock some functions in call chain of ;; sieve-manage-getscript (cl-letf (((symbol-function 'sieve-manage-send) - ;; simulate sieve server response with a single - ;; multibyte character `=C3=A4` - (lambda (_) - (with-current-buffer sieve-buffer + ;; simulate sieve server getscript response + ;; containing 'script' + (lambda (&rest _) + (with-current-buffer process-buffer (goto-char (point-min)) - (insert (mk-rsp-getscript-ok script))))) + (insert (smt/mk-rsp-getscript-ok script))))) ((symbol-function 'accept-process-output) (lambda (&optional _ _ _ _) nil)) ((symbol-function 'get-buffer-process) (lambda (_) nil))) - ;; extract sieve script from sieve-buffer and put it into + ;; extract sieve script from process-buffer and put it into ;; output-buffer - (sieve-manage-getscript script-name output-buffer sieve-buffer) + (sieve-manage-getscript script-name output-buffer process-buffer) ;; extract script from output-buffer and return it as a string (let ((script (with-current-buffer output-buffer (set-buffer-modified-p nil) - (buffer-string)))) + (buffer-string))) + (coding-system + (buffer-local-value 'buffer-file-coding-system + output-buffer))) ;; cleanup - (kill-buffer sieve-buffer) - (kill-buffer output-buffer) - (when (get-buffer sieve-manage-log) - (kill-buffer sieve-manage-log)) - ;; return script - script)))) - -(ert-deftest ert/managesieve-getscript-multibyte () - (should (string=3D sieve-script-multibyte-unix - (managesieve-getscript sieve-script-multibyte-unix)))) + (unless nocleanup + (kill-buffer process-buffer) + (kill-buffer output-buffer) + (when (get-buffer sieve-manage-log) + (kill-buffer sieve-manage-log))) + ;; return (script . coding-system) + (cons script coding-system))))) + +;;; tests + +(ert-deftest smt/sieve-manage--mk-quoted () + (should-error (sieve-manage--mk-quoted 42) + :type 'sieve-manage-encode-error) + (should-error (sieve-manage--mk-quoted nil) + :type 'sieve-manage-encode-error) + (should (string-equal (format "\"%s\"" smt/script-name-unibyte) + (sieve-manage--mk-quoted smt/script-name-unibyte))) + (should (string-equal "\"\"" + (sieve-manage--mk-quoted "")))) + +(ert-deftest smt/sieve-manage--mk-literal-c2s () + (should-error (sieve-manage--mk-literal-c2s 42) + :type 'sieve-manage-encode-error) + (should-error (sieve-manage--mk-literal-c2s nil) + :type 'sieve-manage-encode-error) + (should (string-equal (format "{%d+}\r\n%s" + (length smt/script-name-unibyte) + smt/script-name-unibyte) + (sieve-manage--mk-literal-c2s smt/script-name-unib= yte))) + (should (string-equal "{0+}\r\n" + (sieve-manage--mk-literal-c2s "")))) + +(ert-deftest smt/sieve-manage--mk-literal-s2c () + (should-error (sieve-manage--mk-literal-s2c 42) + :type 'sieve-manage-encode-error) + (should-error (sieve-manage--mk-literal-s2c nil) + :type 'sieve-manage-encode-error) + (should (string-equal (format "{%d}\r\n%s" + (length smt/script-name-unibyte) + smt/script-name-unibyte) + (sieve-manage--mk-literal-s2c smt/script-name-unib= yte))) + (should (string-equal "{0}\r\n" + (sieve-manage--mk-literal-s2c "")))) + +(ert-deftest smt/sieve-manage-encode () + ;;; unsupported data types + (should-error + ;; RFC5804 doesn't support negative numbers + (sieve-manage-encode -1) :type 'sieve-manage-encode-error) + + ;;; symbol (supported) commands + (should-error (sieve-manage-encode nil) + :type 'sieve-manage-unsupported-cmd) + (should-error (sieve-manage-encode t) + :type 'sieve-manage-unsupported-cmd) + (should-error (sieve-manage-encode 'CHECKSCRIPT) + :type 'sieve-manage-unsupported-cmd) + (should (sieve-manage-encode 'LISTSCRIPTS)) + + ;;; number [0..4294967296] + (should (string-equal (format "%d" 0) (sieve-manage-encode 0))) + (should (string-equal (format "%d" 255) (sieve-manage-encode 255))) + (should (string-equal (format "%d" 4294967296) + (sieve-manage-encode 4294967296))) + + ;;; (simple) string + (should (string-equal (format "\"%s\"" smt/script-name-unibyte) + (sieve-manage-encode smt/script-name-unibyte))) + (should (string-equal (format "\"%s\"" (encode-coding-string + smt/script-name-multibyte + 'utf-8-unix)) + (sieve-manage-encode smt/script-name-multibyte))) + + ;;; TODO: (number . ) + + ;;; TODO: (cmd-name . ) + + ;;; (quoted/literal-c2s/literal-s2c . ) + ;; 'quoted + (should (string-equal "\"\"" (sieve-manage-encode (cons 'quoted "")))) + (should (string-equal (format "\"%s\"" smt/script-name-unibyte) + (sieve-manage-encode + (cons 'quoted smt/script-name-unibyte)))) + + ;; 'literal-c2s + (should (string-equal "{0+}\r\n" + (sieve-manage-encode (cons 'literal-c2s "")))) + (should (string-equal (format "{%d+}\r\n%s" + (length smt/script-name-unibyte) + smt/script-name-unibyte) + (sieve-manage-encode + (cons 'literal-c2s smt/script-name-unibyte)))) + + ;; 'literal-s2c + (should (string-equal "{0}\r\n" + (sieve-manage-encode (cons 'literal-s2c "")))) + (should (string-equal (format "{%d}\r\n%s" + (length smt/script-name-unibyte) + smt/script-name-unibyte) + (sieve-manage-encode + (cons 'literal-s2c smt/script-name-unibyte)))) + + ;;; TODO: (mb-quoted/mb-literal-c2s/mb-literal-s2c . ) + ) + + +(ert-deftest smt/sieve-manage--detect-buffer-coding-system () + ;; TODO + ) + + +(ert-deftest smt/sieve-manage-decode () + ;; TODO + ) + + +(ert-deftest smt/sieve-sasl-auth () + ;; TODO + ) + + +(ert-deftest smt/sieve-manage-authenticate () + ;; TODO + ) + + +;; TODO: check which sieve-manage-parse-... functions can be tested + + +(ert-deftest smt/sieve-manage-is-string () + ;; TODO + ) + + +(ert-deftest smt/sieve-manage--check-script-name () + (should-error (sieve-manage--check-script-name 'PUTSCRIPT "") + :type 'sieve-manage-client-error) + (should (sieve-manage--check-script-name 'SETACTIVE ""))) + + +(ert-deftest smt/managesieve-authenticate () + ;; TODO + ) + + +(ert-deftest smt/managesieve-listscripts () + ;; TODO + ) + + +(ert-deftest smt/managesieve-havespace () + ;; TODO + ) + + +(ert-deftest smt/managesieve-putscript () + ;; TODO + ) + + +(ert-deftest smt/managesieve-deletescript () + ;; TODO + ) + + +(ert-deftest smt/managesieve-getscript () + (let ((ret (smt/managesieve-getscript smt/script-multibyte-unix))) + (should (string=3D smt/script-multibyte-unix (car ret))) + (should (eq 'utf-8-unix (cdr ret)))) + (let ((ret (smt/managesieve-getscript smt/script-multibyte-dos))) + (should (string=3D smt/script-multibyte-unix (car ret))) + (should (eq 'utf-8-dos (cdr ret)))) + (let ((ret (smt/managesieve-getscript smt/script-multibyte-mac))) + (should (string=3D smt/script-multibyte-unix (car ret))) + (should (eq 'utf-8-mac (cdr ret)))) + ) + + +(ert-deftest smt/managesieve-setactive () + ;; TODO + ) =20 ;;; sieve-manage-tests.el ends here --=20 2.39.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 23 11:49:54 2023 Received: (at 54154) by debbugs.gnu.org; 23 Jan 2023 16:49:54 +0000 Received: from localhost ([127.0.0.1]:55183 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pK013-0006RU-UR for submit@debbugs.gnu.org; Mon, 23 Jan 2023 11:49:54 -0500 Received: from eggs.gnu.org ([209.51.188.92]:52704) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pK011-0006RG-Uy for 54154@debbugs.gnu.org; Mon, 23 Jan 2023 11:49:52 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pK00v-0007tI-Op; Mon, 23 Jan 2023 11:49:45 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=edqr0Ov9Ek2bvOOccFDfizN05QvVsfhh40VlVUmRbFk=; b=bSB4zdp60ejF DmIl4XMZBMvK3ArtduiHOR2agTiYqCvK9QbwxXcjy5/4o71+LaoV1xpQG05SDCu5oEaD3z7yJWYzS /DA6lV8fED1KtZ9pC4tnhWfdikB4c6J/iwoJeJi5B2mz09MedxXOO2T7l/q73rRXnRwYmA4B0med/ 8Nnw8+ZItCfPb8Uc0TReTdEF155E6HXTMUgQYXHpBw8VC4hhuItu+J5z+Vd7gLAj2GwMS4OBvIEjU PnXAdMZmbFfZku8wQj36HXr9u0CBaDTXabSZeKU8l9qxF3WdRyqwYhH7XIeGSxfy7atR4RiVQU9AN vkgc3ZwKq1stzw6mUmpxpw==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pK00v-0004ue-8W; Mon, 23 Jan 2023 11:49:45 -0500 Date: Mon, 23 Jan 2023 18:49:54 +0200 Message-Id: <83fsc1i4b1.fsf@gnu.org> From: Eli Zaretskii To: Kai Tetzlaff In-Reply-To: <871qnlrzka.fsf@tetzco.de> (message from Kai Tetzlaff on Mon, 23 Jan 2023 17:22:13 +0100) Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) References: <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> <83sfg1id2t.fsf@gnu.org> <871qnlrzka.fsf@tetzco.de> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: Kai Tetzlaff > Cc: herbert@gojira.at, larsi@gnus.org, 54154@debbugs.gnu.org > Date: Mon, 23 Jan 2023 17:22:13 +0100 > > Eli, how do you want to handle this? Apply the first patches now? Wait > until I'm done with all all of them? Or, ...? Feedback appreciated! I have yet to review the final version of this. So if you are still working on the changes, I prefer to wait until you have the final version, and review then. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 23 12:07:23 2023 Received: (at 54154) by debbugs.gnu.org; 23 Jan 2023 17:07:24 +0000 Received: from localhost ([127.0.0.1]:55229 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pK0Hz-0006uC-Is for submit@debbugs.gnu.org; Mon, 23 Jan 2023 12:07:23 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:53459) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pK0Hx-0006ty-O8 for 54154@debbugs.gnu.org; Mon, 23 Jan 2023 12:07:22 -0500 Received: from mail.tetzco.de (unknown [IPv6:2a02:810d:1380:7900::9]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id 8E1CBBD078; Mon, 23 Jan 2023 18:07:13 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id 4E4CC6C00B2; Mon, 23 Jan 2023 18:27:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674494831; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=f+H4Dp2BceNf89/neONUuZ1PTTHO3HV2c4AkofWXnkE=; b=GdxtEgYSEFjWWXqJdaRlccCAUCLjfeTZt19M/Ztj7lPXeOw+IcTUthzG3l1UzrhVhdNN07 p72y/+VmOA4jigekBfdmEUi3buRZtjGEMdRjo6Xe1yEl2nU3p4WC0eP4tA41st44Si0qT4 aKilNxtLxk7Z2792TUcvA+BsEJx1qJd1d8TjUQexVALHUXN4IpojYoe6R28iHN8o53ppG+ pfTWdstGZjGes3QgeWBQ784/hckwiGvJd86iCqp8xghxS16oTl9URRbBzcnZtYz0OARBtH wCRAEm2yNX2ORh9bkvrgmLYbrIML4letyuFVgq0F8JOHKTGynI5D4rOK7WTFNUNEQmv+e+ aGJzKtUwe1BWqq7NZNqXCwXk/yuR/KolDI0fp62A+PIaxrY4XFAjIXr4xVlS+vGjdMWRci 5EUK3n/UEBZihbhhCf7ExZsRaavT9rx2ZYuEL2nt1EGyYcgVu9vFTApBf8IfOEbY2r3/iO UVtnNLF4IpIx2jfQbmCztWund+/hv45XBjd5YZ9uHGY8tq76exQ2yvAuwBdqBExta6kbCw TcLq7fWt3fkBcDprVt0faVz99AGWRksfHbS3uC4SI599QzT+nciQfBG4Fn/Da5oR/jQ9AQ pTvI4KjzSjiajjSOlPjQAQTZumTpmybz5hmJSZOJ7qQ+UEIOZivm4= From: Kai Tetzlaff To: Andreas Schwab Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: (Andreas Schwab's message of "Mon, 23 Jan 2023 15:27:49 +0100") References: <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> <878rhts69s.fsf@tetzco.de> Date: Mon, 23 Jan 2023 18:07:12 +0100 Message-ID: <87pmb5qiwv.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: 4E4CC6C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.83 / 30.00]; BAYES_HAM(-3.00)[99.99%]; NEURAL_HAM(-3.00)[-1.000]; GENERIC_REPUTATION(-0.73)[-0.72664503195413]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TAGGED_FROM(0.00)[bug]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: larsi@gnus.org, "Herbert J. Skuhra" , 54154@debbugs.gnu.org, Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Andreas Schwab writes: > On Jan 23 2023, Kai Tetzlaff wrote: > >> And I learned something about git again. `git log --grep ...` apparently >> silently ignores my origin/master ref and greps through all available >> refs. > > It searches through all commit messages that you would see without the > --grep option. And right again. This really stumped me. So after some investigation on my part, here's where my confusion originated. To check git logs, I typically use an alias (l2) which adds the `--graph` option to git log. And `git log --pretty=oneline --graph` then actually results in: ... | * 0d3b6518e39a28774e4e70ed9bb7ef4aa009c0cf (ruby-ts--indent-rules): Indent inside empty parens properly | * 7fb69ce233b8a655af63d4c47b7359c43660acf6 (emacs-29) ; * doc/emacs/modes.texi (Choosing Modes): Add index entries. * | ede5e82418a0b8cfce2bf96b2a3255ca86b65000 ; Merge from origin/emacs-29 |\| | * 12d7670b90f66f1d45a8c69d9acfc25238a65b02 Fix bug in 'sieve-manage--append-to-log' * | e9ceeee1198aa10cac3cd61ff9537b64640455c2 Merge from origin/emacs-29 |\| | * 21be03cccb611ea9e6c73fb04e578c48edf49a25 CC Mode: Prevent two classes of "type" prematurely entering c-found-types * | 117f90865adca03eab84778db0370ddc05ba8ae7 Add new command `kill-matching-buffers-no-ask' (bug#60714) ... Which seems to indicate that commit 12d7670b90f66f1d45a8c69d9acfc25238a65b02 Fix bug in 'sieve-manage--append-to-log' has been merged to master. However, here's the actual merge commit: $ git log --patch -1 ede5e82418a0b8cfce2bf96b2a3255ca86b65000 commit ede5e82418a0b8cfce2bf96b2a3255ca86b65000 Merge: e9ceeee1198 12d7670b90f Author: Stefan Kangas Date: 2023-01-20 11:30:22 +0100 ; Merge from origin/emacs-29 The following commit was skipped: 12d7670b90f Fix bug in 'sieve-manage--append-to-log' That's it. I.e. it is completely empty because the only commit which was actually (supposed to be) merged was then just skipped. I guess that this is needed for git to recognize that the commit was purposely skipped (so that git will not include it in subsequent merges). So, as ever so often, the problem was sitting in front of the monitor. I must admit that I still find this behaviour surprising/unexpected. But good to know! Beware of `git log --graph` ... BR, Kai From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 23 12:12:33 2023 Received: (at 54154) by debbugs.gnu.org; 23 Jan 2023 17:12:34 +0000 Received: from localhost ([127.0.0.1]:55240 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pK0Mz-000720-LO for submit@debbugs.gnu.org; Mon, 23 Jan 2023 12:12:33 -0500 Received: from mx2.tetzco.de ([152.67.86.91]:48371) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pK0Mw-00071j-M0 for 54154@debbugs.gnu.org; Mon, 23 Jan 2023 12:12:31 -0500 Received: from mail.tetzco.de (unknown [IPv6:2a02:810d:1380:7900::9]) (Authenticated sender: relay@tetzco.de) by mx2.tetzco.de (Postfix) with ESMTPSA id EA1EABD078; Mon, 23 Jan 2023 18:12:24 +0100 (CET) Received: from moka (moka.tetzco.de [172.30.42.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kai@tetzco.de) by mail.tetzco.de (Postfix) with ESMTPSA id 9E4976C00B2; Mon, 23 Jan 2023 18:32:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tetzco.de; s=20210624; t=1674495142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=l3tYjwZzhPh9OsefACFOponnldPEsaC4hUXcOVYbaPU=; b=nlsX2CaGvHzC7ZmpTOdqyNZ5f7vbio/+PzHytZx8ZVe2zwe7H+lQGFLv0CuZUQpxbYow50 AwnoFnaGvGmG2+82P9aeIwuiQPnoFks9olzdKivaXPworw6fzphtneOy0iURNua4BJAMJ6 poDLNTLbtwmmyWL9YtvMl/nkM3rVssK2Fy4fpv5hFdCG6Hp3lrNz4qFdUC2CmduePGSNqT 7/XoibZg78DbRkOWwNwSPnNUo+IH+9kYg4zUy67yAF3CJGRZMitvJhRT8tTIi7e8te3PBL kgKx0xvvnLRYfZgz35rstNP94Tevml1BnZbZ03NSIQzmpRFxF7x9dph5WijMr04s3jLlUl 84+IKVALi6J9VOWmTKhm1pssk9U8Y2TEGhNXi27hFtAd1ylWHzmja6BhpwXyf7s7pSHkkD EScT72avJc3//jrRN2zpKzdJ2t49L4xnd01hVJJllAQdQFvE/1ZaUc6J7bd/SGwqiMPSy4 ywUcefJW2YfznFtUpAFisgNzYgkwEcFO5xT16ooSbK/O+NLyEkSSy3Vg12t3X9m2UrYAcE iWE66I0H/L7Piuu+0pUJ47JcVNEzFaRqsuD1lyPuZs3+FMB7qLpTpisVAdC3Y40M4e4ES5 ChBMQlXIHNWT/VzZzU2X5CYjbSjtgZjtUlffbpRzAHREaAlikgqko= From: Kai Tetzlaff To: Eli Zaretskii Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: <83fsc1i4b1.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 23 Jan 2023 18:49:54 +0200") References: <835yd31wk8.fsf@gnu.org> <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> <83sfg1id2t.fsf@gnu.org> <871qnlrzka.fsf@tetzco.de> <83fsc1i4b1.fsf@gnu.org> Date: Mon, 23 Jan 2023 18:12:23 +0100 Message-ID: <87leltqio8.fsf@tetzco.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Server: rakaposhi X-Rspamd-Queue-Id: 9E4976C00B2 X-Rspamd-Action: no action X-Spamd-Result: default: False [-6.83 / 30.00]; NEURAL_HAM(-3.00)[-1.000]; BAYES_HAM(-3.00)[99.99%]; GENERIC_REPUTATION(-0.73)[-0.72664511944531]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ZERO(0.00)[0]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; DKIM_SIGNED(0.00)[tetzco.de:s=20210624]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_FROM(0.00)[bug]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: herbert@gojira.at, 54154@debbugs.gnu.org, larsi@gnus.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Eli Zaretskii writes: >> From: Kai Tetzlaff >> Cc: herbert@gojira.at, larsi@gnus.org, 54154@debbugs.gnu.org >> Date: Mon, 23 Jan 2023 17:22:13 +0100 >> >> Eli, how do you want to handle this? Apply the first patches now? Wait >> until I'm done with all all of them? Or, ...? Feedback appreciated! > > I have yet to review the final version of this. So if you are still > working on the changes, I prefer to wait until you have the final > version, and review then. Ok, fine with me. Thank, Kai From debbugs-submit-bounces@debbugs.gnu.org Mon Jan 23 12:53:51 2023 Received: (at 54154) by debbugs.gnu.org; 23 Jan 2023 17:53:51 +0000 Received: from localhost ([127.0.0.1]:55265 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pK10w-000868-TI for submit@debbugs.gnu.org; Mon, 23 Jan 2023 12:53:51 -0500 Received: from smtp-out1.suse.de ([195.135.220.28]:35984) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pK10v-00085s-4Y for 54154@debbugs.gnu.org; Mon, 23 Jan 2023 12:53:49 -0500 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 751ED21874; Mon, 23 Jan 2023 17:53:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1674496423; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=T/dlvELstJ6PYNhkHdtOL08KcmJWAvzoRUQFl42Pylw=; b=zj0sVnfx9f0GzjnEuK8xx23k72QItUFn2RNB0y2qlOOmOoflsMrc3zQMj/1JSbvuQRE71e 5a0MjyjKGojm8O7Ht79PQnP2qgVBB4a54KaZdaKZ4Yhs1hLAjVsEYZnmx1gwZE4XHb04mr 95ieWXL8Cp9cYShQxRGdaBybz09Ejhs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1674496423; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=T/dlvELstJ6PYNhkHdtOL08KcmJWAvzoRUQFl42Pylw=; b=zQB5l8LZPHjkTux+xqZMf21fL3wvo0VMIs8UpB4iZALQuP1q5dm9oVZ9LZBv/bcroWRDDM ngBnV6X78zuL+0AQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 5E8351357F; Mon, 23 Jan 2023 17:53:43 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id /V2dFqfJzmP5KAAAMHmgww (envelope-from ); Mon, 23 Jan 2023 17:53:43 +0000 From: Andreas Schwab To: Kai Tetzlaff Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) References: <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> <878rhts69s.fsf@tetzco.de> <87pmb5qiwv.fsf@tetzco.de> X-Yow: Concentrate on th'cute, li'l CARTOON GUYS! Remember the SERIAL NUMBERS!! Follow the WHIPPLE AVE EXIT!! Have a FREE PEPSI!! Turn LEFT at th'HOLIDAY INN!! JOIN the CREDIT WORLD!! MAKE me an OFFER!!! Date: Mon, 23 Jan 2023 18:53:42 +0100 In-Reply-To: <87pmb5qiwv.fsf@tetzco.de> (Kai Tetzlaff's message of "Mon, 23 Jan 2023 18:07:12 +0100") Message-ID: <87bkmpf87t.fsf@igel.home> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 54154 Cc: larsi@gnus.org, 54154@debbugs.gnu.org, "Herbert J. Skuhra" , Eli Zaretskii X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) On Jan 23 2023, Kai Tetzlaff wrote: > That's it. I.e. it is completely empty because the only commit which was > actually (supposed to be) merged was then just skipped. In other words, it is an evil merge. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." From unknown Thu Aug 21 12:12:19 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 21 Feb 2023 12:24:06 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 29 05:11:07 2024 Received: (at control) by debbugs.gnu.org; 29 Sep 2024 09:11:07 +0000 Received: from localhost ([127.0.0.1]:40029 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1supxL-0004Pp-1B for submit@debbugs.gnu.org; Sun, 29 Sep 2024 05:11:07 -0400 Received: from mail.bsd4all.net ([94.130.200.20]:18247) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1supxI-0004PJ-PS for control@debbugs.gnu.org; Sun, 29 Sep 2024 05:11:05 -0400 Date: Sun, 29 Sep 2024 11:09:57 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1727600997; bh=ZfO4lLJuFcCQabs74O0CrrtFcn9oc2tDZeiyAYWZ94k=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=zyQQBPlofJg0J+Dvq+Z8drgT0z1xSJNbSph+HOiSJvPbt01MKRMhFCOoIjWi9zfYm 05n146frVQLbtA9lFJd4XET+M9/EZM5sPuQ7fBI2pMc6SWIEf8kD492eEvgAJXaOYT Vp5B8IarfnFQRP0ikeJbvcrEkUbAbltZP79oWjeiHxqy4Wu7oQzvstR4AVyHvtyUbI lKdNgvnEe5A7meNgYANmCmldobgNLVz9c4L8ThxDhm6VIlsYkbTTtD3dARG5/h3Tlt 8bNRq8G3/AuHGTXJ0dYVLmtQHypYikeujviDgkui3sSXvX+UEoKAq3wDtfVPCRqWkW HHqAdd7xwfzYg== From: "Herbert J. Skuhra" To: control@debbugs.gnu.org Subject: unarchive 54154 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) unarchive 54154 From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 29 05:13:16 2024 Received: (at 54154) by debbugs.gnu.org; 29 Sep 2024 09:13:16 +0000 Received: from localhost ([127.0.0.1]:40035 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1supzQ-0004TX-ER for submit@debbugs.gnu.org; Sun, 29 Sep 2024 05:13:16 -0400 Received: from mail.bsd4all.net ([94.130.200.20]:46117) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1supzO-0004TH-Cc for 54154@debbugs.gnu.org; Sun, 29 Sep 2024 05:13:15 -0400 Date: Sun, 29 Sep 2024 11:11:52 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1727601157; bh=h1xzPgHpUBzUGIsnLbiR/bf7RjT4NlTaiz3zNdTwfaE=; h=Date:Message-ID:From:To:Subject:MIME-Version:Content-Type; b=u5PP4bMp4CxsQKqHK2wnP/gIkTgdmtt5Ly6aYR6AJP3Vxs7YoGUaMHL77aYe32zzT zLydYb9qaGtiK+NL7PUrTFfHE2wHYhgH4rTR1aDnPj544T8InxRJXx+3yIE7iLiFzC miWEl3ccre69o0sN7zt7ojF342eaimoz7gdfWiTPp0h/Thwo07hIAz4D0wUnCAqILU UL2OO3E6zpGgwDf8H1dGFCP8vLMwhBH2rbMIt8lmR97jmnk4U4+4wtjoaNVPvVSfjy n6JkztUZ6/syJHUVDMZmAtS9arhtBQ9vmiDL9KzBajzMUTXFKTE32YdBTWGorsSpUr uhOEkQA0SbkoQ== Message-ID: <87ldza963r.wl-herbert@gojira.at> From: "Herbert J. Skuhra" To: 54154@debbugs.gnu.org Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: <87bkmpf87t.fsf@igel.home> References: <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> <878rhts69s.fsf@tetzco.de> <87pmb5qiwv.fsf@tetzco.de> <87bkmpf87t.fsf@igel.home> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/31.0 Mule/6.0 MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 54154 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, Unfortunately I still see the following issue in master and emacs-30: - Run emacs -Q - M-x sieve-manage - Input IMAP server (e.g. imap.mailbox.org) Debugger entered--Lisp error: (wrong-type-argument stringp t) sieve-manage--append-to-log("sieve-manage: Connecting to imap.mailbox.org..." "\n") sieve-manage--message("Connecting to %s..." "imap.mailbox.org") sieve-manage-open("imap.mailbox.org" nil) sieve-open-server("imap.mailbox.org" nil) sieve-manage("imap.mailbox.org") funcall-interactively(sieve-manage "imap.mailbox.org") command-execute(sieve-manage record) execute-extended-command(nil "sieve-manage" "sieve-ma") funcall-interactively(execute-extended-command nil "sieve-manage" "sieve-ma") command-execute(execute-extended-command) On 2nd try it works! -- Herber From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 29 05:30:25 2024 Received: (at 54154) by debbugs.gnu.org; 29 Sep 2024 09:30:25 +0000 Received: from localhost ([127.0.0.1]:40068 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1suqG0-0000j8-LZ for submit@debbugs.gnu.org; Sun, 29 Sep 2024 05:30:25 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50288) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1suqFy-0000ip-HH for 54154@debbugs.gnu.org; Sun, 29 Sep 2024 05:30:23 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1suqFL-0003mF-Uj; Sun, 29 Sep 2024 05:29:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=MRtuxO/P1bF3viFanzBD8BA9K5lUYfIMHyL1qXAmqO8=; b=n5w3/PQtsp8w KHPoPOaARfYzzRi0EPt7sM5AefD3Ynblwt2eTkvI4dw7iIWQrSFVR7hfB7BNHGDk7T68U4HRvVHJC rxjxjEaUyUjsnEjTG084XUGVMq1QABqsw0v88sQhyY8j9AGi0JPDo3Qsc4xBwNVxHUGFM2DMKMWVq TzdkISpNuVgvR6eUP6EI8EiBo+3xyI+W3XNBvImxYuTSttNb8uclx0mrMv53RQmdJARq5mGYBSoen WuN7nhBN0FA7wqEf/x9JYAWcpSTDoznPXxkGxQE1ixijl0BjBgG8prXPT2pBg2N6X8ZgDmlPwYRDd xAUqHFJha1yx46xaQMZGzA==; Date: Sun, 29 Sep 2024 12:29:39 +0300 Message-Id: <86seti4xks.fsf@gnu.org> From: Eli Zaretskii To: "Herbert J. Skuhra" In-Reply-To: <87o74696he.wl-herbert@gojira.at> Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) References: <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> <878rhts69s.fsf@tetzco.de> <87pmb5qiwv.fsf@tetzco.de> <87bkmpf87t.fsf@igel.home> <87o74696he.wl-herbert@gojira.at> X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: larsi@gnus.org, 54154@debbugs.gnu.org, emacs+bug@tetzco.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Date: Sun, 29 Sep 2024 11:03:41 +0200 > From: "Herbert J. Skuhra" > Cc: Kai Tetzlaff , > larsi@gnus.org, > 54154@debbugs.gnu.org, > Eli Zaretskii > > Hi, > > Unfortunately I still see the following issue in master and emacs-30: > > - Run emacs -Q > - M-x sieve-manage > - Input IMAP server (e.g. imap.mailbox.org) > > Debugger entered--Lisp error: (wrong-type-argument stringp t) > sieve-manage--append-to-log("sieve-manage: Connecting to imap.mailbox.org..." "\n") > sieve-manage--message("Connecting to %s..." "imap.mailbox.org") > sieve-manage-open("imap.mailbox.org" nil) > sieve-open-server("imap.mailbox.org" nil) > sieve-manage("imap.mailbox.org") > funcall-interactively(sieve-manage "imap.mailbox.org") > command-execute(sieve-manage record) > execute-extended-command(nil "sieve-manage" "sieve-ma") > funcall-interactively(execute-extended-command nil "sieve-manage" "sieve-ma") > command-execute(execute-extended-command) > > On 2nd try it works! The fix was not merged to the-then master branch, so it is not in Emacs 30. Therefore, it is not a surprise you still see the problem: it was only fixed in Emacs 29. I lost the context long ago, so I hope Kai will suggest how to handle this. From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 29 06:23:53 2024 Received: (at 54154) by debbugs.gnu.org; 29 Sep 2024 10:23:53 +0000 Received: from localhost ([127.0.0.1]:40139 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sur5i-0007KN-DB for submit@debbugs.gnu.org; Sun, 29 Sep 2024 06:23:53 -0400 Received: from eggs.gnu.org ([209.51.188.92]:59336) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sur5c-0007IY-RO for 54154@debbugs.gnu.org; Sun, 29 Sep 2024 06:23:48 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sur4z-0000fP-TC; Sun, 29 Sep 2024 06:23:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=96JT67m/bMmyzvN8ozfgyMeTDky1ap+KULVTBEF+ydY=; b=j9h1X2QY7dNl ukiwuoI/wOabZSXVHfx4gtVe1gP3DLjquoU0o16UxotWd2aiCIBEgRQYRHlfGvcNE5t2IZdNuo+ga lGkH+B2dKKdnYc01ok+uvu9o6mkW4D7FtIsqy9Z+0Zl6T91uNQTrex8luwJwpepen9ouIvUu3gJx+ KG4y5yxXSpFryFEwFld690ioEMCoWQ3qqvFzHdvKkRbcazC5a9Ki1UmCi4a8VsocxWHXJ+ugdkPmF yasleBTVvIIdgoPWS43RClg7gPTxpflAkbSk+mIqAc12sgR8uUaRj8RZAOZSnXWZZ2gjbEZM5kbH3 EXd6qO76y7yGmIbZDndrug==; Date: Sun, 29 Sep 2024 13:23:03 +0300 Message-Id: <86plom4v3s.fsf@gnu.org> From: Eli Zaretskii To: herbert@gojira.at In-Reply-To: <86seti4xks.fsf@gnu.org> (message from Eli Zaretskii on Sun, 29 Sep 2024 12:29:39 +0300) Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) References: <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> <878rhts69s.fsf@tetzco.de> <87pmb5qiwv.fsf@tetzco.de> <87bkmpf87t.fsf@igel.home> <87o74696he.wl-herbert@gojira.at> <86seti4xks.fsf@gnu.org> X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: larsi@gnus.org, 54154@debbugs.gnu.org, emacs+bug@tetzco.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Cc: larsi@gnus.org, 54154@debbugs.gnu.org, emacs+bug@tetzco.de > Date: Sun, 29 Sep 2024 12:29:39 +0300 > From: Eli Zaretskii > > > - Run emacs -Q > > - M-x sieve-manage > > - Input IMAP server (e.g. imap.mailbox.org) > > > > Debugger entered--Lisp error: (wrong-type-argument stringp t) > > sieve-manage--append-to-log("sieve-manage: Connecting to imap.mailbox.org..." "\n") > > sieve-manage--message("Connecting to %s..." "imap.mailbox.org") > > sieve-manage-open("imap.mailbox.org" nil) > > sieve-open-server("imap.mailbox.org" nil) > > sieve-manage("imap.mailbox.org") > > funcall-interactively(sieve-manage "imap.mailbox.org") > > command-execute(sieve-manage record) > > execute-extended-command(nil "sieve-manage" "sieve-ma") > > funcall-interactively(execute-extended-command nil "sieve-manage" "sieve-ma") > > command-execute(execute-extended-command) > > > > On 2nd try it works! > > The fix was not merged to the-then master branch, so it is not in > Emacs 30. Therefore, it is not a surprise you still see the problem: > it was only fixed in Emacs 29. > > I lost the context long ago, so I hope Kai will suggest how to handle > this. However, it's quite possible that the following band-aid should fix this particular issue: diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el index 0faeb02..da2167c 100644 --- a/lisp/net/sieve-manage.el +++ b/lisp/net/sieve-manage.el @@ -178,7 +178,8 @@ sieve-manage--append-to-log (with-current-buffer (get-buffer-create sieve-manage-log) (set-buffer-multibyte nil) - (buffer-disable-undo))) + (buffer-disable-undo) + (current-buffer))) (goto-char (point-max)) (apply #'insert args)))) Can you test this? From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 29 08:17:25 2024 Received: (at 54154) by debbugs.gnu.org; 29 Sep 2024 12:17:25 +0000 Received: from localhost ([127.0.0.1]:40212 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1susrd-000364-94 for submit@debbugs.gnu.org; Sun, 29 Sep 2024 08:17:25 -0400 Received: from mail.bsd4all.net ([94.130.200.20]:20737) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1susrb-00035p-Gg for 54154@debbugs.gnu.org; Sun, 29 Sep 2024 08:17:24 -0400 Date: Sun, 29 Sep 2024 14:15:47 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1727612206; bh=nUjBv4MBuwZ3TqeWh3t0EYFZ9B2ZioSG8uZG5VWs3KE=; h=Date:Message-ID:From:To:Cc:Subject:MIME-Version:Content-Type; b=nuuop65s5QpeaonImVAbFLy/Q9jxiYheP2rXmw7xV2BkUYBy6qasUe7iT2Cibey7M 5pJv/Ps+DVcqnE6fGFlOwjbZ8q1c4PV5gEnpQV5T8wOljT9eI0AOZ+4hx+Iie+r1EY UQlUx4YiXnWjMMK/hmQ4iW6ppeFEXxL5V/QsWh3E+1IfSfzqYdc5fesXjDewa5QYNH 5vpsdjlfUtMDD7cXBnehwDtmrr66j+z5QMTY5SH/b0VH6s/zvoMxjX5BSTFPF8A016 29PDQ+rTSUKziQmv8c8hSmEpQtf4HMpckGV6fPQj8fjBG+MkA3pzWJn5IJGSKbk5rT 5jLVJC9ZQYxeA== Message-ID: <87jzeu8xl8.wl-herbert@gojira.at> From: "Herbert J. Skuhra" To: Eli Zaretskii Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) In-Reply-To: <86plom4v3s.fsf@gnu.org> References: <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> <878rhts69s.fsf@tetzco.de> <87pmb5qiwv.fsf@tetzco.de> <87bkmpf87t.fsf@igel.home> <87o74696he.wl-herbert@gojira.at> <86seti4xks.fsf@gnu.org> <86plom4v3s.fsf@gnu.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/31.0 Mule/6.0 MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: larsi@gnus.org, 54154@debbugs.gnu.org, emacs+bug@tetzco.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) On Sun, 29 Sep 2024 12:23:03 +0200, Eli Zaretskii wrote: > > > Cc: larsi@gnus.org, 54154@debbugs.gnu.org, emacs+bug@tetzco.de > > Date: Sun, 29 Sep 2024 12:29:39 +0300 > > From: Eli Zaretskii > > > > > - Run emacs -Q > > > - M-x sieve-manage > > > - Input IMAP server (e.g. imap.mailbox.org) > > > > > > Debugger entered--Lisp error: (wrong-type-argument stringp t) > > > sieve-manage--append-to-log("sieve-manage: Connecting to imap.mailbox.org..." "\n") > > > sieve-manage--message("Connecting to %s..." "imap.mailbox.org") > > > sieve-manage-open("imap.mailbox.org" nil) > > > sieve-open-server("imap.mailbox.org" nil) > > > sieve-manage("imap.mailbox.org") > > > funcall-interactively(sieve-manage "imap.mailbox.org") > > > command-execute(sieve-manage record) > > > execute-extended-command(nil "sieve-manage" "sieve-ma") > > > funcall-interactively(execute-extended-command nil "sieve-manage" "sieve-ma") > > > command-execute(execute-extended-command) > > > > > > On 2nd try it works! > > > > The fix was not merged to the-then master branch, so it is not in > > Emacs 30. Therefore, it is not a surprise you still see the problem: > > it was only fixed in Emacs 29. > > > > I lost the context long ago, so I hope Kai will suggest how to handle > > this. > > However, it's quite possible that the following band-aid should fix > this particular issue: > > diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el > index 0faeb02..da2167c 100644 > --- a/lisp/net/sieve-manage.el > +++ b/lisp/net/sieve-manage.el > @@ -178,7 +178,8 @@ sieve-manage--append-to-log > (with-current-buffer > (get-buffer-create sieve-manage-log) > (set-buffer-multibyte nil) > - (buffer-disable-undo))) > + (buffer-disable-undo) > + (current-buffer))) > (goto-char (point-max)) > (apply #'insert args)))) Yes, this works. Thanks. As manage-sieve logs credentials maybe logging should be disabled by default and/or credentials shouldn't be logged when logging is enabled? From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 29 08:44:25 2024 Received: (at 54154) by debbugs.gnu.org; 29 Sep 2024 12:44:26 +0000 Received: from localhost ([127.0.0.1]:40233 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sutHj-0006fM-0I for submit@debbugs.gnu.org; Sun, 29 Sep 2024 08:44:25 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45770) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sutHf-0006dd-DV for 54154@debbugs.gnu.org; Sun, 29 Sep 2024 08:44:20 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sutH4-0001FO-25; Sun, 29 Sep 2024 08:43:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=EZ2/iohp7N6ByBB5vlx0dxmTUuIQG6B7VI74I9kDUeQ=; b=YEshIk+w7KGC TvDOa8EfeVQLdlP9OyUdiaRZttv2HJrxsJCn1aPeOCJjLkhqv8uG/BHEqXwxubE6IAwM+LPHYtOgc CrNmlYr+ASBFx1SDJ70cfHy9w24wPy9+VpNC9iayaVxNf7VXvsdduvuMHgn0wC3GoAOYc6Tg8+YH/ rfb5QGQ6lQT6bKvjKcWjzrakY3RGkigq6C3buwlWhlEnq12HiZln+P84TP60WOYnu/8nXRQiPyiOb lxMrQfSM87Y5Oj0WgIq+d8kcn5HGEAy0LjA3n3KztFsODT3g9jllQJC7l5cwhONwihCWan7ZAR2Ry cEep8q0KiLxvVjsNcQl/Zw==; Date: Sun, 29 Sep 2024 15:43:37 +0300 Message-Id: <86ldza4oli.fsf@gnu.org> From: Eli Zaretskii To: "Herbert J. Skuhra" In-Reply-To: <87jzeu8xl8.wl-herbert@gojira.at> Subject: Re: bug#54154: Emacs commit ae963e80a79f5a9184daabfc8197f211a39b136d (sieve-manage) References: <874jsngod2.fsf@tetzco.de> <83tu0nynla.fsf@gnu.org> <873586d7ii.fsf@tetzco.de> <834jsmpqf9.fsf@gnu.org> <87bkmua51z.fsf@tetzco.de> <83v8l2o212.fsf@gnu.org> <878rhut6aq.fsf@tetzco.de> <87ilgxs8vl.fsf@tetzco.de> <878rhts69s.fsf@tetzco.de> <87pmb5qiwv.fsf@tetzco.de> <87bkmpf87t.fsf@igel.home> <87o74696he.wl-herbert@gojira.at> <86seti4xks.fsf@gnu.org> <86plom4v3s.fsf@gnu.org> <87jzeu8xl8.wl-herbert@gojira.at> X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 54154 Cc: larsi@gnus.org, 54154@debbugs.gnu.org, emacs+bug@tetzco.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) > Date: Sun, 29 Sep 2024 14:15:47 +0200 > From: "Herbert J. Skuhra" > Cc: larsi@gnus.org, > 54154@debbugs.gnu.org, > emacs+bug@tetzco.de > > On Sun, 29 Sep 2024 12:23:03 +0200, > Eli Zaretskii wrote: > > > > > Cc: larsi@gnus.org, 54154@debbugs.gnu.org, emacs+bug@tetzco.de > > > Date: Sun, 29 Sep 2024 12:29:39 +0300 > > > From: Eli Zaretskii > > > > > > > - Run emacs -Q > > > > - M-x sieve-manage > > > > - Input IMAP server (e.g. imap.mailbox.org) > > > > > > > > Debugger entered--Lisp error: (wrong-type-argument stringp t) > > > > sieve-manage--append-to-log("sieve-manage: Connecting to imap.mailbox.org..." "\n") > > > > sieve-manage--message("Connecting to %s..." "imap.mailbox.org") > > > > sieve-manage-open("imap.mailbox.org" nil) > > > > sieve-open-server("imap.mailbox.org" nil) > > > > sieve-manage("imap.mailbox.org") > > > > funcall-interactively(sieve-manage "imap.mailbox.org") > > > > command-execute(sieve-manage record) > > > > execute-extended-command(nil "sieve-manage" "sieve-ma") > > > > funcall-interactively(execute-extended-command nil "sieve-manage" "sieve-ma") > > > > command-execute(execute-extended-command) > > > > > > > > On 2nd try it works! > > > > > > The fix was not merged to the-then master branch, so it is not in > > > Emacs 30. Therefore, it is not a surprise you still see the problem: > > > it was only fixed in Emacs 29. > > > > > > I lost the context long ago, so I hope Kai will suggest how to handle > > > this. > > > > However, it's quite possible that the following band-aid should fix > > this particular issue: > > > > diff --git a/lisp/net/sieve-manage.el b/lisp/net/sieve-manage.el > > index 0faeb02..da2167c 100644 > > --- a/lisp/net/sieve-manage.el > > +++ b/lisp/net/sieve-manage.el > > @@ -178,7 +178,8 @@ sieve-manage--append-to-log > > (with-current-buffer > > (get-buffer-create sieve-manage-log) > > (set-buffer-multibyte nil) > > - (buffer-disable-undo))) > > + (buffer-disable-undo) > > + (current-buffer))) > > (goto-char (point-max)) > > (apply #'insert args)))) > > Yes, this works. Thanks. Thanks, installed on the emacs-30 branch. > As manage-sieve logs credentials maybe logging should be disabled by > default and/or credentials shouldn't be logged when logging is > enabled? I'll let Kai and others comment on that. From unknown Thu Aug 21 12:12:19 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Mon, 28 Oct 2024 11:24:10 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator