From unknown Sat Jun 21 10:31:28 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#44072 <44072@debbugs.gnu.org> To: bug#44072 <44072@debbugs.gnu.org> Subject: Status: 27.1; url-retrieve-synchronously via proxy gets into wait loop without process. Reply-To: bug#44072 <44072@debbugs.gnu.org> Date: Sat, 21 Jun 2025 17:31:28 +0000 retitle 44072 27.1; url-retrieve-synchronously via proxy gets into wait loo= p without process. reassign 44072 emacs submitter 44072 Peter Wagemans severity 44072 normal thanks From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 18 18:51:07 2020 Received: (at submit) by debbugs.gnu.org; 18 Oct 2020 22:51:07 +0000 Received: from localhost ([127.0.0.1]:39329 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kUHW4-0001E4-8j for submit@debbugs.gnu.org; Sun, 18 Oct 2020 18:51:07 -0400 Received: from lists.gnu.org ([209.51.188.17]:33504) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kUGgR-000695-0S for submit@debbugs.gnu.org; Sun, 18 Oct 2020 17:57:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51444) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUGgQ-0000GP-Mr for bug-gnu-emacs@gnu.org; Sun, 18 Oct 2020 17:57:42 -0400 Received: from lb2-smtp-cloud9.xs4all.net ([194.109.24.26]:42973) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUGgO-0002ME-7L for bug-gnu-emacs@gnu.org; Sun, 18 Oct 2020 17:57:42 -0400 Received: from cust-7b3ff6f9 ([IPv6:fc0c:c181:994d:cd57:27db:a082:bdc1:1f0e]) by smtp-cloud9.xs4all.net with ESMTPA id UGgHk6CTIGDeXUGgLknNQe; Sun, 18 Oct 2020 23:57:37 +0200 From: Peter Wagemans Subject: 27.1; url-retrieve-synchronously via proxy gets into wait loop without process. To: bug-gnu-emacs@gnu.org Message-ID: Date: Sun, 18 Oct 2020 23:57:33 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4xfJ/g6eujc8lhfS5cWB/GHgZlOmrm1Aod7VdmSXEugd6ixRUXUliO/IvHWhaEyxudtHElPL6E2vMV22XCRoE+D8w+AgCaEfNJCNlngtTkfF4/A+yEzDZp Ws4WzlR8TV1OJnG1HRebEfHRjUYNgAFVoiWNGi05bl4m+s9o5Bo9SsUW2R92+TZNRt1hyHuAC3rU3QVlLecvR75Tllq1jIIjBLQ= Received-SPF: pass client-ip=194.109.24.26; envelope-from=pwml_no001spam@xs4all.nl; helo=lb2-smtp-cloud9.xs4all.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/18 17:57:38 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Sun, 18 Oct 2020 18:51:03 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) The problem was encountered on Windows 10, after migration from Emacs 26.3 to 27.1, with url retrieval via a Cygwin cntlm proxy and VPN to company proxy (difficult to reproduce the timing characteristics I guess). Emacs was no longer able to install or update many melpa packages from the (package-)list-packages buffer. Package update successfully downloads the package and then looks for a .sig file that doesn't exist on melpa.org. The melpa nginx returns http status 404 with a small gzipped html content body that contains the "404 Not Found" message. This html body becomes about 100 bytes gzipped. For this, the melpa nginx doesn't use chunked transfer but also doesn't send Content-Length (hmm, rfc7230: ..., a server SHOULD generate encoding or length-delimited messages whenever possible.), so Emacs can't use the length to detect that the transfer is complete (url-debug: "http -> No content-length, being dumb") and should conclude that the transfer is complete when the connection is closed. It seems that the loop in url-retrieve-synchronously doesn't pick up that event and doesn't notice that the process is gone. Note: melpa nginx not sending Content-Length seems to depend on the url-privacy-level of Emacs; the setting paranoid (used above) doesn't send a User-Agent header but the default setting does tell melpa that the user agent is Emacs (User-Agent: URL/Emacs Emacs/...) and with that privacy setting you get an uncompressed response with Content-Length and the problem doesn't occur. Without proxy, I haven't seen the problem: url-debug shows that url-http-end-of-document-sentinel gets called and the package install proceeds. Below is an example url-debug trace, with the routine modified to print the buffer content in the loop. The binary gzip data isn't very readable, so it was replaced by the string BINARYGZIPDATA. In the first iteration, the complete response is already returned, but without length indication, so Emacs doesn't know that until the connection is closed. The second time, the string "Process melpa.org connection broken by remote peer" has been added and after that the looping continues with no further change in the content. When you interrupt the loop, the buffer has no process and the backtrace shows that the value of proc in the loop is nil. Workaround for the problem: (setq url-mime-encoding-string "identity") to not compress the response content, or lower the privacy level. Then you get Content-Length from nginx and the problem is avoided. Or arrange for internet access without proxy. Perhaps similar to https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34103 26.1; url-retrieve-synchronously freezes in an infinite loop when behind a proxy because of `url-asynchronous' being bound to nil although I haven't encountered it with the Emacs 26.x versions. An example url-debug extract: http -> Contacting host: melpa.org:443 http -> Marking connection as busy: melpa.org:443 # http -> getting referer from buffer: buffer:# target-url:#s(url "https" nil nil "melpa.org" nil "/packages/async-20200809.501.tar.sig" nil nil t nil t nil) lastloc:nil retrieval -> Spinning in url-retrieve-synchronously: nil ("") http -> Calling after change function `url-https-proxy-after-change-function' for `#' http -> url-http-parse-response called in ( *http melpa.org:443*-499627) http -> Request is: GET /packages/async-20200809.501.tar.sig HTTP/1.1 MIME-Version: 1.0 Connection: close Extension: Security/Digest Security/SSL Host: melpa.org Accept-encoding: gzip Accept: */* retrieval -> Spinning in url-retrieve-synchronously: nil ("") http -> Calling after change function `url-http-wait-for-headers-change-function' for `#' http -> url-http-wait-for-headers-change-function ( *http melpa.org:443*-499627) http -> Saw end of headers... ( *http melpa.org:443*-499627) http -> url-http-parse-response called in ( *http melpa.org:443*-499627) http -> No content-length, being dumb. http -> Spinning waiting for headers... retrieval -> Spinning in url-retrieve-synchronously: nil ("HTTP/1.1 404 Not Found Server: nginx Date: Fri, 16 Oct 2020 14:33:46 GMT Content-Type: text/html; charset=utf-8 Connection: close Vary: Accept-Encoding Content-Encoding: gzip BINARYGZIPDATA") retrieval -> Spinning in url-retrieve-synchronously: nil ("HTTP/1.1 404 Not Found Server: nginx Date: Fri, 16 Oct 2020 14:33:46 GMT Content-Type: text/html; charset=utf-8 Connection: close Vary: Accept-Encoding Content-Encoding: gzip BINARYGZIPDATA Process melpa.org connection broken by remote peer ") retrieval -> Spinning in url-retrieve-synchronously: nil ("HTTP/1.1 404 Not Found Server: nginx Date: Fri, 16 Oct 2020 14:33:46 GMT Content-Type: text/html; charset=utf-8 Connection: close Vary: Accept-Encoding Content-Encoding: gzip BINARYGZIPDATA Process melpa.org connection broken by remote peer ") retrieval -> Spinning in url-retrieve-synchronously: nil ("HTTP/1.1 404 Not Found ... looping Regards, Peter Wagemans