From debbugs-submit-bounces@debbugs.gnu.org Tue Aug 27 21:22:16 2024 Received: (at submit) by debbugs.gnu.org; 28 Aug 2024 01:22:16 +0000 Received: from localhost ([127.0.0.1]:47925 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sj7O2-0008W9-9r for submit@debbugs.gnu.org; Tue, 27 Aug 2024 21:22:16 -0400 Received: from lists.gnu.org ([209.51.188.17]:40846) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sj5OS-0004uF-83 for submit@debbugs.gnu.org; Tue, 27 Aug 2024 19:14:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sj5NX-00028i-4r for bug-gnu-emacs@gnu.org; Tue, 27 Aug 2024 19:13:37 -0400 Received: from out-178.mta1.migadu.com ([95.215.58.178]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sj5NU-00006j-6G for bug-gnu-emacs@gnu.org; Tue, 27 Aug 2024 19:13:34 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lipklim.org; s=key1; t=1724800408; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=8/rHMyNG/uPcovRz6cceL1iJEZlxcVtjZx9tmdc4DSU=; b=FwnT/dSNID02x/Efm+/pzXdpXTHw+GThBEkxBAdf8UcdYTDpRYLoh6S+UoUuPgtXv1gjJK kQ98sdrH0LCKg45SjUIYuHVNe3ztliz5JdkvuJwBURJ02l+XmeUEqAubdfqzk1rCL6S3k6 xsrDDIhhKrA4Rvf7Vscgsox8Pn0FYcM= From: Evgenii Klimov To: bug-gnu-emacs@gnu.org Subject: [PATCH] Keep project's exec-path during with-temp-buffer call Date: Wed, 28 Aug 2024 00:13:25 +0100 Message-ID: <87v7zlo8yy.fsf@lipklim.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Migadu-Flow: FLOW_OUT Received-SPF: pass client-ip=95.215.58.178; envelope-from=eugene.dev@lipklim.org; helo=out-178.mta1.migadu.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Tue, 27 Aug 2024 21:22:13 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Tags: patch `with-temp-buffer' doesn't respect buffer-local environment variables, `exec-path' in this case. Which results in executables not being found, or the wrong versions of executables being picked up. E.g. if environment variable is modified via .dir-local file or direnv/envrc package. I see that this function tries to be remote-host friendly (uses `process-file') so I tried to ensure that this patch doesn't break this effort, but I'm not sure that I understand the machinery behind TRAMP correctly. So please consider this aspect from your side. This patch shouldn't interfere with TRAMP, if I understand `process-file`s doc correctly: If a file name handler is invoked, it determines the program to run based on the first argument PROGRAM. For instance, suppose that a handler for remote files is invoked. Then the path that is used for searching for the program might be different from =E2=80=98exec-pa= th=E2=80=99. In GNU Emacs 30.0.60 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.41, cairo version 1.18.0) Windowing system distributor 'The X.Org Foundation', version 11.0.12101011 System Description: Guix System Configured using: 'configure CONFIG_SHELL=3D/gnu/store/fl3l5wx8qynjrvx5lilz6c38hb77cf36-bash-minimal-5.= 1.16/bin/bash SHELL=3D/gnu/store/fl3l5wx8qynjrvx5lilz6c38hb77cf36-bash-minimal-5.1.16/bi= n/bash --prefix=3D/gnu/store/45nwc8hc8fn1fhvr9qw01ylkfpvzxwsw-emacs-next-30.0.60-= 1.4e22ef8 --enable-fast-install --with-cairo --with-modules --with-native-compilation=3Daot --disable-build-details' --=-=-= Content-Type: text/patch Content-Disposition: attachment; filename=0001-Keep-project-s-exec-path-during-with-temp-buffer-cal.patch >From 9a6ca1c11a2849753fd3b854a79561224629a6bf Mon Sep 17 00:00:00 2001 From: Evgenii Klimov Date: Tue, 27 Aug 2024 23:08:47 +0100 Subject: [PATCH] Keep project's exec-path during with-temp-buffer call * lisp/progmodes/python.el (python-shell-prompt-detect): `with-temp-buffer' doesn't respect buffer-local environment variables, `exec-path' in this case. Which results in executables not being found, or the wrong versions of executables being picked up. E.g. if env var is modified via .dir-local file or direnv/envrc package. --- lisp/progmodes/python.el | 44 +++++++++++++++++++++------------------- 1 file changed, 23 insertions(+), 21 deletions(-) diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el index 7193cc19425..d6bb409c286 100644 --- a/lisp/progmodes/python.el +++ b/lisp/progmodes/python.el @@ -3116,27 +3116,29 @@ (defun python-shell-prompt-detect () (interpreter python-shell-interpreter) (interpreter-arg python-shell-interpreter-interactive-arg) (output - (with-temp-buffer - ;; TODO: improve error handling by using - ;; `condition-case' and displaying the error message to - ;; the user in the no-prompts warning. - (ignore-errors - (let ((code-file - ;; Python 2.x on Windows does not handle - ;; carriage returns in unbuffered mode. - (let ((inhibit-eol-conversion (getenv "PYTHONUNBUFFERED"))) - (python-shell--save-temp-file code)))) - (unwind-protect - ;; Use `process-file' as it is remote-host friendly. - (process-file - interpreter - code-file - '(t nil) - nil - interpreter-arg) - ;; Try to cleanup - (delete-file code-file)))) - (buffer-string))) + (let ((current-exec-path exec-path)) + (with-temp-buffer + ;; TODO: improve error handling by using + ;; `condition-case' and displaying the error message to + ;; the user in the no-prompts warning. + (ignore-errors + (let ((code-file + ;; Python 2.x on Windows does not handle + ;; carriage returns in unbuffered mode. + (let ((inhibit-eol-conversion (getenv "PYTHONUNBUFFERED"))) + (python-shell--save-temp-file code)))) + (unwind-protect + ;; Use `process-file' as it is remote-host friendly. + (let ((exec-path current-exec-path)) + (process-file + interpreter + code-file + '(t nil) + nil + interpreter-arg)) + ;; Try to cleanup + (delete-file code-file)))) + (buffer-string)))) (prompts (catch 'prompts (dolist (line (split-string output "\n" t)) -- 2.45.2 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 28 03:13:43 2024 Received: (at submit) by debbugs.gnu.org; 28 Aug 2024 07:13:43 +0000 Received: from localhost ([127.0.0.1]:48106 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sjCsB-0001gN-Au for submit@debbugs.gnu.org; Wed, 28 Aug 2024 03:13:43 -0400 Received: from lists.gnu.org ([209.51.188.17]:55298) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sjCs9-0001gD-J8 for submit@debbugs.gnu.org; Wed, 28 Aug 2024 03:13:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sjCrG-0006iI-Pn for bug-gnu-emacs@gnu.org; Wed, 28 Aug 2024 03:12:47 -0400 Received: from mout.gmx.net ([212.227.15.15]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sjCrE-0001kT-Sl for bug-gnu-emacs@gnu.org; Wed, 28 Aug 2024 03:12:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmx.de; s=s31663417; t=1724829151; x=1725433951; i=michael.albinus@gmx.de; bh=VqieClpDxg4McxrBXsqxfAen1KPPHY60fAjrhJq1iZg=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date: Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=E29D4OrXD3bSP9cBWxPp2UUa4CxGkkYPtrReFd3sB46l/ihwjX5GgGjDEOJRa5kv Ke/jhPsX38+wslbeVgcD5nDQOWCA4suMhozwb8Hwkeg+E3mFhoDytGXv/rTMTKKTl uIowQODQN6VElWGgOqYvIbWFO1umJUiZsKk93iRDZ/MJ5qaf/5rQt/FvVg0zA+ToB Slg5Y2A55DdmfmRRB41Cn254vCneQexxAigsIavVNKb/6T9dTHlBNF/HdtU+Kxtcu VeVe+HXCflcmWqHQaSlppisFAbjtOuYwVjY3PyhY2Y1K3WJ+xdWTDoVchZt8Oq7KX bUHbzD9dW75qeWz7Xw== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.38.155]) by mail.gmx.net (mrgmx004 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MXp5a-1sbTbX1oVw-00XMda; Wed, 28 Aug 2024 09:12:31 +0200 From: Michael Albinus To: Evgenii Klimov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Subject: Re: bug#72849: [PATCH] Keep project's exec-path during with-temp-buffer call In-Reply-To: <87v7zlo8yy.fsf@lipklim.org> (Evgenii Klimov via's message of "Wed, 28 Aug 2024 00:13:25 +0100") References: <87v7zlo8yy.fsf@lipklim.org> Date: Wed, 28 Aug 2024 09:12:28 +0200 Message-ID: <87jzg1p1cz.fsf@gmx.de> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:1sLH/pqWdmiNnEQaLJp/8neLRZ1neZ4BtUtXWIfBnI20kgyQHKO EolukMWU3txmj4aVVSSnuVZnYuEcdaupEb0vX+EVGqNYxyPjf01zM9Ik5x6cRLq7RgLJaDG zBgfpQO4d7Wi76DOs0Shi5U2KtEJUAkzlT8DQYAkm5GJHfZinf176yirAVXp9HCCUgtRLXu t1TZQM/IXYwZFI3rnSRtw== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:isq6wW4beqg=;5Wu+GoWZ0Ja2Q9FNSZwraG0yLN3 lIixeZ5gYj3y65FpbzjWKO8tahv4WSA85OA7L2lItan7UHoE7M0zm+DUxYXT5wM+5H6CLIefs mgqsY/tPFUB3f+KxC742J1EVEmcitZEuHrVedj90p9QpbWKd+FCR1/evw5Q3a4S2rwYIiJ3yM s2q95mXeNcpDXiysl+G4KlUs6PoGJ7vWn67ob0TmANhpRTsWgbX0hs480J7m3yx+FiXQfgAvF fx2b8wyhPc7k/P0XtjKCbkIhTjmf4LTFDMZPAc+I7HY1dYobsQY7oNGiV9p13Iyi9j+KCOEdg Jt2micvook6w+wpL/nCw+OTtkpRXzKjlKdcbyo3sj7I23idDjViMp49JpboDQIBU4WOEZYO80 sZH7CN0TrNQzUJTY9fBsvIUoXZ370tQ24ql6LXaRLcXF6dvJkdbyQ+9/N/yXiDunYpstbiT9S hZ72dRI0aP9PxYKBn869W3WbXJhBg/jBumjuG4e33fy5JUlZc++Oc579uNDetgUayoSEaItBg /EN5rhnnju3z4NfqgsOkkhQtTevb4IUv3xtR6kcTipQkXPOWuV0skH/DdcuwF3AudM1xfPNL/ Ealy5wCXU7OkV3y7so/8aDdvs1EE3GlvxqYXFWSdAwgb7xvxwhLxdgOv/8INsEoVt0s7TiZlP jM1Olb8PMqQcG7U+L9A/xVVjYUB8BniNvvJ6dd8syVdkkbm0ZKRB/wgckzOrLL4CpKU5GJL12 /L0RvXi0vGms8gMNLdZJWxQODPtagzT0FTtkUEP9UXdFyknJU0V9XTevxhltEXbaXVILgh3vf SfO+8/VAy7qb4UNYH91JGFlA== Received-SPF: pass client-ip=212.227.15.15; envelope-from=michael.albinus@gmx.de; helo=mout.gmx.net X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: Evgenii Klimov , 72849@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) Evgenii Klimov via "Bug reports for GNU Emacs, the Swiss army knife of text editors" writes: Hi Evgenii, > I see that this function tries to be remote-host friendly (uses > `process-file') so I tried to ensure that this patch doesn't break this > effort, but I'm not sure that I understand the machinery behind TRAMP > correctly. So please consider this aspect from your side. > > This patch shouldn't interfere with TRAMP, if I understand > `process-file`s doc correctly: > > If a file name handler is invoked, it determines the program to run > based on the first argument PROGRAM. For instance, suppose that a > handler for remote files is invoked. Then the path that is used > for searching for the program might be different from =E2=80=98exec-= path=E2=80=99. Correct. Remote processes do not use the variable exec-path, so you can change it at will, w/o breaking remote processes. Of course, exec-path should still ensure that local programs, needed for accessing a remote host (like ssh), shall still be usable. Best regards, Michael. From debbugs-submit-bounces@debbugs.gnu.org Wed Aug 28 08:13:59 2024 Received: (at 72849) by debbugs.gnu.org; 28 Aug 2024 12:13:59 +0000 Received: from localhost ([127.0.0.1]:48386 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sjHYl-00026j-3x for submit@debbugs.gnu.org; Wed, 28 Aug 2024 08:13:59 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47792) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sjHYj-00026S-HM for 72849@debbugs.gnu.org; Wed, 28 Aug 2024 08:13:58 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sjHXk-0008EJ-VI; Wed, 28 Aug 2024 08:12:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=ompPeFPR6s3WzQO54Duxcx3udK2WiIBSEaABjPoXuZs=; b=JAYYd6g2+vLl bTrdOm2p9l8cq07mV7SmTExo0AygGbYYUog+MQ/mzKwCZG4lEtp7tQetMWGnql9ykvo3cJ2FFI42z n9TIly2IFSx/VDKGDWV3ktc7BHUmeXESx9hEsZLPDw5PzfHEG7WqxtXA+p8fCE3RJeQGfiaCMuLSZ fPAJFsnLvmIlbwaUCL/te+yICWEN0fTrNN0atbMCGIUIxntsv/9XU1bR8d0dlvWLpeRQ7rItWPH1r boNAtHrEBKmq2ubESl+JX5RCj1P6Fdz2hH0vRRlgQ7Tfx9v6Gc154FiZwI5IAx3QmC/Gxz0MQ8vNF SJ2ArPjhcWZRTGtNw83sEQ==; Date: Wed, 28 Aug 2024 15:12:54 +0300 Message-Id: <86y14ghmm1.fsf@gnu.org> From: Eli Zaretskii To: Evgenii Klimov , kobarity In-Reply-To: <87v7zlo8yy.fsf@lipklim.org> (bug-gnu-emacs@gnu.org) Subject: Re: bug#72849: [PATCH] Keep project's exec-path during with-temp-buffer call References: <87v7zlo8yy.fsf@lipklim.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 72849 Cc: 72849@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Wed, 28 Aug 2024 00:13:25 +0100 > From: Evgenii Klimov via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" > > `with-temp-buffer' doesn't respect buffer-local environment variables, > `exec-path' in this case. Which results in executables not being found, > or the wrong versions of executables being picked up. E.g. if > environment variable is modified via .dir-local file or direnv/envrc > package. Hmm, this doesn't look clean to me: exec-path is just one variable, what makes it special here? Moreover, it sounds like python-shell-with-environment, which python-shell-prompt-detect calls, already attempts to have buffer-local value of exec-path to be available to Python, so why isn't that working for you? And if it isn't work, I think we should amend python-shell-with-environment to do this, so we don't need to do it "by hand". kobarity, any comments or suggestions? > >From 9a6ca1c11a2849753fd3b854a79561224629a6bf Mon Sep 17 00:00:00 2001 > From: Evgenii Klimov > Date: Tue, 27 Aug 2024 23:08:47 +0100 > Subject: [PATCH] Keep project's exec-path during with-temp-buffer call > > * lisp/progmodes/python.el (python-shell-prompt-detect): > `with-temp-buffer' doesn't respect buffer-local environment > variables, `exec-path' in this case. Which results in executables > not being found, or the wrong versions of executables being picked > up. E.g. if env var is modified via .dir-local file or > direnv/envrc package. > --- > lisp/progmodes/python.el | 44 +++++++++++++++++++++------------------- > 1 file changed, 23 insertions(+), 21 deletions(-) > > diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el > index 7193cc19425..d6bb409c286 100644 > --- a/lisp/progmodes/python.el > +++ b/lisp/progmodes/python.el > @@ -3116,27 +3116,29 @@ (defun python-shell-prompt-detect () > (interpreter python-shell-interpreter) > (interpreter-arg python-shell-interpreter-interactive-arg) > (output > - (with-temp-buffer > - ;; TODO: improve error handling by using > - ;; `condition-case' and displaying the error message to > - ;; the user in the no-prompts warning. > - (ignore-errors > - (let ((code-file > - ;; Python 2.x on Windows does not handle > - ;; carriage returns in unbuffered mode. > - (let ((inhibit-eol-conversion (getenv "PYTHONUNBUFFERED"))) > - (python-shell--save-temp-file code)))) > - (unwind-protect > - ;; Use `process-file' as it is remote-host friendly. > - (process-file > - interpreter > - code-file > - '(t nil) > - nil > - interpreter-arg) > - ;; Try to cleanup > - (delete-file code-file)))) > - (buffer-string))) > + (let ((current-exec-path exec-path)) > + (with-temp-buffer > + ;; TODO: improve error handling by using > + ;; `condition-case' and displaying the error message to > + ;; the user in the no-prompts warning. > + (ignore-errors > + (let ((code-file > + ;; Python 2.x on Windows does not handle > + ;; carriage returns in unbuffered mode. > + (let ((inhibit-eol-conversion (getenv "PYTHONUNBUFFERED"))) > + (python-shell--save-temp-file code)))) > + (unwind-protect > + ;; Use `process-file' as it is remote-host friendly. > + (let ((exec-path current-exec-path)) > + (process-file > + interpreter > + code-file > + '(t nil) > + nil > + interpreter-arg)) > + ;; Try to cleanup > + (delete-file code-file)))) > + (buffer-string)))) > (prompts > (catch 'prompts > (dolist (line (split-string output "\n" t)) > -- > 2.45.2 > From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 29 12:10:35 2024 Received: (at 72849) by debbugs.gnu.org; 29 Aug 2024 16:10:35 +0000 Received: from localhost ([127.0.0.1]:52033 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sjhjG-0004eK-SJ for submit@debbugs.gnu.org; Thu, 29 Aug 2024 12:10:35 -0400 Received: from mail-pg1-f182.google.com ([209.85.215.182]:48610) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sjhjE-0004e3-2C for 72849@debbugs.gnu.org; Thu, 29 Aug 2024 12:10:33 -0400 Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-7cb3db0932cso569583a12.1 for <72849@debbugs.gnu.org>; Thu, 29 Aug 2024 09:09:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724947710; x=1725552510; darn=debbugs.gnu.org; h=mime-version:user-agent:references:in-reply-to:subject:cc:to:from :message-id:date:from:to:cc:subject:date:message-id:reply-to; bh=diDjS/SvO8xrSqGPSemnPtkLyX08isjvgDUXEbNGsAE=; b=GusikFvnS+uHt+sGeXpbp4D3bZxUrkDANWg//e1wX7iRP1jtlZbblgCIis1UIHVfQR qfcoLpWGjQmjuw3AfElQYjFEfS+iCOkZNtbdjy4m53qwXp9NN+wyc0aJd28949c7kQeT d5tHT5G0xXGgMhFnkJ4iV/kT8Xhfd2HwbZsDXvkLMg0KYe6Kdk7Zzb5RGxO20N6YjsZQ 8EVwQ9OuGT8g584/qGTTefHvX7H/YfnsqN0jM4PXTmfLITwnLQVAc4NtMcbb3hvUCZCa AMyFCkdlXy7YuwwNQWf/UtVcyqBTintVuZmb0J6+kVGaCnx4rvtgshBBbEbsvTwaZQXV cPwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724947710; x=1725552510; h=mime-version:user-agent:references:in-reply-to:subject:cc:to:from :message-id:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=diDjS/SvO8xrSqGPSemnPtkLyX08isjvgDUXEbNGsAE=; b=RCrm74BJ+6c9BhKoiZ8JWA4kVEC2+kjUhbLgOQqPKeC2uL+59QHr7DB28nix7d0+Ym 9nQE1eplOTBLGIV6ZoV/tghM3ijt3DZlDYQLC1pq/U41EKcgsiurx5+gVQBsZHVcVdNC VzXHci5zOg/f85LoydqqhUHLBgPINl8TRK23U2IgA9EWUSsKnJV5KlooHiO24XVu4zDg NxOHUHaWMp8tr/rk8rV1EVXWijVc7qg6PxRqnyxznA5ICFFJMkhY63j/1crzuMhaSOUQ 2H8HCGn7+NSd2j9ceHOB/7JcSoWm1w30dQa8vKSWb5CJCMIRW6oJEMAuhaFfyJJzwaeF krzQ== X-Gm-Message-State: AOJu0YyoJhiLLpwiQD6Y73pz4Pes64LxHwbfb6LQcM8TBBlE7LxBcRW4 WJ/6QYDSlpKCrxS4VQWeMQLI7uUlT6agAbINwsdpSQVcBRHqXbcq X-Google-Smtp-Source: AGHT+IEY2ujG+zrbtuz/Cs0GnG5uLMn6Pht1fgdYveTV2D/NSHFcrjoZKMNBdlnxwcECpUy+CuPqEA== X-Received: by 2002:a17:90a:51c4:b0:2c2:df58:bb8c with SMTP id 98e67ed59e1d1-2d85619f138mr3750970a91.18.1724947710056; Thu, 29 Aug 2024 09:08:30 -0700 (PDT) Received: from localhost (58x12x133x161.ap58.ftth.ucom.ne.jp. [58.12.133.161]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d8445e7faesm4340707a91.16.2024.08.29.09.08.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Aug 2024 09:08:29 -0700 (PDT) Date: Fri, 30 Aug 2024 01:08:25 +0900 Message-ID: From: kobarity To: Eli Zaretskii , Evgenii Klimov Subject: Re: bug#72849: [PATCH] Keep project's exec-path during with-temp-buffer call In-Reply-To: <86y14ghmm1.fsf@gnu.org> References: <87v7zlo8yy.fsf@lipklim.org> <86y14ghmm1.fsf@gnu.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.0.50 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: multipart/mixed; boundary="Multipart_Fri_Aug_30_01:08:25_2024-1" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 72849 Cc: 72849@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --Multipart_Fri_Aug_30_01:08:25_2024-1 Content-Type: text/plain; charset=US-ASCII Eli Zaretskii wrote: > > Date: Wed, 28 Aug 2024 00:13:25 +0100 > > From: Evgenii Klimov via "Bug reports for GNU Emacs, > > the Swiss army knife of text editors" > > > > `with-temp-buffer' doesn't respect buffer-local environment variables, > > `exec-path' in this case. Which results in executables not being found, > > or the wrong versions of executables being picked up. E.g. if > > environment variable is modified via .dir-local file or direnv/envrc > > package. > > Hmm, this doesn't look clean to me: exec-path is just one variable, > what makes it special here? > > Moreover, it sounds like python-shell-with-environment, which > python-shell-prompt-detect calls, already attempts to have > buffer-local value of exec-path to be available to Python, so why > isn't that working for you? And if it isn't work, I think we should > amend python-shell-with-environment to do this, so we don't need to do > it "by hand". > > kobarity, any comments or suggestions? The problem is that `with-temp-buffer' creates a new buffer, so the buffer local value of `exec-path' is not maintained in the new buffer. It is used to find the Python interpreter unless the interpreter is specified using the absolute path. I think Evgenii's patch is logically correct, but it would be better to create a modified version of `with-temp-buffer' which keeps `exec-path' in the new buffer. The attached is my proposal to do so. --Multipart_Fri_Aug_30_01:08:25_2024-1 Content-Type: application/octet-stream; type=patch; name="0001-Fix-Python-prompt-detection-when-exec-path-is-buffer.patch" Content-Disposition: attachment; filename="0001-Fix-Python-prompt-detection-when-exec-path-is-buffer.patch" Content-Transfer-Encoding: 7bit >From b94ba0d4e07635a0abddf0cba7548b10b47dad92 Mon Sep 17 00:00:00 2001 From: kobarity Date: Fri, 30 Aug 2024 00:13:27 +0900 Subject: [PATCH] Fix Python prompt detection when 'exec-path' is buffer local * lisp/progmodes/python.el (python-with-temp-buffer): New macro same as 'with-temp-buffer', except the variable 'exec-path' is kept in the new buffer. (python-shell-prompt-detect): Use it. (Bug#72849) Co-authored-by: Evgenii Klimov --- lisp/progmodes/python.el | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el index a00289d6de9..480713c3aca 100644 --- a/lisp/progmodes/python.el +++ b/lisp/progmodes/python.el @@ -3088,6 +3088,16 @@ 'python--parse-json-array (json-read-from-string string)))) "Parse the JSON array in STRING into a Lisp list.") +(defmacro python-with-temp-buffer (&rest body) + "Python mode specific version of `with-temp-buffer'. +Call `with-temp-buffer' with BODY. The variable `exec-path' is kept in +the new buffer even if it is buffer local." + (declare (indent 0) (debug t)) + `(let ((python--exec-path exec-path)) + (with-temp-buffer + (let ((exec-path python--exec-path)) + ,@body)))) + (defun python-shell-prompt-detect () "Detect prompts for the current `python-shell-interpreter'. When prompts can be retrieved successfully from the @@ -3116,7 +3126,7 @@ python-shell-prompt-detect (interpreter python-shell-interpreter) (interpreter-arg python-shell-interpreter-interactive-arg) (output - (with-temp-buffer + (python-with-temp-buffer ;; TODO: improve error handling by using ;; `condition-case' and displaying the error message to ;; the user in the no-prompts warning. -- 2.34.1 --Multipart_Fri_Aug_30_01:08:25_2024-1-- From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 29 12:26:57 2024 Received: (at 72849) by debbugs.gnu.org; 29 Aug 2024 16:26:57 +0000 Received: from localhost ([127.0.0.1]:52047 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sjhz6-0005BS-Qy for submit@debbugs.gnu.org; Thu, 29 Aug 2024 12:26:57 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45100) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sjhz3-0005B8-39 for 72849@debbugs.gnu.org; Thu, 29 Aug 2024 12:26:55 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sjhy3-0006Et-DN; Thu, 29 Aug 2024 12:25:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=JcXAcMBpjqO2K1/MGQhqJkGEAn50z4bu6a1qmSiZrWQ=; b=hefP1iUxm5gw U41PeO5jkNA8vEoXqsRmRGgn76WPz0x49g0n9piakoTKnS3TVJyWRKQBDSac42J2ciNT6rFqNUfwC 800XVkT8juG2odFvei5xvHD0OZa9r4v4L+wXobyh7vOVlBCsNSWrjpYx5SKWDehQvMZGkGZdVBTVm i3AVeGfSX9Aapdzw3rSdR2GEaCCjqndSLLNNnPX9yG3bZqOxmXzlaOdAZiUTbzpx1cB/RrhlF+pgz ayiJbqjSmPJ1k3pr/72K0s2ci81EXIABDZyE9J/cdIQfUPIM8Ij0l9pAkCU1CuMXCFyzvDv3MKIAF YrzEi0d0+3Zk6H72jxdUfw==; Date: Thu, 29 Aug 2024 19:25:49 +0300 Message-Id: <864j73fg8i.fsf@gnu.org> From: Eli Zaretskii To: kobarity In-Reply-To: (message from kobarity on Fri, 30 Aug 2024 01:08:25 +0900) Subject: Re: bug#72849: [PATCH] Keep project's exec-path during with-temp-buffer call References: <87v7zlo8yy.fsf@lipklim.org> <86y14ghmm1.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 72849 Cc: eugene.dev@lipklim.org, 72849@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Fri, 30 Aug 2024 01:08:25 +0900 > From: kobarity > Cc: 72849@debbugs.gnu.org > > Eli Zaretskii wrote: > > > Date: Wed, 28 Aug 2024 00:13:25 +0100 > > > From: Evgenii Klimov via "Bug reports for GNU Emacs, > > > the Swiss army knife of text editors" > > > > > > `with-temp-buffer' doesn't respect buffer-local environment variables, > > > `exec-path' in this case. Which results in executables not being found, > > > or the wrong versions of executables being picked up. E.g. if > > > environment variable is modified via .dir-local file or direnv/envrc > > > package. > > > > Hmm, this doesn't look clean to me: exec-path is just one variable, > > what makes it special here? > > > > Moreover, it sounds like python-shell-with-environment, which > > python-shell-prompt-detect calls, already attempts to have > > buffer-local value of exec-path to be available to Python, so why > > isn't that working for you? And if it isn't work, I think we should > > amend python-shell-with-environment to do this, so we don't need to do > > it "by hand". > > > > kobarity, any comments or suggestions? > > The problem is that `with-temp-buffer' creates a new buffer, so the > buffer local value of `exec-path' is not maintained in the new buffer. > It is used to find the Python interpreter unless the interpreter is > specified using the absolute path. > > I think Evgenii's patch is logically correct, but it would be better > to create a modified version of `with-temp-buffer' which keeps > `exec-path' in the new buffer. The attached is my proposal to do so. Thanks, but what about python-shell-with-environment: doesn't it already try to do this, including preserving exec-path from the original buffer? From debbugs-submit-bounces@debbugs.gnu.org Thu Aug 29 18:53:09 2024 Received: (at 72849) by debbugs.gnu.org; 29 Aug 2024 22:53:10 +0000 Received: from localhost ([127.0.0.1]:52238 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sjo0r-0007p3-Bz for submit@debbugs.gnu.org; Thu, 29 Aug 2024 18:53:09 -0400 Received: from out-177.mta0.migadu.com ([91.218.175.177]:65086) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sjo0n-0007oU-Td for 72849@debbugs.gnu.org; Thu, 29 Aug 2024 18:53:07 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lipklim.org; s=key1; t=1724971893; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Ip7cSZheQO2n06m7S65Y9sPrF+YrBLc8VquQGTWEwd0=; b=WpQtV5QBnCcSjlSmwVsckNpO26uIhH0QPMEOB+Jp19G+cg7JqvDIUcZg8u1NbIQEItYGu0 0k0IwkVYLN7j2YNAvLB77uMU9JXmeNVcvET0DUbPsB9cH0dq2MVgBh64IPhg/GrXHIK8cQ aX6hMTLwvGHMgLz1M6CzZ2+xg1l4RNQ= From: Evgenii Klimov To: Eli Zaretskii Subject: Re: bug#72849: [PATCH] Keep project's exec-path during with-temp-buffer call In-Reply-To: <86y14ghmm1.fsf@gnu.org> (Eli Zaretskii's message of "Wed, 28 Aug 2024 15:12:54 +0300") References: <87v7zlo8yy.fsf@lipklim.org> <86y14ghmm1.fsf@gnu.org> Date: Thu, 29 Aug 2024 23:51:29 +0100 Message-ID: <87ed677xji.fsf@lipklim.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Migadu-Flow: FLOW_OUT X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 72849 Cc: kobarity , 72849@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Eli Zaretskii writes: >> Date: Wed, 28 Aug 2024 00:13:25 +0100 >> From: Evgenii Klimov via "Bug reports for GNU Emacs, >> the Swiss army knife of text editors" >> >> `with-temp-buffer' doesn't respect buffer-local environment variables, >> `exec-path' in this case. Which results in executables not being found, >> or the wrong versions of executables being picked up. E.g. if >> environment variable is modified via .dir-local file or direnv/envrc >> package. > > Hmm, this doesn't look clean to me: exec-path is just one variable, > what makes it special here? > > Moreover, it sounds like python-shell-with-environment, which > python-shell-prompt-detect calls, already attempts to have > buffer-local value of exec-path to be available to Python, so why > isn't that working for you? And if it isn't work, I think we should > amend python-shell-with-environment to do this, so we don't need to do > it "by hand". Indeed, my initial approach is too manual. Here the problem that I have: I don't use Python's "venv" module to create virtual environment for the project. Instead, I use GNU Guix's "guix shell" command [1] which provides augmented PATH and PYTHONPATH, etc. to link project's dependencies. Then, envrc.el package picks up these environment variables and makes them buffer-local project-wise (`exec-path' and `process-environment'). You are correct that `python-shell-with-environment' provides buffer-local variables, but `with-temp-buffer' treats `exec-path' and `process-environment' variables very specially. I didn't find this behavior in documentation, but look at this example: (setq-default exec-path (list "global" "list")) (setq-local exec-path (cons "local" (default-value 'exec-path))) (setq-default myvar (list "global" "list")) (setq-local myvar (cons "local" (default-value 'myvar))) (let ((exec-path exec-path) ; takes buffer-local (myvar myvar)) ; takes buffer-local (with-temp-buffer (insert (car exec-path) ; uses global "\n" (car myvar)) ; uses `let'-binded (buffer-string))) ;; => "global ;; local" (require 'cl-lib) (let ((myvar myvar)) ;; temporarily binds buffer-local value to global symbol (cl-letf (((default-value 'exec-path) exec-path)) (with-temp-buffer ;; global variable is used, but it's value is temporarily equal ;; to buffer-local value (insert (car exec-path) "\n" (car myvar)) (buffer-string)))) ;; => "local ;; local" It's a simplified and expanded version of `python-shell-with-environment' and `python-shell-prompt-detect'. As you can see, `exec-path' is treated differently inside of `with-temp-buffer' and `cl-letf' is needed to force `with-temp-buffer' to use buffer-local value of `exec-path'. In the new patch attached I show how this can be overcome. Don't know if you'll consider my use case too narrow and specific, but I'll be glad to hear your thoughts on this. [1] https://guix.gnu.org/en/manual/en/guix.html#Invoking-guix-environment --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Make-python-shell-with-environment-respect-buffer-lo.patch >From 6db90f6087d0d3270e1d7a7d77d8830eac1f0154 Mon Sep 17 00:00:00 2001 From: Evgenii Klimov Date: Thu, 29 Aug 2024 23:36:12 +0100 Subject: [PATCH] Make python-shell--with-environment respect buffer-local variables * lisp/progmodes/python.el (python-shell--with-environment): Make `with-temp-buffer' respect buffer-local values of `process-environment' and `exec-path', if set. --- lisp/progmodes/python.el | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el index 7193cc19425..3e543442dba 100644 --- a/lisp/progmodes/python.el +++ b/lisp/progmodes/python.el @@ -3030,11 +3030,11 @@ (defun python-shell--with-environment (extraenv bodyfun) (tramp-dissect-file-name default-directory 'noexpand))))) (if vec (python-shell--tramp-with-environment vec extraenv bodyfun) - (let ((process-environment - (append extraenv process-environment)) - (exec-path - ;; FIXME: This is still Python-specific. - (python-shell-calculate-exec-path))) + (cl-letf (((default-value 'process-environment) + (append extraenv process-environment)) + ((default-value 'exec-path) + ;; FIXME: This is still Python-specific. + (python-shell-calculate-exec-path))) (funcall bodyfun))))) (defun python-shell--tramp-with-environment (vec extraenv bodyfun) -- 2.45.2 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Fri Aug 30 10:46:34 2024 Received: (at 72849) by debbugs.gnu.org; 30 Aug 2024 14:46:34 +0000 Received: from localhost ([127.0.0.1]:53329 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sk2tV-0000pz-N5 for submit@debbugs.gnu.org; Fri, 30 Aug 2024 10:46:34 -0400 Received: from mail-pl1-f180.google.com ([209.85.214.180]:56345) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sk2tS-0000pk-Cb for 72849@debbugs.gnu.org; Fri, 30 Aug 2024 10:46:32 -0400 Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-20231aa8908so14294275ad.0 for <72849@debbugs.gnu.org>; Fri, 30 Aug 2024 07:45:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725029067; x=1725633867; darn=debbugs.gnu.org; h=mime-version:user-agent:references:in-reply-to:subject:cc:to:from :message-id:date:from:to:cc:subject:date:message-id:reply-to; bh=i03LE1baFoPXpr4nHRE+Th6Ed0gTijEzY3qIv8NJGjw=; b=UQHc/WXUnzs2pt0peBNqm8mKxUlSw8DI0Y/qp5rn6I2wQEUBMbUv2rmZb1ZXn/DMz2 HTcEpzoBxeXSXmCZlCq2PH8xl4fck/svWUocpwA13TnsfzFP04rZ0/WKOvLU0CM+mf/p LxLtoyfXYjNibA091snZ0MDEZx1HK99+KiO46wtNm+iqM2ldNGpMvyD/uytWGNHHncwX F8SpY/fCl9Xpg43rVo62TGQ5PT4Ibx+/gSCWCS67SAp7/60+K8hSbQXFyZj/PVMpfW1P Kd9Pn3fBmGxQpVcBnXrCopfoMQqdlV0Xde+B8iPp33VBBQWRFz8+lUD5WOzRAM3KHtvE OeYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725029067; x=1725633867; h=mime-version:user-agent:references:in-reply-to:subject:cc:to:from :message-id:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=i03LE1baFoPXpr4nHRE+Th6Ed0gTijEzY3qIv8NJGjw=; b=uo4uk3Qa+sLD79RqIHUgFUm4qGJtN2Ih6aJRAzvzfmk3BKIv1IjEStmB2f+3Pak1jT ELVJn7zvRY9hRX2KYISEqkxEUUCluptSX1vL7eHB3gASRP1oHP7C6sbxVKOR+bQAqh1D pI5By7ujI4Z/DY9JdPz2AlNiqrsIKh1/i/Ho4Mzn/K+dyxjoLlfDIEHIwgm7LcdFk8b9 kid876F152S5AKiMqBMTXs8Lo5vtcgVhLVGZhoPTblYSBS5kU7NMcKPVkH3CvU/d7q0d ObwBno5v04PzH/9JdOfBjavv0Cma/oixFOSNn6lwyUPG2igXunUbNtdLzvksVl5y+Lpp zPmg== X-Gm-Message-State: AOJu0Yy+ArOmJTNB6KyTkcNhN6p9rMee7qJyiGqu0EkA67sMX33VTr6v KALtrQTClsPss2TlZCUPgYAOS98PTIUpiUbIprU3yEywAsLxPRHO X-Google-Smtp-Source: AGHT+IHuBWIEeLlIewfpJZwLV0QR1YZW8I0hPQYfYoehp6yj4tGYRv8A6oH7t95ikMi8wrFyeX4Dsw== X-Received: by 2002:a17:902:f54f:b0:202:26d:146c with SMTP id d9443c01a7336-2050c34f040mr64792325ad.5.1725029067209; Fri, 30 Aug 2024 07:44:27 -0700 (PDT) Received: from localhost (58x12x133x161.ap58.ftth.ucom.ne.jp. [58.12.133.161]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20515534334sm27687185ad.134.2024.08.30.07.44.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Aug 2024 07:44:26 -0700 (PDT) Date: Fri, 30 Aug 2024 23:43:57 +0900 Message-ID: From: kobarity To: Evgenii Klimov , Eli Zaretskii Subject: Re: bug#72849: [PATCH] Keep project's exec-path during with-temp-buffer call In-Reply-To: <87ed677xji.fsf@lipklim.org> References: <87v7zlo8yy.fsf@lipklim.org> <86y14ghmm1.fsf@gnu.org> <87ed677xji.fsf@lipklim.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.0.50 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 72849 Cc: 72849@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Evgenii Klimov wrote: > Eli Zaretskii writes: > > >> Date: Wed, 28 Aug 2024 00:13:25 +0100 > >> From: Evgenii Klimov via "Bug reports for GNU Emacs, > >> the Swiss army knife of text editors" > >> > >> `with-temp-buffer' doesn't respect buffer-local environment variables, > >> `exec-path' in this case. Which results in executables not being found, > >> or the wrong versions of executables being picked up. E.g. if > >> environment variable is modified via .dir-local file or direnv/envrc > >> package. > > > > Hmm, this doesn't look clean to me: exec-path is just one variable, > > what makes it special here? > > > > Moreover, it sounds like python-shell-with-environment, which > > python-shell-prompt-detect calls, already attempts to have > > buffer-local value of exec-path to be available to Python, so why > > isn't that working for you? And if it isn't work, I think we should > > amend python-shell-with-environment to do this, so we don't need to do > > it "by hand". > > Indeed, my initial approach is too manual. > > Here the problem that I have: I don't use Python's "venv" module to > create virtual environment for the project. Instead, I use GNU Guix's > "guix shell" command [1] which provides augmented PATH and PYTHONPATH, > etc. to link project's dependencies. Then, envrc.el package picks up > these environment variables and makes them buffer-local project-wise > (`exec-path' and `process-environment'). > > You are correct that `python-shell-with-environment' provides > buffer-local variables, but `with-temp-buffer' treats `exec-path' and > `process-environment' variables very specially. > > I didn't find this behavior in documentation, but look at this example: > > (setq-default exec-path (list "global" "list")) > (setq-local exec-path (cons "local" > (default-value 'exec-path))) > (setq-default myvar (list "global" "list")) > (setq-local myvar (cons "local" (default-value 'myvar))) > > (let ((exec-path exec-path) ; takes buffer-local > (myvar myvar)) ; takes buffer-local > (with-temp-buffer > (insert (car exec-path) ; uses global > "\n" > (car myvar)) ; uses `let'-binded > (buffer-string))) > ;; => "global > ;; local" > > (require 'cl-lib) > (let ((myvar myvar)) > ;; temporarily binds buffer-local value to global symbol > (cl-letf (((default-value 'exec-path) exec-path)) > (with-temp-buffer > ;; global variable is used, but it's value is temporarily equal > ;; to buffer-local value > (insert (car exec-path) > "\n" > (car myvar)) > (buffer-string)))) > ;; => "local > ;; local" > > It's a simplified and expanded version of > `python-shell-with-environment' and `python-shell-prompt-detect'. As > you can see, `exec-path' is treated differently inside of > `with-temp-buffer' and `cl-letf' is needed to force `with-temp-buffer' > to use buffer-local value of `exec-path'. > > In the new patch attached I show how this can be overcome. Don't know > if you'll consider my use case too narrow and specific, but I'll be glad > to hear your thoughts on this. > > [1] https://guix.gnu.org/en/manual/en/guix.html#Invoking-guix-environment Thank you for your explanation and the new patch. It helped me to understand the problem correctly. I also confirmed that the new patch resolves the issue. I don't think it's a special use case, as I sometimes set `exec-path' locally using .dir-locals.el. I agree with the new patch. From debbugs-submit-bounces@debbugs.gnu.org Sat Aug 31 06:19:06 2024 Received: (at 72849-done) by debbugs.gnu.org; 31 Aug 2024 10:19:06 +0000 Received: from localhost ([127.0.0.1]:54017 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1skLCD-0001mx-Oq for submit@debbugs.gnu.org; Sat, 31 Aug 2024 06:19:06 -0400 Received: from eggs.gnu.org ([209.51.188.92]:58474) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1skLCB-0001mQ-Sl for 72849-done@debbugs.gnu.org; Sat, 31 Aug 2024 06:19:04 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1skLBA-00028y-Ks; Sat, 31 Aug 2024 06:18:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=MLQ+fj/B74vpoFdB5V5EpajHITtYxKbHVzuuMMx8Hz0=; b=WJLUzPkBWSg0 7An5HGNbsaVwiA3MEZB/YvmQ38OEgFO0kccKJKx/q0QpADvXSNVP1EJ2AwkR1PWECCHV8R8trhVFi I3We0OAlRhcpM5l2aVK4g2lC9lcpgj/iJZuQ/UcGnabgRdZhoYn81jj3YQ3CFRRT6gPRml3YPoOf+ H87Q6dFXI6KZuGDCjAXcQ+jmgu8JSaJcFqZ/6kLUjw2z4c13Yvs27xyp/AJHaG6KGUCUh4EKjiaj+ YaLoqoKFj/L4VXaeL/gFvcHr2QMUqJLHLhaK9FgWgKWYObM0wQb3NrEVu6QQyoyxSFg8KR9xhzzOn XtTyeU63IF3sB3gCs67QaQ==; Date: Sat, 31 Aug 2024 13:17:58 +0300 Message-Id: <86r0a5atd5.fsf@gnu.org> From: Eli Zaretskii To: kobarity In-Reply-To: (message from kobarity on Fri, 30 Aug 2024 23:43:57 +0900) Subject: Re: bug#72849: [PATCH] Keep project's exec-path during with-temp-buffer call References: <87v7zlo8yy.fsf@lipklim.org> <86y14ghmm1.fsf@gnu.org> <87ed677xji.fsf@lipklim.org> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 72849-done Cc: 72849-done@debbugs.gnu.org, eugene.dev@lipklim.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > Date: Fri, 30 Aug 2024 23:43:57 +0900 > From: kobarity > Cc: 72849@debbugs.gnu.org > > Evgenii Klimov wrote: > > Eli Zaretskii writes: > > > > >> Date: Wed, 28 Aug 2024 00:13:25 +0100 > > >> From: Evgenii Klimov via "Bug reports for GNU Emacs, > > >> the Swiss army knife of text editors" > > >> > > >> `with-temp-buffer' doesn't respect buffer-local environment variables, > > >> `exec-path' in this case. Which results in executables not being found, > > >> or the wrong versions of executables being picked up. E.g. if > > >> environment variable is modified via .dir-local file or direnv/envrc > > >> package. > > > > > > Hmm, this doesn't look clean to me: exec-path is just one variable, > > > what makes it special here? > > > > > > Moreover, it sounds like python-shell-with-environment, which > > > python-shell-prompt-detect calls, already attempts to have > > > buffer-local value of exec-path to be available to Python, so why > > > isn't that working for you? And if it isn't work, I think we should > > > amend python-shell-with-environment to do this, so we don't need to do > > > it "by hand". > > > > Indeed, my initial approach is too manual. > > > > Here the problem that I have: I don't use Python's "venv" module to > > create virtual environment for the project. Instead, I use GNU Guix's > > "guix shell" command [1] which provides augmented PATH and PYTHONPATH, > > etc. to link project's dependencies. Then, envrc.el package picks up > > these environment variables and makes them buffer-local project-wise > > (`exec-path' and `process-environment'). > > > > You are correct that `python-shell-with-environment' provides > > buffer-local variables, but `with-temp-buffer' treats `exec-path' and > > `process-environment' variables very specially. > > > > I didn't find this behavior in documentation, but look at this example: > > > > (setq-default exec-path (list "global" "list")) > > (setq-local exec-path (cons "local" > > (default-value 'exec-path))) > > (setq-default myvar (list "global" "list")) > > (setq-local myvar (cons "local" (default-value 'myvar))) > > > > (let ((exec-path exec-path) ; takes buffer-local > > (myvar myvar)) ; takes buffer-local > > (with-temp-buffer > > (insert (car exec-path) ; uses global > > "\n" > > (car myvar)) ; uses `let'-binded > > (buffer-string))) > > ;; => "global > > ;; local" > > > > (require 'cl-lib) > > (let ((myvar myvar)) > > ;; temporarily binds buffer-local value to global symbol > > (cl-letf (((default-value 'exec-path) exec-path)) > > (with-temp-buffer > > ;; global variable is used, but it's value is temporarily equal > > ;; to buffer-local value > > (insert (car exec-path) > > "\n" > > (car myvar)) > > (buffer-string)))) > > ;; => "local > > ;; local" > > > > It's a simplified and expanded version of > > `python-shell-with-environment' and `python-shell-prompt-detect'. As > > you can see, `exec-path' is treated differently inside of > > `with-temp-buffer' and `cl-letf' is needed to force `with-temp-buffer' > > to use buffer-local value of `exec-path'. > > > > In the new patch attached I show how this can be overcome. Don't know > > if you'll consider my use case too narrow and specific, but I'll be glad > > to hear your thoughts on this. > > > > [1] https://guix.gnu.org/en/manual/en/guix.html#Invoking-guix-environment > > Thank you for your explanation and the new patch. It helped me to > understand the problem correctly. I also confirmed that the new patch > resolves the issue. I don't think it's a special use case, as I > sometimes set `exec-path' locally using .dir-locals.el. I agree with > the new patch. Thanks to both of you. I've now installed this on the emacs-30 release branch, and I'm therefore closing this bug. From unknown Fri Jun 20 07:29:16 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 28 Sep 2024 11:24:07 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator