GNU bug report logs - #10980
23.4; Variable init_environment incorrectly set

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 10980 in the body.
You can then email your comments to 10980 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: "Bo Johansson" <bo.johansson <at> lsn.se>
To: <bug-gnu-emacs <at> gnu.org>
Subject: 23.4; Variable init_environment incorrectly set
Date: Fri, 9 Mar 2012 12:01:53 +0100

[Message part 1 (text/plain, inline)]

Hej!

--------------------------- Background
I am used to "live" in Emacs and want to get the same result doing something from within Emacs as doing it from a Windows command prompt.
For example (compile "dmake" nil) in Emacs and executing dmake in a Windows command window should give the same result.

--------------------------- Example of Problems
Emacs replaces the environment variable SHELL at start up in Windows to something like "C:/Program Files (x86)/GNU Emacs 23.4/bin/cmdproxy.exe". Processes started by Emacs will inherit this value.

Many utilities uses SHELL. One example is:  With " ... cmdproxy.exe" as value of SHELL dmake is configured to use a Kornshell compatible shell.

--------------------------- Solution ideas
A long time goal should be to avoid changing the environment variable within Emacs. Emacs should (internally) at least not change the value of SHELL.

A short time solution is to make the initial environment variable available as stated in http://www.gnu.org/software/emacs/manual/html_node/emacs/Environment.html: "The variable initial-environment stores the initial environment inherited by Emacs".

This make it possible in
emacs-git/lisp/progmodes/compile.el:1479
(defun compilation-start (command &optional mode name-function highlight-regexp)
to use the variable compilation-environment to temporary override the in emacs internal used environment variables.

In emacs-git/lisp/progmodes/simple.el:2088
(defun shell-command (command &optional output-buffer error-buffer)
the corresponding possibility is however not available.

--------------------------- Description of the Bug
Below follows a list of steps done at start up. The problem is that step 2) changes the environment variables before step 4) sets the variable initial_environment.

1) The function init_environment is called

emacs-git/src/emacs.c:1416
#ifdef MSDOS
  /* Call early 'cause init_environment needs it.  */
  init_dosfns ();
  /* Set defaults for several environment variables.  */
  if (initialized)
    init_environment (argc, argv, skip_args);
  else
    tzset ();
#endif /* MSDOS */

#ifdef WINDOWSNT
  globals_of_w32 ();
  /* Initialize environment from registry settings.  */
  init_environment (argv);
  init_ntproc ();    /* must precede init_editfns.  */
#endif

2) The function init_environment changes the environment variables

T:/emacs-git/src/w32.c:1528:init_environment (char ** argv)
Do a lot of changes to the environment variables.

3) The function set_initial_environment is called to save the "initial_environment"

emacs-git/src/emacs.c:1440
  /* Initialize and GC-protect Vinitial_environment and
     Vprocess_environment before set_initial_environment fills them
     in.  */
  if (!initialized)
    syms_of_callproc ();
  /* egetenv is a pretty low-level facility, which may get called in
     many circumstances; it seems flimsy to put off initializing it
     until calling init_callproc.  Do not do it when dumping.  */
  if (initialized || ((strcmp (argv[argc-1], "dump") != 0
               && strcmp (argv[argc-1], "bootstrap") != 0)))
    set_initial_environment ();

4) The variable initial_environment is set by set_initial_environment

emacs-git/src/callproc.c:1616
set_initial_environment (void)
emacs-git/src/callproc.c:1624
Vinitial_environment = Fcopy_sequence (Vprocess_environment);


-----------------------------------------------------
In GNU Emacs 23.4.1 (i386-mingw-nt6.1.7601)
of 2012-02-04 on MARVIN
Windowing system distributor `Microsoft Corp.', version 6.1.7601
configured using `configure --with-gcc (4.4) --cflags -ID:/devel/emacs/libs/libXpm-3.5.8/include -ID:/devel/emacs/libs/libXpm-3.5.8/src -ID:/devel/emacs/libs/libpng-dev_1.4.3-1/include -ID:/devel/emacs/libs/zlib-dev_1.2.5-2/include -ID:/devel/emacs/libs/giflib-4.1.4-1/include -ID:/devel/emacs/libs/jpeg-6b-4/include -ID:/devel/emacs/libs/tiff-3.8.2-1/include'

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: SVE
  value of $XMODIFIERS: nil
  locale-coding-system: cp1252
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  diff-auto-refine-mode: t
  shell-dirtrack-mode: t
  delete-selection-mode: t
  show-paren-mode: t
  cua-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-fill-mode: t

Recent input:
<escape> x r e p o <tab> r <tab> <return>

Recent messages:
Loading cua-base...done
Loading paren...done
userDotEmacs Done
Finding all versions of R on your system...
Sorry, no version of R could be found on your system.
Making completion list...

Load-path shadows:
None found.

Features:
(shadow sort mail-extr message ecomplete rfc822 mml mml-sec
password-cache mm-decode mm-bodies mm-encode mailcap mail-parse rfc2231
rfc2047 rfc2045 qp ietf-drums mailabbrev nnheader gnus-util netrc
time-date mm-util mail-prsvr gmm-utils mailheader canlock sha1 hex-util
hashcash mail-utils emacsbug help-mode view psvn byte-opt warnings
bytecomp byte-compile derived edmacro kmacro wid-edit log-edit pcvs-util
add-log diff-mode easy-mmode pp elp ediff-merg ediff-diff ediff-wind
ediff-mult ediff-help ediff-init ediff-util dired tex-mik preview-latex
tex-site auto-loads ess-eldoc ess-toolbar ess-mouse mouseme thingatpt
browse-url ess-menu ess-swv ess-noweb noweb-font-lock-mode ess-bugs-l
essd-els ess-sas-d ess-sas-l ess-sas-a executable shell ess-arc-d
ess-vst-d ess-xls-d ess-lsp-l ess-sta-d ess-sta-l cc-vars cc-defs
make-regexp ess-sp6w-d ess-sp4-d ess-sp3-d ess-r-d ess-r-args assoc
ess-s-l ess-inf ess-utils comint ring ess-mode noweb-mode ess ess-custom
regexp-opt ess-compat ess-site htmlize-view easymenu w32-winprint
htmlize cl cl-19 delsel server advice help-fns advice-preload paren
cua-base cus-start cus-load tooltip ediff-hook vc-hooks lisp-float-type
mwheel dos-w32 disp-table ls-lisp w32-win w32-vars tool-bar dnd fontset
image fringe lisp-mode register page menu-bar rfn-eshadow timer select
scroll-bar mldrag mouse jit-lock font-lock syntax facemenu font-core
frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev loaddefs button minibuffer faces cus-face files
text-properties overlay md5 base64 format env code-pages mule custom
widget hashtable-print-readable backquote make-network-process multi-tty
emacs)

[Message part 2 (text/html, inline)]

Message #8 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: "Bo Johansson" <bo.johansson <at> lsn.se>
To: <10980 <at> debbugs.gnu.org>
Subject: bug#10980: 23.4; Variable initial-environment incorrectly set
Date: Fri, 23 Mar 2012 07:30:42 +0100

[Message part 1 (text/plain, inline)]

In the mail topic is “init_environment” corrected to  “initial-environment”.

Summary of bug:

On Windows the Emacs variable initial-environment is NOT set according to the documentation:
"The variable initial-environment stores the initial environment inherited by Emacs".

[Message part 2 (text/html, inline)]

Message #11 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Bo Johansson <bo.johansson <at> lsn.se>
Cc: 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: 23.4; Variable initial-environment incorrectly set
Date: Fri, 23 Mar 2012 18:06:57 +0200

> From: "Bo Johansson" <bo.johansson <at> lsn.se>
> Date: Fri, 23 Mar 2012 07:30:42 +0100
> 
> On Windows the Emacs variable initial-environment is NOT set according to the documentation:
> "The variable initial-environment stores the initial environment inherited by Emacs".

I'm sorry, I'm not sure I understand the essence of your report.  What
exactly would you like the documentation to say that it doesn't say
now?  Or, to put it differently, what in the way initial-environment
is set on Windows contradicts the documentation?

Thanks.

Message #14 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: "Bo Johansson" <bo.johansson <at> lsn.se>
To: "Eli Zaretskii" <eliz <at> gnu.org>
Cc: 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: 23.4; Variable initial-environment incorrectly set
Date: Fri, 23 Mar 2012 21:20:23 +0100

>From: Eli Zaretskii
>Date:     Fri, 23 Mar 2012 18:06:57 +0200
>> From: "Bo Johansson" <address <at> hidden>
>> Date: Fri, 23 Mar 2012 07:30:42 +0100
>>
>> On Windows the Emacs variable initial-environment is NOT set according to 
>> the
>> documentation:
>> "The variable initial-environment stores the initial environment 
>> inherited by
>> Emacs".
>
>I'm sorry, I'm not sure I understand the essence of your report.  What
>exactly would you like the documentation to say that it doesn't say
>now?  Or, to put it differently, what in the way initial-environment
>is set on Windows contradicts the documentation?
>
>Thanks.

Hej Eli!

My goal is:
I want to get the same result doing something from within Emacs as doing it 
from a Windows command prompt. To do "(compile "make" nil)" within Emacs 
should give the same result as doing "make" from the Windows command prompt.

The problem is:
The environment variables inherited by Emacs is changed. The changes in the 
environment leads to diffren results doing something from within Emacs 
compared to doing it from a Windows command prompt.

The BUG in Emacs is:
In Windows the variable initial-environment is set first after "the initial 
environment inherited by Emacs" is changed. The function init_environment 
(char ** argv) (in emacs-git/src/w32.c at line 1528)  changes the 
environment variables. The CHANGED "initial environment" is stored in the 
variable initial_environment by the function set_initial_environment (in 
emacs-git/src/callproc.c at line 1616).
However the documentation says: "The variable initial-environment stores the 
initial environment inherited by Emacs".

My goal can be achieved if the bug is corrected:
A correct value in the variable initial-environment is need to be able to 
know "the environment inherited by Emacs". This make it possible to use 
initial-environment in the lisp function (defun compilation-start (command 
&optional mode name-function highlight-regexp) in 
emacs-git/lisp/progmodes/simple.el:2088. The in emacs internal used 
environment variables can temporary be override by those in the variable 
initial-environment.
(It is of course also possible to use a new variable which contains "the 
initial environment inherited by Emacs".)

Summary: A value in the variable initial-environment according to the 
documentation makes it possible to "get the same result doing something from 
within Emacs as doing it from a Windows command prompt".

(See also 
http://lists.gnu.org/archive/html/bug-gnu-emacs/2012-03/msg00178.html.)

Best regards
Bosse 

Message #17 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Bo Johansson <bo.johansson <at> lsn.se>
Cc: 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: 23.4; Variable initial-environment incorrectly set
Date: Fri, 23 Mar 2012 23:05:31 +0200

> From: "Bo Johansson" <bo.johansson <at> lsn.se>
> Cc: <10980 <at> debbugs.gnu.org>
> Date: Fri, 23 Mar 2012 21:20:23 +0100
> 
> Summary: A value in the variable initial-environment according to the 
> documentation makes it possible to "get the same result doing something from 
> within Emacs as doing it from a Windows command prompt".
> 
> (See also 
> http://lists.gnu.org/archive/html/bug-gnu-emacs/2012-03/msg00178.html.)

Is this only about SHELL, and only with dmake, or are there other
variables that get in the way of other programs?  Please be specific;
the few changes done to the variables outside Emacs are done for a
good reason, so it's important to know the exact adverse effects of
that to judge how justified would it be to change the current code.

Message #20 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: "Bo Johansson" <bo.johansson <at> lsn.se>
To: "Eli Zaretskii" <eliz <at> gnu.org>
Cc: 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: 23.4; Variable initial-environment incorrectly set
Date: Sat, 24 Mar 2012 00:26:06 +0100

>From: Eli Zaretskii
>Sent: Friday, March 23, 2012 10:05 PM
>> From: "Bo Johansson" <bo.johansson <at> lsn.se>
>> Cc: <10980 <at> debbugs.gnu.org>
>> Date: Fri, 23 Mar 2012 21:20:23 +0100
>>
>> Summary: A value in the variable initial-environment according to the
>> documentation makes it possible to "get the same result doing something 
>> from
>> within Emacs as doing it from a Windows command prompt".
>>
>> (See also
>> http://lists.gnu.org/archive/html/bug-gnu-emacs/2012-03/msg00178.html.)
>
>Is this only about SHELL, and only with dmake, or are there other
>variables that get in the way of other programs?  Please be specific;
>the few changes done to the variables outside Emacs are done for a
>good reason, so it's important to know the exact adverse effects of
>that to judge how justified would it be to change the current code.

It is not only about SHELL. But SHELL is the most frequent problem.
Dmake and other make programs is just one among many other cases.

I searched internet for "emacs" and "warning: extra args ignored after". 
There was +1400 hits.
Many of theese depends on that SHELL is set to .../cmdproxy.exe. 
Adding -dmake to the search, resulted in 1280 hits. This indicates that 
dmake is only one case among others.

In the function compilation-start (lisp/progmodes/compile.el:1166) is the 
variable process-environment changed. This is done TEMPORARY by using "(let 
((process-environment ...".  However, in many other places are "persistent" 
changes made by using the lisp function setenv. This is in many cases not 
good. Activating a function can then result in an unexpected "persistent" 
change of the environment.

The last 20 years I have seen many unexpected things using "(compile 
"something" nil)", but it is first lately I have understood that one of the 
reasons are the changed environment.

At least the functions compile and shell-command should use an as far as 
possible unmodified "initial environment inherited by Emacs". I am missing a 
strategy how to handle the environment within Emacs.

I will try to find examples of other variables then SHELL!

Message #23 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: "Bo Johansson" <bo.johansson <at> lsn.se>
To: "Eli Zaretskii" <eliz <at> gnu.org>
Cc: 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: 23.4; Variable initial-environment incorrectly set
Date: Wed, 28 Mar 2012 10:56:44 +0200

>From: Eli Zaretskii
>Sent: Friday, March 23, 2012 10:05 PM
>> From: "Bo Johansson" <bo.johansson <at> lsn.se>
>> Cc: <10980 <at> debbugs.gnu.org>
>> Date: Fri, 23 Mar 2012 21:20:23 +0100
>>
>> Summary: A value in the variable initial-environment according to the
>> documentation makes it possible to "get the same result doing something 
>> from
>> within Emacs as doing it from a Windows command prompt".
>>
>Is this only about SHELL, and only with dmake, or are there other
>variables that get in the way of other programs?  Please be specific;
>the few changes done to the variables outside Emacs are done for a
>good reason, so it's important to know the exact adverse effects of
>that to judge how justified would it be to change the current code.

Hej Eli!

Here is some more information.

I have looked in makefile using http://code.google.com/codesearch. SHELL and 
COMSPEC are frequently used. In some cases is even ComSpec used.
To set SHELL to c:/Program Files (x86)/GNU Emacs 23.4/bin/cmdproxy.exe 
causes a lot of problems.
The resulting errors are often difficult to find. This partly because the 
warning messages given by cmdproxy.exe is often given without any context.

A program developed to run in Windows, expects a “Windows environment”!?

I have compared the environment variables in different situations. The 
differences between the environment in Windows cmd.exe and the Emacs 
variable initial-environment are:

-ComSpec=C:\Windows\system32\cmd.exe
+COMSPEC=C:\Windows\system32\cmd.exe
+EMACSDATA=c:/Program Files (x86)/GNU Emacs 23.4/etc
+EMACSDOC=c:/Program Files (x86)/GNU Emacs 23.4/etc
+EMACSLOADPATH=c:/Program Files (x86)/GNU Emacs 23.4/site-lisp;c:/Program 
Files (x86)/GNU Emacs 23.4/../site-lisp;c:/Program Files (x86)/GNU Emacs 
23.4/lisp;c:/Program Files (x86)/GNU Emacs 23.4/leim
+EMACSPATH=c:/Program Files (x86)/GNU Emacs 23.4/bin
+LANG=SVE
-Path=...
+PATH=...
+SHELL=c:/Program Files (x86)/GNU Emacs 23.4/bin/cmdproxy.exe
+TMPDIR=C:\Users\bo\AppData\Local\Temp
+emacs_dir=c:/Program Files (x86)/GNU Emacs 23.4

Comspec and Path are renamed. SHELL is added and set to the cmdproxy.exe.

Differences between initial-environment to shell-command environment are:

+EM_PARENT_PROCESS_ID=...


Differences initial-environment to compile environment are:

+DISPLAY=
+EMACS=t
+EM_PARENT_PROCESS_ID=...
+INSIDE_EMACS=t
-TERM=dumb
+TERM=emacs
+TERMCAP=emacs:co#80:tc=unknown:

This comment in emacs-git/src/w32.c:1778:
/* Another special case: on NT, the PATH variable is actually named
    "Path" although cmd.exe (perhaps NT itself) arranges for
    environment variable lookup and setting to be case insensitive.
    However, Emacs assumes a fully case sensitive environment, so we
    need to change "Path" to "PATH" to match the expectations of
    various elisp packages.  We do this by the sneaky method of
    modifying the string in the C runtime environ entry.

    The same applies to COMSPEC.  */
indicates that changes to the environment are mainly? made for Emacs, NOT 
for the processes started by Emacs. Emacs should separate the "internal 
environment" used internally by Emacs and the external used to set up 
external processes!?

My idea to get a read-only "inherited environment" is:
1) To save the "inherited environment" early in "c-code" at start up of 
Emacs
2) Implement a lisp function which can return the saved "inherited 
environment".

The new read-only "inherited environment" can then later be used to start 
external processes with a more "transparent" environment.
To start to change the current handling of the variable initial-environment 
is probably difficult and error prone.

Message #26 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: 10980 <at> debbugs.gnu.org
Cc: bo.johansson <at> lsn.se
Subject: Re: GNU bugs information: logs for bug#10980
Date: Tue, 7 Jun 2016 23:54:17 -0400

severity 10980 wishlist
quit

> From: "Bo Johansson" <bo.johansson <at> lsn.se>
> To: "Eli Zaretskii" <eliz <at> gnu.org>
> Cc: 10980 <at> debbugs.gnu.org
> Subject: Re: bug#10980: 23.4; Variable initial-environment incorrectly set
> Date: Wed, 28 Mar 2012 10:56:44 +0200
>
> My idea to get a read-only "inherited environment" is:
> 1) To save the "inherited environment" early in "c-code" at start up of
> Emacs
> 2) Implement a lisp function which can return the saved "inherited
> environment".
>
> The new read-only "inherited environment" can then later be used to start
> external processes with a more "transparent" environment.
> To start to change the current handling of the variable initial-environment
> is probably difficult and error prone.
>

I read this as a feature request to let lisp programs be able to see
the environment from before Emacs startup routines have changed it. I
have an additional use case for this: in magit it would be useful to
see if the user has HOME or if they just let Emacs choose a default
value for it.  In the latter case, I would pop up a warning to tell
them not to do that because git chooses a different default value
HOME, and having disagreement causes confusion (of the "why does X
work from command line and not in magit?" variety).

Message #31 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Wed, 08 Jun 2016 19:40:24 +0300

> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Date: Tue, 7 Jun 2016 23:54:17 -0400
> Cc: bo.johansson <at> lsn.se
> 
> > My idea to get a read-only "inherited environment" is:
> > 1) To save the "inherited environment" early in "c-code" at start up of
> > Emacs
> > 2) Implement a lisp function which can return the saved "inherited
> > environment".
> >
> > The new read-only "inherited environment" can then later be used to start
> > external processes with a more "transparent" environment.
> > To start to change the current handling of the variable initial-environment
> > is probably difficult and error prone.
> >
> 
> I read this as a feature request to let lisp programs be able to see
> the environment from before Emacs startup routines have changed it.

I don't think we want to have environment-related functions that are
specific to Windows, that goes against the goal of portability of
Emacs packages.

I'm okay with considering patches specific to w32 that would eliminate
the need for pushing variables into the environment of subprocesses,
and/or leave initial-environment unaffected by the pushed values, but
no one has stepped forward for the job.

> I have an additional use case for this: in magit it would be useful
> to see if the user has HOME or if they just let Emacs choose a
> default value for it.  In the latter case, I would pop up a warning
> to tell them not to do that because git chooses a different default
> value HOME, and having disagreement causes confusion (of the "why
> does X work from command line and not in magit?" variety).

I guess you refer to the fact that msysgit uses $USERPROFILE as the
alternative home directory if $HOME is not set?  If so, I'd rather
suggest to report a bug to msysgit maintainers: they are behaving
against platform recommendations.  From
https://msdn.microsoft.com/en-us/library/windows/desktop/bb762494%28v=vs.85%29.aspx:

  CSIDL_PROFILE
  FOLDERID_Profile

    The user's profile folder. A typical path is
    C:\Users\username. Applications should not create files or folders
    at this level; they should put their data under the locations
    referred to by CSIDL_APPDATA or CSIDL_LOCAL_APPDATA. However, if you
    are creating a new Known Folder the profile root referred to by
    CSIDL_PROFILE is appropriate.

If you look in the $USERPROFILE directory on a typical Windows
machine, you won't see there any sub-directory or file created by an
application, only a few standard sub-directories.  Applications do
generally follow the above recommendations; for example, I have
Firefox installed, which keeps my customizations in
$APPDATA/Mozilla/Firefox/Profiles/.  So Git is the odd one out if it
puts its ~/.gitconfig file in $USERPROFILE.

Message #34 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Mon, 20 Jun 2016 20:23:26 -0400

[Message part 1 (text/plain, inline)]

On Wed, Jun 8, 2016 at 12:40 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> I don't think we want to have environment-related functions that are
> specific to Windows, that goes against the goal of portability of
> Emacs packages.
>
> I'm okay with considering patches specific to w32 that would eliminate
> the need for pushing variables into the environment of subprocesses,
> and/or leave initial-environment unaffected by the pushed values

How about splitting apart initialization of Vinitial_environment and
Vprocess_environment and moving the former earlier so that it's
unaffected by Emacs' manipulations of the environment? See attached
patch.

> I guess you refer to the fact that msysgit uses $USERPROFILE as the
> alternative home directory if $HOME is not set?  If so, I'd rather
> suggest to report a bug to msysgit maintainers: they are behaving
> against platform recommendations.
[...]
> If you look in the $USERPROFILE directory on a typical Windows
> machine, you won't see there any sub-directory or file created by an
> application, only a few standard sub-directories.  Applications do
> generally follow the above recommendations; for example, I have
> Firefox installed, which keeps my customizations in
> $APPDATA/Mozilla/Firefox/Profiles/.  So Git is the odd one out if it
> puts its ~/.gitconfig file in $USERPROFILE.

To me it makes sense to have $HOME map to $USERPROFILE, and $APPDATA
is like $XDG_CONFIG_HOME (usually ~/.config/ on GNU/Linux). However,
this is purely subjective as there are no platform recommendations
about translating environment variables to other platforms.

[0001-Set-Vinitial_environment-before-changing-env-vars.patch (application/octet-stream, attachment)]

Message #37 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Tue, 21 Jun 2016 16:27:54 +0300

> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Date: Mon, 20 Jun 2016 20:23:26 -0400
> Cc: 10980 <at> debbugs.gnu.org, bo.johansson <at> lsn.se
> 
> > I'm okay with considering patches specific to w32 that would eliminate
> > the need for pushing variables into the environment of subprocesses,
> > and/or leave initial-environment unaffected by the pushed values
> 
> How about splitting apart initialization of Vinitial_environment and
> Vprocess_environment and moving the former earlier so that it's
> unaffected by Emacs' manipulations of the environment? See attached
> patch.

Thanks.  However, I wonder if we could do better.  First, your patch
only fixed initial-environment, which means Lisp applications will
need to explicitly use it, and probably only on Windows, something
that is not the best solution, IMO.  I hoped we could come up with a
way of pushing the additional variables into Emacs's own environment
after Vprocess_environment is already computed -- can you try doing
that?

In any case, the reasons for calling the same function twice in two
different places should be explained, at least in the comments, or
else someone might become confused at some future point in time.
Better yet, perhaps only the Windows build should do something like
that, and the other platforms could continue using the current code
mostly unaltered, as they don't need this.

> > If you look in the $USERPROFILE directory on a typical Windows
> > machine, you won't see there any sub-directory or file created by an
> > application, only a few standard sub-directories.  Applications do
> > generally follow the above recommendations; for example, I have
> > Firefox installed, which keeps my customizations in
> > $APPDATA/Mozilla/Firefox/Profiles/.  So Git is the odd one out if it
> > puts its ~/.gitconfig file in $USERPROFILE.
> 
> To me it makes sense to have $HOME map to $USERPROFILE, and $APPDATA
> is like $XDG_CONFIG_HOME (usually ~/.config/ on GNU/Linux). However,
> this is purely subjective as there are no platform recommendations
> about translating environment variables to other platforms.

I quoted the platform recommendations that discourage putting files
directly under $USERPROFILE, and most programs, including those ported
from Posix platforms, do seem to follow those recommendations.

Message #40 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Tue, 21 Jun 2016 17:03:21 -0400

On Tue, Jun 21, 2016 at 9:27 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> How about splitting apart initialization of Vinitial_environment and
>> Vprocess_environment and moving the former earlier so that it's
>> unaffected by Emacs' manipulations of the environment? See attached
>> patch.
>
> Thanks.  However, I wonder if we could do better.  First, your patch
> only fixed initial-environment, which means Lisp applications will
> need to explicitly use it, and probably only on Windows,

Well, Lisp applications that want the environment as Emacs originally
received it will use initial-environment regardless of platform
(without the patch, on Windows, they get an environment with some
modifications from Emacs).

> that is not the best solution, IMO.  I hoped we could come up with a
> way of pushing the additional variables into Emacs's own environment
> after Vprocess_environment is already computed -- can you try doing
> that?

I feel like I'm missing some important point here. If these
environment variables won't affect subprocess environments, why set
them at all?

>
> In any case, the reasons for calling the same function twice in two
> different places should be explained, at least in the comments, or
> else someone might become confused at some future point in time.

Right.

> Better yet, perhaps only the Windows build should do something like
> that, and the other platforms could continue using the current code
> mostly unaltered, as they don't need this.

Isn't it simpler to do the same thing on all platforms? They don't
need a different approach, but it doesn't hurt either.

Message #43 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Wed, 22 Jun 2016 05:39:14 +0300

> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Date: Tue, 21 Jun 2016 17:03:21 -0400
> Cc: 10980 <at> debbugs.gnu.org, bo.johansson <at> lsn.se
> 
> I feel like I'm missing some important point here. If these
> environment variables won't affect subprocess environments, why set
> them at all?

Because Emacs itself needs them.  They must be in Emacs's environment,
but don't have to be in Vprocess_environment.

Message #46 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Tue, 21 Jun 2016 22:54:25 -0400

On Tue, Jun 21, 2016 at 10:39 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
>> Date: Tue, 21 Jun 2016 17:03:21 -0400
>> Cc: 10980 <at> debbugs.gnu.org, bo.johansson <at> lsn.se
>>
>> I feel like I'm missing some important point here. If these
>> environment variables won't affect subprocess environments, why set
>> them at all?
>
> Because Emacs itself needs them.  They must be in Emacs's environment,
> but don't have to be in Vprocess_environment.

What does it need them for? Just because Emacs consults the values
from its environment? Is there some reason not to use just plain old
global variables?

Message #49 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Wed, 22 Jun 2016 17:57:30 +0300

> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Date: Tue, 21 Jun 2016 22:54:25 -0400
> Cc: 10980 <at> debbugs.gnu.org, bo.johansson <at> lsn.se
> 
> On Tue, Jun 21, 2016 at 10:39 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> >> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> >> Date: Tue, 21 Jun 2016 17:03:21 -0400
> >> Cc: 10980 <at> debbugs.gnu.org, bo.johansson <at> lsn.se
> >>
> >> I feel like I'm missing some important point here. If these
> >> environment variables won't affect subprocess environments, why set
> >> them at all?
> >
> > Because Emacs itself needs them.  They must be in Emacs's environment,
> > but don't have to be in Vprocess_environment.
> 
> What does it need them for?

Various features in Emacs rely on them to be present and valid
(because that's what happens on Posix systems).

> Just because Emacs consults the values from its environment?

Yes.

> Is there some reason not to use just plain old global variables?

On all platforms, or just on Windows?

In any case, doing that would mean a much larger job, even if it's
possible.  E.g., how do you deal with Lisp code that expects
(expand-file-name "~") and (getenv "HOME") to yield the same value?

Message #52 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Wed, 29 Jun 2016 09:12:39 -0400

On Wed, Jun 22, 2016 at 10:57 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> In any case, doing that would mean a much larger job, even if it's
> possible.  E.g., how do you deal with Lisp code that expects
> (expand-file-name "~") and (getenv "HOME") to yield the same value?

Hmm, so it's easy enough to move setting of both Vprocess_environment
and Vinitial_environment before the Windows code starts adding to the
environment. And getenv would have to be modified to consult Emacs'
environment so that that (expand-file-name "~") and (getenv "HOME")
give the same values.

But it seems we would then need 2 sets of functions: getenv/setenv and
get-subproc-env/set-subproc-env (the latter working on
Vprocess_environment only). This feels like a complication with not
much benefit.

Message #55 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Wed, 29 Jun 2016 18:15:44 +0300

> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Date: Wed, 29 Jun 2016 09:12:39 -0400
> Cc: 10980 <at> debbugs.gnu.org, bo.johansson <at> lsn.se
> 
> On Wed, Jun 22, 2016 at 10:57 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > In any case, doing that would mean a much larger job, even if it's
> > possible.  E.g., how do you deal with Lisp code that expects
> > (expand-file-name "~") and (getenv "HOME") to yield the same value?
> 
> Hmm, so it's easy enough to move setting of both Vprocess_environment
> and Vinitial_environment before the Windows code starts adding to the
> environment.

I'd actually suggest doing it the other way around: move the
Windows-specific code in w32.c that pushes these variables into the
environment after Vprocess_environment and Vinitial_environment were
already computed.  That way, we are sure the only affected platform is
w32.  (Order of initialization at startup matters, so best not to rock
the boat there, unless we absolutely have to.)

> And getenv would have to be modified to consult Emacs'
> environment so that that (expand-file-name "~") and (getenv "HOME")
> give the same values.

C 'getenv' or Lisp 'getenv'?

> But it seems we would then need 2 sets of functions: getenv/setenv and
> get-subproc-env/set-subproc-env (the latter working on
> Vprocess_environment only). This feels like a complication with not
> much benefit.

We already have that: there's 'getenv' and 'egetenv' on the C level.
And the Lisp 'setenv' is already documented to modify
process-environment.  So I'm not sure I see the problem (although it's
clear that getenv_internal_1 will probably need some Windows specific
changes.)

Message #58 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Wed, 29 Jun 2016 11:26:04 -0400

On Wed, Jun 29, 2016 at 11:15 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
>> And getenv would have to be modified to consult Emacs'
>> environment so that that (expand-file-name "~") and (getenv "HOME")
>> give the same values.
>
> C 'getenv' or Lisp 'getenv'?

Sorry, I meant Lisp getenv.

>> But it seems we would then need 2 sets of functions: getenv/setenv and
>> get-subproc-env/set-subproc-env (the latter working on
>> Vprocess_environment only). This feels like a complication with not
>> much benefit.
>
> We already have that: there's 'getenv' and 'egetenv' on the C level.
> And the Lisp 'setenv' is already documented to modify
> process-environment.  So I'm not sure I see the problem

What about a lisp program that expects (expand-file-name "~") and
(getenv "HOME") to give the same results after running (setenv "HOME"
<whatever>)

Message #61 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Wed, 29 Jun 2016 18:34:00 +0300

> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Date: Wed, 29 Jun 2016 11:26:04 -0400
> Cc: 10980 <at> debbugs.gnu.org, bo.johansson <at> lsn.se
> 
> >> But it seems we would then need 2 sets of functions: getenv/setenv and
> >> get-subproc-env/set-subproc-env (the latter working on
> >> Vprocess_environment only). This feels like a complication with not
> >> much benefit.
> >
> > We already have that: there's 'getenv' and 'egetenv' on the C level.
> > And the Lisp 'setenv' is already documented to modify
> > process-environment.  So I'm not sure I see the problem
> 
> What about a lisp program that expects (expand-file-name "~") and
> (getenv "HOME") to give the same results after running (setenv "HOME"
> <whatever>)

The changes in getenv_internal_1 (or, rather, in getenv_internal) that
I mentioned will have to make that happen.  It already does something
similar.

Message #64 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Wed, 29 Jun 2016 19:02:25 -0400

[Message part 1 (text/plain, inline)]

On Wed, Jun 29, 2016 at 11:34 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
>> Date: Wed, 29 Jun 2016 11:26:04 -0400
>> Cc: 10980 <at> debbugs.gnu.org, bo.johansson <at> lsn.se
>>
>> >> But it seems we would then need 2 sets of functions: getenv/setenv and
>> >> get-subproc-env/set-subproc-env (the latter working on
>> >> Vprocess_environment only). This feels like a complication with not
>> >> much benefit.
>> >
>> > We already have that: there's 'getenv' and 'egetenv' on the C level.
>> > And the Lisp 'setenv' is already documented to modify
>> > process-environment.  So I'm not sure I see the problem
>>
>> What about a lisp program that expects (expand-file-name "~") and
>> (getenv "HOME") to give the same results after running (setenv "HOME"
>> <whatever>)
>
> The changes in getenv_internal_1 (or, rather, in getenv_internal) that
> I mentioned will have to make that happen.  It already does something
> similar.

Ah, I think I missed the idea that changing Vprocess_environment would
still affect Emacs' environment (technically, what Emacs thinks its
environment is, which has the same effect). Patch attached.

[v2-0001-Keep-w32-environment-settings-internal-only.patch (application/octet-stream, attachment)]

Message #69 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Noam Postavsky <npostavs <at> users.sourceforge.net>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Sat, 09 Jul 2016 13:57:33 +0300

> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Date: Wed, 29 Jun 2016 19:02:25 -0400
> Cc: 10980 <at> debbugs.gnu.org, bo.johansson <at> lsn.se
> 
> Ah, I think I missed the idea that changing Vprocess_environment would
> still affect Emacs' environment (technically, what Emacs thinks its
> environment is, which has the same effect). Patch attached.

Thanks, please push to master, after fixing the following minor
issues:

> * src/emacs.c (main): For WINDOWSNT platform, move init_environment
> calls after the set_initial_environment call.  This prevents Emacs'
> modifications to the environment from contaminating Vprocess_environment
> and Vinitial_environment (Bug #10980).

Code changes in fragments guarded by preprocessor conditionals should
be formatted like this:

  * src/emacs.c (main) [WINDOWSNT]: Move init_environment calls after ...

> * src/callproc.c (getenv_internal): Consult Emacs' internal environment
> in as a fallback to Vprocess_environment on WINDOWSNT platforms.

Same here.

> +        *valuelen = strlen(tmpval);
                       ^^^^^^^
Please leave a single blank between a function's name and the
following opening parenthesis.

> +  /* Initialize environment from registry settings.  Make sure to do
> +     this only after calling set_initial_environment so that
> +     Vinitial_environment and Vprocess_environment will contain only
> +     variables from the parent process without modifications from
> +     Emacs. */

Please leave 2 blanks between the last period of the comment text and
the comment terminator "*/".

Bonus points for adding a test for this issue.

Message #72 received at 10980 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> users.sourceforge.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: bo.johansson <at> lsn.se, 10980 <at> debbugs.gnu.org
Subject: Re: bug#10980: GNU bugs information: logs for bug#10980
Date: Mon, 18 Jul 2016 17:52:43 -0400

close 10980 25.2
quit

On Sat, Jul 9, 2016 at 6:57 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> Thanks, please push to master, after fixing the following minor
> issues:
[...]
> Bonus points for adding a test for this issue.

Done and done, 73f0715d

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.