From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 03 13:46:36 2013 Received: (at submit) by debbugs.gnu.org; 3 Sep 2013 17:46:36 +0000 Received: from localhost ([127.0.0.1]:39390 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VGugW-0004ig-21 for submit@debbugs.gnu.org; Tue, 03 Sep 2013 13:46:36 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:60204 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VGugT-0004iW-MC for submit@debbugs.gnu.org; Tue, 03 Sep 2013 13:46:34 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1VGugS-0006Pn-RA; Tue, 03 Sep 2013 13:46:32 -0400 From: Glenn Morris To: submit@debbugs.gnu.org Subject: cannot build in a directory with non-ascii characters X-Spook: Syria arrangements kilderkin Capricorn SHA Waco, Texas X-Ran: "Q".jGu)^%KF?tRg=E>:*L;%0M.{I@1PBlHjgRk{^W<9Daa#(eu$ga-P(XZ*LT|4GVJ|0e X-Hue: yellow X-Debbugs-No-Ack: yes X-Attribution: GM Date: Tue, 03 Sep 2013 13:46:32 -0400 Message-ID: User-Agent: Gnus (www.gnus.org), GNU Emacs (www.gnu.org/software/emacs/) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Score: -6.7 (------) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -6.7 (------) Package: emacs Severity: important Version: 24.3 It seems Emacs (still) cannot be built in a directory whose name contains non-ascii characters. Ref: http://lists.gnu.org/archive/html/help-gnu-emacs/2013-09/msg00033.html If it cannot be made to work, configure should abort with an error in such cases. I have some vague memory that it also might not work with spaces in the names, but did not test. Similar restrictions may apply to the install --prefix as well. From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 21 13:36:26 2013 Received: (at control) by debbugs.gnu.org; 21 Oct 2013 17:36:26 +0000 Received: from localhost ([127.0.0.1]:32909 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VYJP0-0005zs-BC for submit@debbugs.gnu.org; Mon, 21 Oct 2013 13:36:26 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:52551 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VYJOy-0005zk-Sz for control@debbugs.gnu.org; Mon, 21 Oct 2013 13:36:25 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1VYJOy-0000cV-AG for control@debbugs.gnu.org; Mon, 21 Oct 2013 13:36:24 -0400 Date: Mon, 21 Oct 2013 13:36:24 -0400 Message-Id: Subject: control message for bug 15675 To: X-Mailer: mail (GNU Mailutils 2.1) From: Glenn Morris X-Spam-Score: -5.4 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.4 (-----) forcemerge 15260 15675 From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 21 21:14:02 2013 Received: (at control) by debbugs.gnu.org; 22 Oct 2013 01:14:02 +0000 Received: from localhost ([127.0.0.1]:33778 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VYQXq-0006wv-0t for submit@debbugs.gnu.org; Mon, 21 Oct 2013 21:14:02 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:58351 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VYQXm-0006wO-Tp for control@debbugs.gnu.org; Mon, 21 Oct 2013 21:13:59 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1VYQXm-0006NR-3m for control@debbugs.gnu.org; Mon, 21 Oct 2013 21:13:58 -0400 Date: Mon, 21 Oct 2013 21:13:58 -0400 Message-Id: Subject: control message for bug 15675 To: X-Mailer: mail (GNU Mailutils 2.1) From: Glenn Morris X-Spam-Score: -5.4 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.4 (-----) unarchive 13078 unmerge 15675 From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 21 23:14:16 2013 Received: (at control) by debbugs.gnu.org; 22 Oct 2013 03:14:16 +0000 Received: from localhost ([127.0.0.1]:33991 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VYSQC-0002b9-0J for submit@debbugs.gnu.org; Mon, 21 Oct 2013 23:14:16 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:59385 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VYSQ8-0002b0-Me for control@debbugs.gnu.org; Mon, 21 Oct 2013 23:14:13 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1VYSQ8-0005PV-80 for control@debbugs.gnu.org; Mon, 21 Oct 2013 23:14:12 -0400 Date: Mon, 21 Oct 2013 23:14:12 -0400 Message-Id: Subject: control message for bug 15675 To: X-Mailer: mail (GNU Mailutils 2.1) From: Glenn Morris X-Spam-Score: -5.4 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.4 (-----) forcemerge 15260 15675 From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 23 02:23:41 2013 Received: (at control) by debbugs.gnu.org; 23 Oct 2013 06:23:42 +0000 Received: from localhost ([127.0.0.1]:36746 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VYrr2-0001rO-SE for submit@debbugs.gnu.org; Wed, 23 Oct 2013 02:23:41 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:52896 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VYrr0-0001rF-FB for control@debbugs.gnu.org; Wed, 23 Oct 2013 02:23:38 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1VYrqz-000780-S8 for control@debbugs.gnu.org; Wed, 23 Oct 2013 02:23:38 -0400 Date: Wed, 23 Oct 2013 02:23:37 -0400 Message-Id: Subject: control message for bug 15675 To: X-Mailer: mail (GNU Mailutils 2.1) From: Glenn Morris X-Spam-Score: -5.4 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.4 (-----) unmerge 15675 retitle 15675 building in directory with spaces in name From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 23 16:48:45 2013 Received: (at 15260) by debbugs.gnu.org; 23 Oct 2013 20:48:46 +0000 Received: from localhost ([127.0.0.1]:38842 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZ5MD-0001o0-G6 for submit@debbugs.gnu.org; Wed, 23 Oct 2013 16:48:45 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:38673 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZ5MB-0001ns-2Q for 15260@debbugs.gnu.org; Wed, 23 Oct 2013 16:48:43 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1VZ5MA-0004By-73; Wed, 23 Oct 2013 16:48:42 -0400 From: Glenn Morris To: 15260@debbugs.gnu.org Subject: Re: bug#15260: cannot build in a directory with non-ascii characters References: X-Spook: Aladdin Sundevil bank broadside Venezuela Dateline asset X-Ran: !s_#kq*yPx!T[=:C4ImHzO)-\?R\j*Q>dY3-<4'|HA2(v"^.(ugI_I}IcAgqZ(4]zUWg\} X-Hue: green X-Debbugs-No-Ack: yes X-Attribution: GM Date: Wed, 23 Oct 2013 16:48:42 -0400 In-Reply-To: (Glenn Morris's message of "Tue, 03 Sep 2013 13:46:32 -0400") Message-ID: User-Agent: Gnus (www.gnus.org), GNU Emacs (www.gnu.org/software/emacs/) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Score: -5.4 (-----) X-Debbugs-Envelope-To: 15260 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.4 (-----) Glenn Morris wrote: > If it cannot be made to work, configure should abort with an error in > such cases. [non-ascii directories] Done. Leaving this open as a wishlist to make it work. > I have some vague memory that it also might not work with spaces in the > names, but did not test. This works now - http://debbugs.gnu.org/15675 . From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 23 16:48:57 2013 Received: (at control) by debbugs.gnu.org; 23 Oct 2013 20:48:57 +0000 Received: from localhost ([127.0.0.1]:38845 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZ5MM-0001oO-Vj for submit@debbugs.gnu.org; Wed, 23 Oct 2013 16:48:55 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:38675 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZ5MK-0001oF-8t for control@debbugs.gnu.org; Wed, 23 Oct 2013 16:48:52 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1VZ5MJ-0004Dn-R5 for control@debbugs.gnu.org; Wed, 23 Oct 2013 16:48:52 -0400 Date: Wed, 23 Oct 2013 16:48:51 -0400 Message-Id: Subject: control message for bug 15260 To: X-Mailer: mail (GNU Mailutils 2.1) From: Glenn Morris X-Spam-Score: -5.4 (-----) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.4 (-----) severity 15260 wishlist From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 24 14:25:55 2013 Received: (at 15260) by debbugs.gnu.org; 24 Oct 2013 18:25:56 +0000 Received: from localhost ([127.0.0.1]:41590 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZPbW-00067W-VN for submit@debbugs.gnu.org; Thu, 24 Oct 2013 14:25:55 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:39543) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZPbT-00067C-1a for 15260@debbugs.gnu.org; Thu, 24 Oct 2013 14:25:52 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MV600E00R65X800@a-mtaout20.012.net.il> for 15260@debbugs.gnu.org; Thu, 24 Oct 2013 21:25:44 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MV600E76R6VUA30@a-mtaout20.012.net.il>; Thu, 24 Oct 2013 21:25:44 +0300 (IDT) Date: Thu, 24 Oct 2013 21:25:32 +0300 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Glenn Morris Message-id: <83y55ih6g3.fsf@gnu.org> References: X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Glenn Morris > Date: Wed, 23 Oct 2013 16:48:42 -0400 > > Glenn Morris wrote: > > > If it cannot be made to work, configure should abort with an error in > > such cases. [non-ascii directories] > > Done. Leaving this open as a wishlist to make it work. dnl configure sets LC_ALL=C early on, so this range should work. case "$var" in *[[^\ -~]]*) AC_MSG_ERROR([Emacs cannot be built or installed in a directory whose name contains non-ASCII characters: $var]) ;; esac This is quite drastic. Do we understand what is the underlying technical reason for the build failures? The bug reports didn't give any explanations, only the fact that moving to a pure-ASCII directory fixed the problem. From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 24 14:35:19 2013 Received: (at 15260) by debbugs.gnu.org; 24 Oct 2013 18:35:19 +0000 Received: from localhost ([127.0.0.1]:41605 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZPkc-0006N8-KC for submit@debbugs.gnu.org; Thu, 24 Oct 2013 14:35:18 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:57032 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZPka-0006Mz-NV for 15260@debbugs.gnu.org; Thu, 24 Oct 2013 14:35:17 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1VZPkZ-0003is-F0; Thu, 24 Oct 2013 14:35:15 -0400 From: Glenn Morris To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters References: <83y55ih6g3.fsf@gnu.org> X-Spook: smuggle Maple ASIO Waco, Texas Operation Iraqi Freedom X-Ran: 2N$:"tB-ofI4W'a|E#Ve!n36Jw"qj,@AT.`cP%Me8kkpU%&Zie]-zoKA6{jXj7#KZ8YFU} X-Hue: green X-Debbugs-No-Ack: yes X-Attribution: GM Date: Thu, 24 Oct 2013 14:35:15 -0400 In-Reply-To: <83y55ih6g3.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 24 Oct 2013 21:25:32 +0300") Message-ID: <81ppqums9o.fsf@fencepost.gnu.org> User-Agent: Gnus (www.gnus.org), GNU Emacs (www.gnu.org/software/emacs/) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Score: -5.4 (-----) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.4 (-----) Eli Zaretskii wrote: > case "$var" in > *[[^\ -~]]*) AC_MSG_ERROR([Emacs cannot be built or installed in a directory whose name contains non-ASCII characters: $var]) ;; > esac > > This is quite drastic. I don't think so. The alternative is a cryptic failure during the build stage. > Do we understand what is the underlying technical reason for the > build failures? Something to do with failure to find files, just as it was 6 years ago. http://lists.gnu.org/archive/html/emacs-devel/2007-05/msg00984.html The immediate problem for me is a dump failure: Finding pointers to doc strings... Finding pointers to doc strings...done Dumping under the name emacs emacs: Can't open /path/to/non-ascii/src/temacs for reading: No such file or directory make[1]: *** [bootstrap-emacs] Error 1 Why not make a non-ASCII directory and try it yourself... From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 25 10:25:35 2013 Received: (at 15260) by debbugs.gnu.org; 25 Oct 2013 14:25:35 +0000 Received: from localhost ([127.0.0.1]:43523 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZiKS-0005ln-Dd for submit@debbugs.gnu.org; Fri, 25 Oct 2013 10:25:34 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:39360) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZiKO-0005lY-Kh for 15260@debbugs.gnu.org; Fri, 25 Oct 2013 10:25:29 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MV800M00AP0PG00@a-mtaout20.012.net.il> for 15260@debbugs.gnu.org; Fri, 25 Oct 2013 17:25:14 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MV800MCEAQ1N020@a-mtaout20.012.net.il>; Fri, 25 Oct 2013 17:25:14 +0300 (IDT) Date: Fri, 25 Oct 2013 17:25:05 +0300 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: <81ppqums9o.fsf@fencepost.gnu.org> X-012-Sender: halo1@inter.net.il To: Glenn Morris Message-id: <83hac5h1ha.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Glenn Morris > Cc: 15260@debbugs.gnu.org > Date: Thu, 24 Oct 2013 14:35:15 -0400 > > Eli Zaretskii wrote: > > > case "$var" in > > *[[^\ -~]]*) AC_MSG_ERROR([Emacs cannot be built or installed in a directory whose name contains non-ASCII characters: $var]) ;; > > esac > > > > This is quite drastic. > > I don't think so. The alternative is a cryptic failure during the build stage. > > > Do we understand what is the underlying technical reason for the > > build failures? > > Something to do with failure to find files, just as it was 6 years ago. > http://lists.gnu.org/archive/html/emacs-devel/2007-05/msg00984.html > > The immediate problem for me is a dump failure: > > Finding pointers to doc strings... > Finding pointers to doc strings...done > Dumping under the name emacs > emacs: Can't open /path/to/non-ascii/src/temacs for reading: No such file > or directory > make[1]: *** [bootstrap-emacs] Error 1 Does the change below help? > Why not make a non-ASCII directory and try it yourself... It requires too much setup on my part (this cannot be simulated on Windows without too much hassle). But I will do that if there's no easier way. I just thought that some analysis has been done already. === modified file 'src/emacs.c' --- src/emacs.c 2013-10-20 16:47:42 +0000 +++ src/emacs.c 2013-10-25 14:21:47 +0000 @@ -2044,11 +2044,15 @@ You must run Emacs in batch mode in orde CHECK_STRING (filename); filename = Fexpand_file_name (filename, Qnil); + filename = ENCODE_FILE (filename); if (!NILP (symfile)) { CHECK_STRING (symfile); if (SCHARS (symfile)) - symfile = Fexpand_file_name (symfile, Qnil); + { + symfile = Fexpand_file_name (symfile, Qnil); + symfile = ENCODE_FILE (symfile); + } } tem = Vpurify_flag; From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 25 13:08:17 2013 Received: (at 15260) by debbugs.gnu.org; 25 Oct 2013 17:08:17 +0000 Received: from localhost ([127.0.0.1]:43749 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZkrv-0005II-Ly for submit@debbugs.gnu.org; Fri, 25 Oct 2013 13:08:16 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:58810 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZkrt-0005IA-8B for 15260@debbugs.gnu.org; Fri, 25 Oct 2013 13:08:13 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1VZkrq-0001vc-4h; Fri, 25 Oct 2013 13:08:10 -0400 From: Glenn Morris To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> X-Spook: JSOFC3IP Qaddafi class struggle argus import pipeline X-Ran: ur8Nvg5I\l|8*H+aXXUFkZxB*ewzs@E~Wn_ZCL9p}ITwhG:.!~3,nqFK;;:8K2(W=g2%K* X-Hue: blue X-Debbugs-No-Ack: yes X-Attribution: GM Date: Fri, 25 Oct 2013 13:08:08 -0400 In-Reply-To: <83hac5h1ha.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 25 Oct 2013 17:25:05 +0300") Message-ID: User-Agent: Gnus (www.gnus.org), GNU Emacs (www.gnu.org/software/emacs/) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Score: -5.4 (-----) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.4 (-----) Eli Zaretskii wrote: > It requires too much setup on my part (this cannot be simulated on > Windows without too much hassle). Sorry, I assumed you could build on fencepost. I'm expecting multiple points of failure, so this might not be an efficient process... The first time, I was trying an out-of-tree build in a non-ascii build directory, but still with ascii srcdir. Using an in-place build in a non-ascii directory fails to even start temacs (this is with or without your patch): Warning: arch-independent data dir `/tmp/EMACS/share/emacs/24.3.50/etc/': No such file or directory Error: charsets directory not found: /tmp/EMACS/share/emacs/24.3.50/etc/charsets Emacs will not function correctly without the character map files. Please check your installation! make[1]: *** [bootstrap-emacs] Error 1 /tmp/EMACS was my install --prefix. It's not supposed to exist until after installation, but the code that tries to find etc/ is presumably mistakenly concluding that srcdir/etc does not exist and that it must be running installed. From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 25 14:31:28 2013 Received: (at 15260) by debbugs.gnu.org; 25 Oct 2013 18:31:28 +0000 Received: from localhost ([127.0.0.1]:43875 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZmAR-0007Ml-Io for submit@debbugs.gnu.org; Fri, 25 Oct 2013 14:31:27 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:52248) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZmAO-0007MU-Dy for 15260@debbugs.gnu.org; Fri, 25 Oct 2013 14:31:25 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MV800000LM9Z800@a-mtaout20.012.net.il> for 15260@debbugs.gnu.org; Fri, 25 Oct 2013 21:31:17 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MV80009FM45VG50@a-mtaout20.012.net.il>; Fri, 25 Oct 2013 21:31:17 +0300 (IDT) Date: Fri, 25 Oct 2013 21:31:09 +0300 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Glenn Morris Message-id: <83bo2dgq36.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Glenn Morris > Cc: 15260@debbugs.gnu.org > Date: Fri, 25 Oct 2013 13:08:08 -0400 > > Eli Zaretskii wrote: > > > It requires too much setup on my part (this cannot be simulated on > > Windows without too much hassle). > > Sorry, I assumed you could build on fencepost. That's the backup plan, yes. > I'm expecting multiple > points of failure, so this might not be an efficient process... I don't think we should do it this way. I was just asking about the current state of knowledge. I presume that the changes I suggested didn't help? (They are TRT anyway, so I will install them regardless.) > The first time, I was trying an out-of-tree build in a non-ascii build > directory, but still with ascii srcdir. Using an in-place build in a > non-ascii directory fails to even start temacs (this is with or without > your patch): > > Warning: arch-independent data dir > `/tmp/EMACS/share/emacs/24.3.50/etc/': No such file or directory > Error: charsets directory not found: > /tmp/EMACS/share/emacs/24.3.50/etc/charsets > Emacs will not function correctly without the character map files. > Please check your installation! > make[1]: *** [bootstrap-emacs] Error 1 > > /tmp/EMACS was my install --prefix. It's not supposed to exist until > after installation, but the code that tries to find etc/ is presumably > mistakenly concluding that srcdir/etc does not exist and that it must be > running installed. So in the above, /tmp/EMACS/share/emacs/24.3.50/etc/ is pure-ASCII, and the non-ASCII directory is in the source tree, is that right? From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 25 14:40:51 2013 Received: (at 15260) by debbugs.gnu.org; 25 Oct 2013 18:40:51 +0000 Received: from localhost ([127.0.0.1]:43901 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZmJW-0007dd-ET for submit@debbugs.gnu.org; Fri, 25 Oct 2013 14:40:50 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:60025 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZmJT-0007dV-E6 for 15260@debbugs.gnu.org; Fri, 25 Oct 2013 14:40:48 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1VZmJS-0006gk-Bs; Fri, 25 Oct 2013 14:40:46 -0400 From: Glenn Morris To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> X-Spook: cryptanalysis 9/11 Perl-RSA nitrate NSA United Nations X-Ran: W!&;<8}a+N^FL~Zlxb6Z (Eli Zaretskii's message of "Fri, 25 Oct 2013 21:31:09 +0300") Message-ID: User-Agent: Gnus (www.gnus.org), GNU Emacs (www.gnu.org/software/emacs/) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Score: -5.4 (-----) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.4 (-----) Eli Zaretskii wrote: > I presume that the changes I suggested didn't help? (They are TRT > anyway, so I will install them regardless.) It helps for "ascii srcdir, non-ascii builddir", but there are still problems later on, again related to Emacs mistakenly believing that certain directories do not exist, when they do (Warning: arch-dependent data dir `...' No such file or directory; etc). The "non-ascii srcdir == builddir" case fails even earlier, due to not finding etc. > So in the above, /tmp/EMACS/share/emacs/24.3.50/etc/ is pure-ASCII, > and the non-ASCII directory is in the source tree, is that right? Yes. I literally did (in a non-ascii) directory: ./configure --prefix=/tmp/EMACS From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 25 14:47:11 2013 Received: (at 15260) by debbugs.gnu.org; 25 Oct 2013 18:47:11 +0000 Received: from localhost ([127.0.0.1]:43910 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZmPe-0007ne-T4 for submit@debbugs.gnu.org; Fri, 25 Oct 2013 14:47:11 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:54973) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZmPb-0007nN-Up for 15260@debbugs.gnu.org; Fri, 25 Oct 2013 14:47:08 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MV800100M991K00@a-mtaout20.012.net.il> for 15260@debbugs.gnu.org; Fri, 25 Oct 2013 21:47:01 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MV80011RMUD2820@a-mtaout20.012.net.il>; Fri, 25 Oct 2013 21:47:01 +0300 (IDT) Date: Fri, 25 Oct 2013 21:46:52 +0300 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Glenn Morris Message-id: <837gd1gpcz.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Glenn Morris > Cc: 15260@debbugs.gnu.org > Date: Fri, 25 Oct 2013 14:40:46 -0400 > > Eli Zaretskii wrote: > > > I presume that the changes I suggested didn't help? (They are TRT > > anyway, so I will install them regardless.) > > It helps for "ascii srcdir, non-ascii builddir" Good, so one down, N - 1 to go ;-) > but there are still > problems later on, again related to Emacs mistakenly believing that > certain directories do not exist, when they do (Warning: arch-dependent > data dir `...' No such file or directory; etc). > > The "non-ascii srcdir == builddir" case fails even earlier, due to not > finding etc. OK, I will take a closer look. Thanks for the info. From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 25 15:27:42 2013 Received: (at 15260) by debbugs.gnu.org; 25 Oct 2013 19:27:42 +0000 Received: from localhost ([127.0.0.1]:43969 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZn2q-0000Lg-Q2 for submit@debbugs.gnu.org; Fri, 25 Oct 2013 15:27:42 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:49053) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZn2k-0000L9-IP for 15260@debbugs.gnu.org; Fri, 25 Oct 2013 15:27:37 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0MV800300OJBLR00@a-mtaout22.012.net.il> for 15260@debbugs.gnu.org; Fri, 25 Oct 2013 22:27:27 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MV800357OPRIE60@a-mtaout22.012.net.il>; Fri, 25 Oct 2013 22:27:27 +0300 (IDT) Date: Fri, 25 Oct 2013 22:27:19 +0300 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: <837gd1gpcz.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: rgm@gnu.org Message-id: <8361slgnhk.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Fri, 25 Oct 2013 21:46:52 +0300 > From: Eli Zaretskii > Cc: 15260@debbugs.gnu.org > > > but there are still > > problems later on, again related to Emacs mistakenly believing that > > certain directories do not exist, when they do (Warning: arch-dependent > > data dir `...' No such file or directory; etc). > > > > The "non-ascii srcdir == builddir" case fails even earlier, due to not > > finding etc. > > OK, I will take a closer look. Thanks for the info. I think I see the problem. All those PATH_* variables that come from epaths.h yield encoded file names (because they were written by the shell). But we never decode them before using them in init_callproc and init_callproc_1. Similar things happen with decode_env_path: it calls 'getenv', but never decodes the values it gets from that. I will take a crack on fixing these. From debbugs-submit-bounces@debbugs.gnu.org Sat Oct 26 03:50:24 2013 Received: (at 15260) by debbugs.gnu.org; 26 Oct 2013 07:50:24 +0000 Received: from localhost ([127.0.0.1]:44574 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZydb-0003jt-VM for submit@debbugs.gnu.org; Sat, 26 Oct 2013 03:50:24 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:48176) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VZydY-0003je-MC for 15260@debbugs.gnu.org; Sat, 26 Oct 2013 03:50:21 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MV900500N33XZ00@a-mtaout20.012.net.il> for 15260@debbugs.gnu.org; Sat, 26 Oct 2013 10:50:13 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MV90056YN3PJJ90@a-mtaout20.012.net.il>; Sat, 26 Oct 2013 10:50:13 +0300 (IDT) Date: Sat, 26 Oct 2013 10:50:06 +0300 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: <8361slgnhk.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: rgm@gnu.org, Stefan Monnier , Kenichi Handa Message-id: <83zjpwfp3l.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Fri, 25 Oct 2013 22:27:19 +0300 > From: Eli Zaretskii > Cc: 15260@debbugs.gnu.org > > > Date: Fri, 25 Oct 2013 21:46:52 +0300 > > From: Eli Zaretskii > > Cc: 15260@debbugs.gnu.org > > > > > but there are still > > > problems later on, again related to Emacs mistakenly believing that > > > certain directories do not exist, when they do (Warning: arch-dependent > > > data dir `...' No such file or directory; etc). > > > > > > The "non-ascii srcdir == builddir" case fails even earlier, due to not > > > finding etc. > > > > OK, I will take a closer look. Thanks for the info. > > I think I see the problem. All those PATH_* variables that come from > epaths.h yield encoded file names (because they were written by the > shell). But we never decode them before using them in init_callproc > and init_callproc_1. Similar things happen with decode_env_path: it > calls 'getenv', but never decodes the values it gets from that. > > I will take a crack on fixing these. We definitely need to decode file names in init_callproc_1, init_callproc, and init_lread. But here's where things get hairy: when temacs starts, preloaded Lisp files are not yet loaded, and consequently file-name-coding-system and default-file-name-coding-system are both nil. In such a case, currently DECODE_FILE is a no-op. So we need some way of getting temacs to know what coding-system to use to decode file names during its initialization phase, without relying on the database we have in locale-language-names. This probably calls for a separate variable, init-file-name-coding-system, say. But how to assign a correct value to it? I understand that most Posix systems nowadays use UTF-8 for file names, so I guess we can fall back on that. On MS-Windows, there's a system call that returns the necessary information, so there's no problem for MS-Windows. The question is what to do for Posix systems that don't use UTF-8? I see 2 possibilities: . Try to parse the value of LANG with some shell or Sed script, and come up with a suitable value. . Ask the user to specify the encoding as a switch to the configure script. In both cases, communicate the value to temacs via --eval on its command line. Comments and opinions are welcome. From debbugs-submit-bounces@debbugs.gnu.org Sat Oct 26 15:15:11 2013 Received: (at 15260) by debbugs.gnu.org; 26 Oct 2013 19:15:11 +0000 Received: from localhost ([127.0.0.1]:45938 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Va9KJ-000592-4Y for submit@debbugs.gnu.org; Sat, 26 Oct 2013 15:15:11 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:49030 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Va9KG-00058t-7b for 15260@debbugs.gnu.org; Sat, 26 Oct 2013 15:15:09 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1Va9KE-0004qg-C2; Sat, 26 Oct 2013 15:15:06 -0400 From: Glenn Morris To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> X-Spook: Sears Tower Firewalls $400 million in gold bullion X-Ran: UVw{Ir.ohS/.;&*Ia4KKT':T>C!.y3ac[|(zf?w&zbx8$x')UhN9e8KreCFS'/*o=<="._ X-Hue: yellow X-Debbugs-No-Ack: yes X-Attribution: GM Date: Sat, 26 Oct 2013 15:15:06 -0400 In-Reply-To: <83zjpwfp3l.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 26 Oct 2013 10:50:06 +0300") Message-ID: User-Agent: Gnus (www.gnus.org), GNU Emacs (www.gnu.org/software/emacs/) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Score: -5.4 (-----) X-Debbugs-Envelope-To: 15260 Cc: Kenichi Handa , Stefan Monnier , 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.4 (-----) Eli Zaretskii wrote: > Comments and opinions are welcome. Sounds like a fair bit of work, for something that doesn't seem very important. If my testing was correct, the problem only occurs during building, not after Emacs is installed (does that tally with what you found?). And I can't see any reason why anyone _needs_ to build Emacs in a directory with non-ASCII chars. From debbugs-submit-bounces@debbugs.gnu.org Sat Oct 26 16:05:07 2013 Received: (at 15260) by debbugs.gnu.org; 26 Oct 2013 20:05:07 +0000 Received: from localhost ([127.0.0.1]:46019 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaA6b-0006QI-Rd for submit@debbugs.gnu.org; Sat, 26 Oct 2013 16:05:06 -0400 Received: from mtaout23.012.net.il ([80.179.55.175]:56394) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaA6Y-0006Pe-HL for 15260@debbugs.gnu.org; Sat, 26 Oct 2013 16:05:03 -0400 Received: from conversion-daemon.a-mtaout23.012.net.il by a-mtaout23.012.net.il (HyperSendmail v2007.08) id <0MVA00M00KVVUG00@a-mtaout23.012.net.il> for 15260@debbugs.gnu.org; Sat, 26 Oct 2013 23:04:55 +0300 (IDT) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout23.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVA00M4YL47UU00@a-mtaout23.012.net.il>; Sat, 26 Oct 2013 23:04:55 +0300 (IDT) Date: Sat, 26 Oct 2013 23:04:49 +0300 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Glenn Morris Message-id: <83txg3g5ni.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, monnier@iro.umontreal.ca, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Glenn Morris > Cc: Stefan Monnier , Kenichi Handa , 15260@debbugs.gnu.org > Date: Sat, 26 Oct 2013 15:15:06 -0400 > > Sounds like a fair bit of work, for something that doesn't seem very > important. It might be important for people who build Emacs on non-English language systems. > If my testing was correct, the problem only occurs during > building, not after Emacs is installed (does that tally with what you > found?). It definitely happens when building. I didn't look deep enough to see what happens once Emacs is installed. The code is definitely wrong. > And I can't see any reason why anyone _needs_ to build Emacs in > a directory with non-ASCII chars. It might be a natural thing in some quarters. E.g., Emacs sources might be a subdirectory of some parent directory with a non-ASCII name where many other packages are built. Anyway, if the project thinks it's not important enough, I have better things to do. From debbugs-submit-bounces@debbugs.gnu.org Sat Oct 26 23:57:00 2013 Received: (at 15260) by debbugs.gnu.org; 27 Oct 2013 03:57:00 +0000 Received: from localhost ([127.0.0.1]:46821 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaHTI-0002Mc-5R for submit@debbugs.gnu.org; Sat, 26 Oct 2013 23:57:00 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:58589) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaHTF-0002MK-6C for 15260@debbugs.gnu.org; Sat, 26 Oct 2013 23:56:58 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0MVB00H006P51I00@a-mtaout22.012.net.il> for 15260@debbugs.gnu.org; Sun, 27 Oct 2013 05:56:50 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVB00GN66YPXH50@a-mtaout22.012.net.il>; Sun, 27 Oct 2013 05:56:49 +0200 (IST) Date: Sun, 27 Oct 2013 05:56:44 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: <83txg3g5ni.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: rgm@gnu.org Message-id: <83ob6bfjsz.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83txg3g5ni.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Sat, 26 Oct 2013 23:04:49 +0300 > From: Eli Zaretskii > Cc: 15260@debbugs.gnu.org > > > If my testing was correct, the problem only occurs during > > building, not after Emacs is installed (does that tally with what you > > found?). > > It definitely happens when building. I didn't look deep enough to see > what happens once Emacs is installed. The code is definitely wrong. Btw, are you sure the installed Emacs doesn't find the files under the source tree? Did you try to remove or rename it after installing? From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 27 00:28:46 2013 Received: (at 15260) by debbugs.gnu.org; 27 Oct 2013 04:28:46 +0000 Received: from localhost ([127.0.0.1]:46885 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaHy1-0003Be-KG for submit@debbugs.gnu.org; Sun, 27 Oct 2013 00:28:45 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:53613) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaHxz-0003BL-A6 for 15260@debbugs.gnu.org; Sun, 27 Oct 2013 00:28:43 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EABK/CFFLd/hu/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kiB4GwS2RCgOkeoFegxM X-IPAS-Result: Av4EABK/CFFLd/hu/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kiB4GwS2RCgOkeoFegxM X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="36440994" Received: from 75-119-248-110.dsl.teksavvy.com (HELO pastel.home) ([75.119.248.110]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 27 Oct 2013 00:28:37 -0400 Received: by pastel.home (Postfix, from userid 20848) id 5F35260038; Sun, 27 Oct 2013 00:28:36 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters Message-ID: References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> Date: Sun, 27 Oct 2013 00:28:36 -0400 In-Reply-To: <83zjpwfp3l.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 26 Oct 2013 10:50:06 +0300") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 15260 Cc: rgm@gnu.org, 15260@debbugs.gnu.org, Kenichi Handa X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) > But here's where things get hairy: when temacs starts, preloaded Lisp > files are not yet loaded, and consequently file-name-coding-system and > default-file-name-coding-system are both nil. In such a case, > currently DECODE_FILE is a no-op. I don't understand why it wouldn't work to just treat those strings as "binary" (i.e. keep them undecoded in unibyte strings). Then encoding would be a noop and that should hence end up in the right byte-sequence sent to the OS primitives. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 27 12:12:00 2013 Received: (at 15260) by debbugs.gnu.org; 27 Oct 2013 16:12:00 +0000 Received: from localhost ([127.0.0.1]:47627 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaSwa-0005sW-2m for submit@debbugs.gnu.org; Sun, 27 Oct 2013 12:12:00 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:59256) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaSwW-0005sH-L3 for 15260@debbugs.gnu.org; Sun, 27 Oct 2013 12:11:57 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0MVC002004YIHT00@a-mtaout22.012.net.il> for 15260@debbugs.gnu.org; Sun, 27 Oct 2013 18:11:49 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVC002LD4ZP2RB0@a-mtaout22.012.net.il>; Sun, 27 Oct 2013 18:11:49 +0200 (IST) Date: Sun, 27 Oct 2013 18:11:46 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Stefan Monnier Message-id: <83mwlug0cd.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: rgm@gnu.org, 15260@debbugs.gnu.org, handa@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Stefan Monnier > Cc: rgm@gnu.org, Kenichi Handa , 15260@debbugs.gnu.org > Date: Sun, 27 Oct 2013 00:28:36 -0400 > > I don't understand why it wouldn't work to just treat those strings as > "binary" (i.e. keep them undecoded in unibyte strings). Then encoding > would be a noop and that should hence end up in the right byte-sequence > sent to the OS primitives. Not sure I'm following you here. I presume you aren't asking why we generally hold file names in decoded form inside Emacs, nor suggesting that we switch to storing them as undecoded unibyte strings. So I guess you are asking why the particular piece of code being discussed here couldn't keep file names as unibyte strings, is that your question? If so, then the answer is "it could, but that would be even more hair." The problem is that the code involved in this (specifically, init_callproc_1, init_callproc, and probably also init_cmdargs and init_lread) is not something specifically written to stat the directories from epaths.h and announce their non-existence. That code populates important variables with names of files and directories and lists of directories that are henceforth used in Emacs all over the place. Notable examples are data-directory, doc-directory, exec-path, and load-path. Without populating these variables, temacs will not work, and the code which uses these variables assumes their values are decoded strings. The error messages are a by-product: as Emacs computes the values of these variables, it checks the files and directories for existence, and complains if they don't. The root cause is that unibyte strings get stored in variables used by Emacs on the assumption that they are decoded. Given the above, I'm not sure exactly what you are suggesting in practical terms. Can you elaborate? From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 27 12:19:40 2013 Received: (at 15260) by debbugs.gnu.org; 27 Oct 2013 16:19:40 +0000 Received: from localhost ([127.0.0.1]:47632 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaT40-000642-4p for submit@debbugs.gnu.org; Sun, 27 Oct 2013 12:19:40 -0400 Received: from mtaout21.012.net.il ([80.179.55.169]:55471) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaT3x-00063m-Ek for 15260@debbugs.gnu.org; Sun, 27 Oct 2013 12:19:38 -0400 Received: from conversion-daemon.a-mtaout21.012.net.il by a-mtaout21.012.net.il (HyperSendmail v2007.08) id <0MVC007004X6HP00@a-mtaout21.012.net.il> for 15260@debbugs.gnu.org; Sun, 27 Oct 2013 18:19:31 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout21.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVC007U35CICN70@a-mtaout21.012.net.il>; Sun, 27 Oct 2013 18:19:31 +0200 (IST) Date: Sun, 27 Oct 2013 18:19:27 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: <83ob6bfjsz.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: rgm@gnu.org Message-id: <83k3gyfzzk.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83txg3g5ni.fsf@gnu.org> <83ob6bfjsz.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Sun, 27 Oct 2013 05:56:44 +0200 > From: Eli Zaretskii > Cc: 15260@debbugs.gnu.org > > > Date: Sat, 26 Oct 2013 23:04:49 +0300 > > From: Eli Zaretskii > > Cc: 15260@debbugs.gnu.org > > > > > If my testing was correct, the problem only occurs during > > > building, not after Emacs is installed (does that tally with what you > > > found?). > > > > It definitely happens when building. I didn't look deep enough to see > > what happens once Emacs is installed. The code is definitely wrong. > > Btw, are you sure the installed Emacs doesn't find the files under the > source tree? Did you try to remove or rename it after installing? Further testing indicates that it indeed works to install in a non-ASCII directory after building. But it only barely works, at least in my testing: the various files and directories in doc-directory, load-path, etc. are unibyte strings, so using them only works if they are passed to file primitives. If you try to invoke a program with one of these values as a command-line argument, the program will fail (unless your locale encoding is identical to file-name encoding). And even using the unibyte strings in conjunction with files is fragile, as, for example, 'equal' will not compare unibyte and multibyte strings of the same bytes as equal. From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 27 15:03:04 2013 Received: (at 15260) by debbugs.gnu.org; 27 Oct 2013 19:03:04 +0000 Received: from localhost ([127.0.0.1]:47707 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaVc7-0001ae-QP for submit@debbugs.gnu.org; Sun, 27 Oct 2013 15:03:04 -0400 Received: from mtaout23.012.net.il ([80.179.55.175]:46326) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaVc5-0001a6-D3 for 15260@debbugs.gnu.org; Sun, 27 Oct 2013 15:03:02 -0400 Received: from conversion-daemon.a-mtaout23.012.net.il by a-mtaout23.012.net.il (HyperSendmail v2007.08) id <0MVC00500CNW2A00@a-mtaout23.012.net.il> for 15260@debbugs.gnu.org; Sun, 27 Oct 2013 21:02:54 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout23.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVC004XHCWUXY70@a-mtaout23.012.net.il>; Sun, 27 Oct 2013 21:02:54 +0200 (IST) Date: Sun, 27 Oct 2013 21:02:51 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: <83k3gyfzzk.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: rgm@gnu.org, Stefan Monnier , Kenichi Handa Message-id: <83eh76fsf8.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83txg3g5ni.fsf@gnu.org> <83ob6bfjsz.fsf@gnu.org> <83k3gyfzzk.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) The first few problems that pop up when building from the source tree whose parent has a non-ASCII name are solved by the changes below. I'm not very fond of these changes, especially the last one: it all looks very fragile and ad-hoc, and that's still on a system with a UTF-8 locale, where things should be relatively easy. After applying these changes, temacs comes up and dumps itself, but fails to find simple.el and bytecomp.el when it proceeds to compiling Lisp files. I guess now load-path is the culprit. Stay tuned. === modified file 'lisp/loadup.el' --- lisp/loadup.el 2013-10-08 15:11:29 +0000 +++ lisp/loadup.el 2013-10-27 18:26:12 +0000 @@ -150,7 +150,9 @@ (load "epa-hook") ;; Any Emacs Lisp source file (*.el) loaded here after can contain ;; multilingual text. -(load "international/mule-cmds") +(let ((dfn-coding default-file-name-coding-system)) + (load "international/mule-cmds") + (setq default-file-name-coding-system dfn-coding)) (load "case-table") ;; This file doesn't exist when building a development version of Emacs ;; from the repository. It is generated just after temacs is built. @@ -163,7 +165,9 @@ (load "language/cyrillic") (load "language/indian") (load "language/sinhala") -(load "language/english") +(let ((dfn-coding default-file-name-coding-system)) + (load "language/english") + (setq default-file-name-coding-system dfn-coding)) (load "language/ethiopic") (load "language/european") (load "language/czech") === modified file 'src/emacs.c' --- src/emacs.c 2013-10-26 13:43:58 +0000 +++ src/emacs.c 2013-10-27 18:48:51 +0000 @@ -2044,14 +2044,22 @@ You must run Emacs in batch mode in orde CHECK_STRING (filename); filename = Fexpand_file_name (filename, Qnil); - filename = ENCODE_FILE (filename); + if (NILP (Vfile_name_coding_system) + && NILP (Vdefault_file_name_coding_system)) + filename = Fstring_to_unibyte (filename); + else + filename = ENCODE_FILE (filename); if (!NILP (symfile)) { CHECK_STRING (symfile); if (SCHARS (symfile)) { symfile = Fexpand_file_name (symfile, Qnil); - symfile = ENCODE_FILE (symfile); + if (NILP (Vfile_name_coding_system) + && NILP (Vdefault_file_name_coding_system)) + symfile = Fstring_to_unibyte (symfile); + else + symfile = ENCODE_FILE (symfile); } } From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 27 15:43:24 2013 Received: (at 15260) by debbugs.gnu.org; 27 Oct 2013 19:43:24 +0000 Received: from localhost ([127.0.0.1]:47731 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaWF9-0002YS-Qm for submit@debbugs.gnu.org; Sun, 27 Oct 2013 15:43:23 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:42096) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaWF8-0002Y5-Dg for 15260@debbugs.gnu.org; Sun, 27 Oct 2013 15:43:23 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0MVC00400EPE5P00@a-mtaout22.012.net.il> for 15260@debbugs.gnu.org; Sun, 27 Oct 2013 21:43:15 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVC00366ES3QIA0@a-mtaout22.012.net.il>; Sun, 27 Oct 2013 21:43:15 +0200 (IST) Date: Sun, 27 Oct 2013 21:43:12 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: <83eh76fsf8.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: rgm@gnu.org, monnier@iro.umontreal.ca, handa@gnu.org Message-id: <83d2mqfqjz.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83txg3g5ni.fsf@gnu.org> <83ob6bfjsz.fsf@gnu.org> <83k3gyfzzk.fsf@gnu.org> <83eh76fsf8.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Sun, 27 Oct 2013 21:02:51 +0200 > From: Eli Zaretskii > Cc: 15260@debbugs.gnu.org > > After applying these changes, temacs comes up and dumps itself, but > fails to find simple.el and bytecomp.el when it proceeds to compiling > Lisp files. ^^^^^^^^^^^^^^^^ Instead of "it" I should have written "bootstrap-emacs". Sorry. From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 27 20:30:44 2013 Received: (at 15260) by debbugs.gnu.org; 28 Oct 2013 00:30:44 +0000 Received: from localhost ([127.0.0.1]:47923 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaajD-0001yK-Gl for submit@debbugs.gnu.org; Sun, 27 Oct 2013 20:30:43 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:62098) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vaaj8-0001n8-KF for 15260@debbugs.gnu.org; Sun, 27 Oct 2013 20:30:39 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EABK/CFFsr+u9/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kiB4GwS2RCgOkeoFegxM X-IPAS-Result: Av4EABK/CFFsr+u9/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kiB4GwS2RCgOkeoFegxM X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="36471562" Received: from 108-175-235-189.dsl.teksavvy.com (HELO pastel.home) ([108.175.235.189]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 27 Oct 2013 20:30:32 -0400 Received: by pastel.home (Postfix, from userid 20848) id 5360260036; Sun, 27 Oct 2013 20:30:32 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters Message-ID: References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> Date: Sun, 27 Oct 2013 20:30:32 -0400 In-Reply-To: <83mwlug0cd.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 27 Oct 2013 18:11:46 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 15260 Cc: rgm@gnu.org, 15260@debbugs.gnu.org, handa@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) > So I guess you are asking why the particular piece of code being > discussed here couldn't keep file names as unibyte strings, is that > your question? IIUC the issue is how to encode when we don't yet have the coding-systems loaded/setup. But it seems if we can't encode, then we can't decode either, so we should just fallback on using unibyte strings (which shouldn't be encoded on the way back to the OS) for those file names we create/manipulate before coding-systems are available. Stefan From debbugs-submit-bounces@debbugs.gnu.org Sun Oct 27 23:40:09 2013 Received: (at 15260) by debbugs.gnu.org; 28 Oct 2013 03:40:09 +0000 Received: from localhost ([127.0.0.1]:47998 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VadgW-0008C4-9h for submit@debbugs.gnu.org; Sun, 27 Oct 2013 23:40:08 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:38593) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VadgS-0008BQ-7M for 15260@debbugs.gnu.org; Sun, 27 Oct 2013 23:40:05 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MVD003000U21400@a-mtaout20.012.net.il> for 15260@debbugs.gnu.org; Mon, 28 Oct 2013 05:39:57 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVD0027B0UKT170@a-mtaout20.012.net.il>; Mon, 28 Oct 2013 05:39:57 +0200 (IST) Date: Mon, 28 Oct 2013 05:39:54 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Stefan Monnier Message-id: <83bo2af4hh.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: rgm@gnu.org, 15260@debbugs.gnu.org, handa@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Stefan Monnier > Cc: rgm@gnu.org, handa@gnu.org, 15260@debbugs.gnu.org > Date: Sun, 27 Oct 2013 20:30:32 -0400 > > > So I guess you are asking why the particular piece of code being > > discussed here couldn't keep file names as unibyte strings, is that > > your question? > > IIUC the issue is how to encode when we don't yet have the > coding-systems loaded/setup. But it seems if we can't encode, then we > can't decode either, so we should just fallback on using unibyte strings > (which shouldn't be encoded on the way back to the OS) for those file > names we create/manipulate before coding-systems are available. As I explained, this would be even more hair than what I proposed, because you are talking about core Emacs data structures and variables that are involved in every file-related op. On top of that, using unibyte strings is inherently fragile in Emacs, as the code is not written to support them too well, as you well know. We always advise users to stay away of unibyte strings, and for a good reason, so doing this ourselves sounds unwise. From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 28 00:05:40 2013 Received: (at 15260) by debbugs.gnu.org; 28 Oct 2013 04:05:40 +0000 Received: from localhost ([127.0.0.1]:48010 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vae5E-0000Ma-2u for submit@debbugs.gnu.org; Mon, 28 Oct 2013 00:05:40 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:13411) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vae5C-0000MN-HV for 15260@debbugs.gnu.org; Mon, 28 Oct 2013 00:05:39 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EABK/CFFsr+u9/2dsb2JhbAA7Cb8OF3OCHgEBBAFWIwULCzQSFBgNJIgeBsEtjRaDdAOkeoFegxM X-IPAS-Result: Av4EABK/CFFsr+u9/2dsb2JhbAA7Cb8OF3OCHgEBBAFWIwULCzQSFBgNJIgeBsEtjRaDdAOkeoFegxM X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="36483869" Received: from 108-175-235-189.dsl.teksavvy.com (HELO pastel.home) ([108.175.235.189]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 28 Oct 2013 00:05:32 -0400 Received: by pastel.home (Postfix, from userid 20848) id 5288560036; Mon, 28 Oct 2013 00:05:32 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters Message-ID: References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> Date: Mon, 28 Oct 2013 00:05:32 -0400 In-Reply-To: <83bo2af4hh.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 28 Oct 2013 05:39:54 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 15260 Cc: rgm@gnu.org, 15260@debbugs.gnu.org, handa@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) > As I explained, this would be even more hair than what I proposed, > because you are talking about core Emacs data structures and variables > that are involved in every file-related op. > On top of that, using unibyte strings is inherently fragile in Emacs, > as the code is not written to support them too well, as you well > know. We always advise users to stay away of unibyte strings, and for > a good reason, so doing this ourselves sounds unwise. I know, but I'm not sure why it doesn't "just work". More specifically, for the bug to appear, you need ENCODE (DECODE (s)) to not be the identity function. Why is not so in the "early" Emacs? Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 28 12:47:54 2013 Received: (at 15260) by debbugs.gnu.org; 28 Oct 2013 16:47:54 +0000 Received: from localhost ([127.0.0.1]:49504 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vapyr-0005jg-OX for submit@debbugs.gnu.org; Mon, 28 Oct 2013 12:47:54 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:37368) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vapym-0005jH-Nv for 15260@debbugs.gnu.org; Mon, 28 Oct 2013 12:47:50 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0MVE00F00158SM00@a-mtaout22.012.net.il> for 15260@debbugs.gnu.org; Mon, 28 Oct 2013 18:47:35 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVE00FAT1B8PS50@a-mtaout22.012.net.il>; Mon, 28 Oct 2013 18:47:33 +0200 (IST) Date: Mon, 28 Oct 2013 18:47:32 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Stefan Monnier Message-id: <8361shfil7.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: rgm@gnu.org, 15260@debbugs.gnu.org, handa@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Stefan Monnier > Cc: rgm@gnu.org, handa@gnu.org, 15260@debbugs.gnu.org > Date: Mon, 28 Oct 2013 00:05:32 -0400 > > More specifically, for the bug to appear, you need ENCODE (DECODE (s)) > to not be the identity function. Why is not so in the "early" Emacs? Because life's a mess that doesn't easily fit into simple and elegant schemes ;-) For starters, we don't really DECODE_FILE with these file- and directory-names. We just use build_string or make_string, as you can easily see in the init_* functions I mentioned. If you are lucky and your file names are UTF-8 encoded, this produces the same result as DECODE_FILE. If you are less lucky, and your file names are encoded in something else, like Latin-N, you get a unibyte string with the same bytes as in the original. Then we pass these strings to various functions, like file_accessible_directory_p, that _do_ ENCODE_FILE... (Luckily, during most of temacs's run, both file-name-coding-system and its default value are nil, so ENCODE_FILE is a no-op -- except when they aren't, see the next paragraph.) Next, it is quite possible that the file-name-coding-system changes between the time we process and store the file name and the time we encode and pass it to a low-level function. This is especially true during "loadup", when many packages are loaded and their top-level forms are executed. It turns out that 2 of them have side effects that do just that: mule-cmds.el calls reset-language-environment, and language/english.el calls set-language-info-alist; both have the effect of resetting default-file-name-coding-system to latin-1 (!? an interesting "default" for a Unicode-era Emacs, perhaps Handa-san could comment why we still do that). When this happens, your symmetry is broken, and ENCODE_FILE (DECODE_FILE (f)) is no longer the identity function. And then there are other players in this game. For example, default-directory, which is used every time we call expand-file-name, IOW "a lot". If you look in init_buffer, you will see that the default-directory of *scratch* is first set to a multibyte representation of the unibyte string we get from getcwd. In a "normal" Emacs session, we promptly fix that in startup.el, after the call to set-locale-environment initializes all the coding-systems. But "temacs -l loadup dump" doesn't run startup.el, so we are left with what init_buffer did, which is a string no file-name API will be able to grok. Another example is the use of 'equal' (and 'member', which calls 'equal') to compare file and directory names, and look them up in lists: as you know, 'equal' will not compare a unibyte and a multibyte string as equal. So having a mix of unibyte and multibyte strings in file names fails some of the code that relies on 'equal', tricking it into doing wrong things, like deciding that Emacs is _not_ run from the source tree. I'm sure there's more to this saga, I'm just half-way through it... From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 28 14:33:23 2013 Received: (at 15260) by debbugs.gnu.org; 28 Oct 2013 18:33:23 +0000 Received: from localhost ([127.0.0.1]:49640 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Varcx-0008Mk-5w for submit@debbugs.gnu.org; Mon, 28 Oct 2013 14:33:23 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:41665) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Varcu-0008MW-Sw for 15260@debbugs.gnu.org; Mon, 28 Oct 2013 14:33:22 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MVE00B00670KT00@a-mtaout20.012.net.il> for 15260@debbugs.gnu.org; Mon, 28 Oct 2013 20:33:08 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVE00B4S677C4A0@a-mtaout20.012.net.il>; Mon, 28 Oct 2013 20:33:08 +0200 (IST) Date: Mon, 28 Oct 2013 20:33:07 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: <8361shfil7.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: monnier@iro.umontreal.ca, Kenichi Handa Message-id: <834n81fdp8.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 8BIT References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Mon, 28 Oct 2013 18:47:32 +0200 > From: Eli Zaretskii > Cc: 15260@debbugs.gnu.org > > I'm sure there's more to this saga, I'm just half-way through it... The next round is here: # The actual Emacs command run in the targets below. emacs = EMACSLOADPATH="$(abs_lisp)" LC_ALL=C "$(EMACS)" $(EMACSOPT) ^^^^^^^^ Does anyone know or remember why we set LC_ALL=C while running commands in lisp/ (and the same in leim/)? The following log entry in lisp/ChangeLog.13 is the only clue: 2008-02-01 Kenichi Handa * Makefile.in: Be sure to run emacs with LC_ALL=C. But there's no explanation as to why this is needed. What this does is prevent bootstrap-emacs from finding Lisp files, because LC_ALL=C implies -- you guessed it -- file-name encoding by Latin-1, whereas the file names are really encoded in UTF-8 on this system: cd ../lisp; make -w compile-first EMACS="/home/e/eliz/bzr/emacs/xáéçö/src/bootstrap-emacs" make[2]: Entering directory `/srv/data/home/e/eliz/bzr/emacs/xáéçö/lisp' Compiling emacs-lisp/macroexp.el Warning: Could not find simple.el or simple.elc The EMACSLOADPATH environment variable is set, please check its value Lisp directory /home/e/eliz/bzr/emacs/x/lisp not readable? If I remove the LC_ALL=C setting from lisp/Makefile.in and leim/Makefile.in, I get past this problem (to the next one ;-). So: any reasons not to remove this setting from lisp/Makefile.in and leim/Makefile.in? From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 28 18:00:33 2013 Received: (at 15260) by debbugs.gnu.org; 28 Oct 2013 22:00:33 +0000 Received: from localhost ([127.0.0.1]:49976 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaurQ-0006eA-Lb for submit@debbugs.gnu.org; Mon, 28 Oct 2013 18:00:33 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:53624 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VaurO-0006at-3f for 15260@debbugs.gnu.org; Mon, 28 Oct 2013 18:00:30 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1VaurK-0003gT-0D; Mon, 28 Oct 2013 18:00:26 -0400 From: Glenn Morris To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> X-Spook: Guantanamo Semtex Putin Crowell JFK INS Al-Qaeda AMEMB X-Ran: xzMJa5.PXu{WC^ User-Agent: Gnus (www.gnus.org), GNU Emacs (www.gnu.org/software/emacs/) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 15260 Cc: Kenichi Handa , monnier@iro.umontreal.ca, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) Eli Zaretskii wrote: > emacs = EMACSLOADPATH="$(abs_lisp)" LC_ALL=C "$(EMACS)" $(EMACSOPT) > ^^^^^^^^ > > Does anyone know or remember why we set LC_ALL=C while running > commands in lisp/ (and the same in leim/)? FWIW, if I change that to use LC_ALL=en_US.UTF8 and bootstrap (after also fixing cl--gensym-counter to a non-random default), all the resulting *.elc files are identical to the LC=ALL=C case. From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 28 21:35:10 2013 Received: (at 15260) by debbugs.gnu.org; 29 Oct 2013 01:35:10 +0000 Received: from localhost ([127.0.0.1]:50260 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VayD7-0006B8-AY for submit@debbugs.gnu.org; Mon, 28 Oct 2013 21:35:09 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:52856) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VayD4-0006Ah-RZ for 15260@debbugs.gnu.org; Mon, 28 Oct 2013 21:35:07 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EABK/CFFsoXfp/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kiB4GwS2RCgOkeoFegxM X-IPAS-Result: Av4EABK/CFFsoXfp/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kiB4GwS2RCgOkeoFegxM X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="36558489" Received: from 108-161-119-233.dsl.teksavvy.com (HELO pastel.home) ([108.161.119.233]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 28 Oct 2013 21:35:01 -0400 Received: by pastel.home (Postfix, from userid 20848) id CE1AA60020; Mon, 28 Oct 2013 21:35:00 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters Message-ID: References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> Date: Mon, 28 Oct 2013 21:35:00 -0400 In-Reply-To: <834n81fdp8.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 28 Oct 2013 20:33:07 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 15260 Cc: Kenichi Handa , 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) > emacs = EMACSLOADPATH="$(abs_lisp)" LC_ALL=C "$(EMACS)" $(EMACSOPT) > ^^^^^^^^ > Does anyone know or remember why we set LC_ALL=C while running > commands in lisp/ (and the same in leim/)? IIRC the issue was to avoid things like misdetecting coding-systems because of the user's locale setting, in the files we load/compile. IOW, it was to work around bugs (e.g. missing coding: cookie) and is likely unneeded nowadays. Stefan From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 28 23:42:45 2013 Received: (at 15260) by debbugs.gnu.org; 29 Oct 2013 03:42:45 +0000 Received: from localhost ([127.0.0.1]:50404 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vb0Ca-0000sL-Mb for submit@debbugs.gnu.org; Mon, 28 Oct 2013 23:42:44 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:42045) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vb0CY-0000s5-92 for 15260@debbugs.gnu.org; Mon, 28 Oct 2013 23:42:43 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MVE00F00VFM2Y00@a-mtaout20.012.net.il> for 15260@debbugs.gnu.org; Tue, 29 Oct 2013 05:42:35 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVE00EWPVMZWY50@a-mtaout20.012.net.il>; Tue, 29 Oct 2013 05:42:35 +0200 (IST) Date: Tue, 29 Oct 2013 05:42:36 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Glenn Morris Message-id: <831u34g2tv.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, monnier@iro.umontreal.ca, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Glenn Morris > Cc: monnier@iro.umontreal.ca, Kenichi Handa , 15260@debbugs.gnu.org > Date: Mon, 28 Oct 2013 18:00:25 -0400 > > Eli Zaretskii wrote: > > > emacs = EMACSLOADPATH="$(abs_lisp)" LC_ALL=C "$(EMACS)" $(EMACSOPT) > > ^^^^^^^^ > > > > Does anyone know or remember why we set LC_ALL=C while running > > commands in lisp/ (and the same in leim/)? > > FWIW, if I change that to use LC_ALL=en_US.UTF8 and bootstrap (after > also fixing cl--gensym-counter to a non-random default), all the > resulting *.elc files are identical to the LC=ALL=C case. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Mon Oct 28 23:47:32 2013 Received: (at 15260) by debbugs.gnu.org; 29 Oct 2013 03:47:32 +0000 Received: from localhost ([127.0.0.1]:50411 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vb0HD-0000zq-Bh for submit@debbugs.gnu.org; Mon, 28 Oct 2013 23:47:31 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:62660) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vb0HA-0000zb-Un for 15260@debbugs.gnu.org; Mon, 28 Oct 2013 23:47:29 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0MVE00K00VU4KR00@a-mtaout22.012.net.il> for 15260@debbugs.gnu.org; Tue, 29 Oct 2013 05:47:22 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVE00KUIVUYCC90@a-mtaout22.012.net.il>; Tue, 29 Oct 2013 05:47:22 +0200 (IST) Date: Tue, 29 Oct 2013 05:47:23 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Stefan Monnier Message-id: <83zjpseo1g.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Stefan Monnier > Cc: Kenichi Handa , 15260@debbugs.gnu.org > Date: Mon, 28 Oct 2013 21:35:00 -0400 > > > emacs = EMACSLOADPATH="$(abs_lisp)" LC_ALL=C "$(EMACS)" $(EMACSOPT) > > ^^^^^^^^ > > > Does anyone know or remember why we set LC_ALL=C while running > > commands in lisp/ (and the same in leim/)? > > IIRC the issue was to avoid things like misdetecting coding-systems > because of the user's locale setting, in the files we load/compile. Makes sense. > IOW, it was to work around bugs (e.g. missing coding: cookie) and is > likely unneeded nowadays. Right, so the way should be clear to remove these. Thanks. From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 29 09:56:34 2013 Received: (at 15260) by debbugs.gnu.org; 29 Oct 2013 13:56:34 +0000 Received: from localhost ([127.0.0.1]:51228 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vb9mb-0000hi-Tb for submit@debbugs.gnu.org; Tue, 29 Oct 2013 09:56:34 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:47613) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vb9mY-0000ha-0o for 15260@debbugs.gnu.org; Tue, 29 Oct 2013 09:56:30 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EABK/CFFsoXfp/2dsb2JhbABEvw4Xc4IeAQEEAVYjEAs0EhQYDSSIHgaxH5AOkQoDpHqBXoMT X-IPAS-Result: Av4EABK/CFFsoXfp/2dsb2JhbABEvw4Xc4IeAQEEAVYjEAs0EhQYDSSIHgaxH5AOkQoDpHqBXoMT X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="36586237" Received: from 108-161-119-233.dsl.teksavvy.com (HELO pastel.home) ([108.161.119.233]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 29 Oct 2013 09:56:23 -0400 Received: by pastel.home (Postfix, from userid 20848) id A72C3608DC; Tue, 29 Oct 2013 09:56:23 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters Message-ID: References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> Date: Tue, 29 Oct 2013 09:56:23 -0400 In-Reply-To: <83zjpseo1g.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 29 Oct 2013 05:47:23 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) >> IOW, it was to work around bugs (e.g. missing coding: cookie) and is >> likely unneeded nowadays. > Right, so the way should be clear to remove these. There may still be problems laying dormant, but we should be able to fix them if/when they show up, Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 30 14:20:11 2013 Received: (at 15260) by debbugs.gnu.org; 30 Oct 2013 18:20:11 +0000 Received: from localhost ([127.0.0.1]:53216 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VbaNG-0000D3-Jg for submit@debbugs.gnu.org; Wed, 30 Oct 2013 14:20:10 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:44113) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VbaNE-0000Cc-1K for 15260@debbugs.gnu.org; Wed, 30 Oct 2013 14:20:09 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MVH00C00USDJX00@a-mtaout20.012.net.il> for 15260@debbugs.gnu.org; Wed, 30 Oct 2013 20:20:01 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVH00CQ9UXBK000@a-mtaout20.012.net.il>; Wed, 30 Oct 2013 20:19:59 +0200 (IST) Date: Wed, 30 Oct 2013 20:19:56 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Kenichi Handa Message-id: <838uxaei43.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: Stefan Monnier , 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) I bumped into this as part of digging into this bug report: there's a strange inconsistency between make_string and string_to_multibyte (or maybe it's just that the "multibyte" part of the name is overloaded). Specifically, if you pass a unibyte string through string_to_multibyte, it will produce a multibyte string, as expected. But if SDATA of the resulting multibyte string, or any of its derivatives, is passed to make_string, the latter will decide that it must make a unibyte string! This is because parse_str_as_multibyte, called internally by make_string, considers the multibyte representation of 8-bit bytes as a sign that the string produced from these bytes must be unibyte. Why do we have this confusing inconsistency? From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 30 21:01:30 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 01:01:30 +0000 Received: from localhost ([127.0.0.1]:53769 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vbgde-0004lt-CY for submit@debbugs.gnu.org; Wed, 30 Oct 2013 21:01:30 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:10266) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vbgdb-0004ld-Ha for 15260@debbugs.gnu.org; Wed, 30 Oct 2013 21:01:27 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EABK/CFFsoXfp/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kiB4GwS2RCgOkeoFegxM X-IPAS-Result: Av4EABK/CFFsoXfp/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kiB4GwS2RCgOkeoFegxM X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="36725730" Received: from 108-161-119-233.dsl.teksavvy.com (HELO pastel.home) ([108.161.119.233]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 30 Oct 2013 21:01:21 -0400 Received: by pastel.home (Postfix, from userid 20848) id 401DE60309; Wed, 30 Oct 2013 21:01:21 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters Message-ID: References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> Date: Wed, 30 Oct 2013 21:01:21 -0400 In-Reply-To: <838uxaei43.fsf@gnu.org> (Eli Zaretskii's message of "Wed, 30 Oct 2013 20:19:56 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 15260 Cc: Kenichi Handa , 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) > Why do we have this confusing inconsistency? make_string is a bug. There's no way to know/guess if the string should be unibyte or multibyte. So, it should be removed and replaced by calls to either make_unibyte_string or make_multibyte_string. Stefan From debbugs-submit-bounces@debbugs.gnu.org Wed Oct 30 23:48:13 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 03:48:13 +0000 Received: from localhost ([127.0.0.1]:53839 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VbjEy-0000Hn-Qk for submit@debbugs.gnu.org; Wed, 30 Oct 2013 23:48:13 -0400 Received: from mtaout21.012.net.il ([80.179.55.169]:47910) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VbjEv-0000HX-4U for 15260@debbugs.gnu.org; Wed, 30 Oct 2013 23:48:10 -0400 Received: from conversion-daemon.a-mtaout21.012.net.il by a-mtaout21.012.net.il (HyperSendmail v2007.08) id <0MVI00C00L1UMG00@a-mtaout21.012.net.il> for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 05:48:02 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout21.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVI00C1KL81MS00@a-mtaout21.012.net.il>; Thu, 31 Oct 2013 05:48:02 +0200 (IST) Date: Thu, 31 Oct 2013 05:47:48 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Stefan Monnier Message-id: <831u32drtn.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Stefan Monnier > Cc: Kenichi Handa , 15260@debbugs.gnu.org > Date: Wed, 30 Oct 2013 21:01:21 -0400 > > > Why do we have this confusing inconsistency? > > make_string is a bug. There's no way to know/guess if the string should > be unibyte or multibyte. Well, there is a way, but it's tricky ;-) Yes, this inconsistency caused me a lot of grief while working on this bug. > So, it should be removed and replaced by calls to either > make_unibyte_string or make_multibyte_string. I presume you think the same about build_string, then. By sheer luck (or maybe something else), this is exactly what I've been doing in every case where it mattered for the non-ASCII build. (The job is almost done, btw, I'm in final testing.) From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 31 09:40:18 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 13:40:18 +0000 Received: from localhost ([127.0.0.1]:54429 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VbsTx-0000DG-0H for submit@debbugs.gnu.org; Thu, 31 Oct 2013 09:40:17 -0400 Received: from pruche.dit.umontreal.ca ([132.204.246.22]:59196) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VbsTo-0000Cz-Nh for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 09:40:09 -0400 Received: from faina.iro.umontreal.ca (lechon.iro.umontreal.ca [132.204.27.242]) by pruche.dit.umontreal.ca (8.14.1/8.14.1) with ESMTP id r9VDe7Iu023803; Thu, 31 Oct 2013 09:40:07 -0400 Received: by faina.iro.umontreal.ca (Postfix, from userid 20848) id D0807B42FC; Thu, 31 Oct 2013 09:40:07 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters Message-ID: References: <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <831u32drtn.fsf@gnu.org> Date: Thu, 31 Oct 2013 09:40:07 -0400 In-Reply-To: <831u32drtn.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 31 Oct 2013 05:47:48 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-NAI-Spam-Flag: NO X-NAI-Spam-Threshold: 5 X-NAI-Spam-Score: 0 X-NAI-Spam-Rules: 1 Rules triggered RV4748=0 X-NAI-Spam-Version: 2.3.0.9362 : core <4748> : inlines <193> : streams <1065574> : uri <1581668> X-Spam-Score: -1.9 (-) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.9 (-) >> So, it should be removed and replaced by calls to either >> make_unibyte_string or make_multibyte_string. > I presume you think the same about build_string, then. Pretty much, except that I tend to think of build_string as only meant for use on string constants, which are all ASCII, normally, so it doesn't matter nearly as much. Stefan From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 31 12:25:44 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 16:25:44 +0000 Received: from localhost ([127.0.0.1]:55117 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vbv42-0004U3-Kl for submit@debbugs.gnu.org; Thu, 31 Oct 2013 12:25:43 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:46270) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vbv3w-0004Tm-Kj for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 12:25:40 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MVJ00000K6MG200@a-mtaout20.012.net.il> for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 18:25:15 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVJ000UDKA30EE0@a-mtaout20.012.net.il>; Thu, 31 Oct 2013 18:25:15 +0200 (IST) Date: Thu, 31 Oct 2013 18:25:00 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Stefan Monnier Message-id: <83y559csrn.fsf@gnu.org> References: <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <831u32drtn.fsf@gnu.org> X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Stefan Monnier > Cc: handa@gnu.org, 15260@debbugs.gnu.org > Date: Thu, 31 Oct 2013 09:40:07 -0400 > > >> So, it should be removed and replaced by calls to either > >> make_unibyte_string or make_multibyte_string. > > I presume you think the same about build_string, then. > > Pretty much, except that I tend to think of build_string as only meant > for use on string constants, which are all ASCII, normally, so it > doesn't matter nearly as much. About 20% of uses of build_string are not guaranteed to act on pure ASCII strings. From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 31 13:16:58 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 17:16:58 +0000 Received: from localhost ([127.0.0.1]:55218 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vbvre-0005tI-0x for submit@debbugs.gnu.org; Thu, 31 Oct 2013 13:16:58 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:35789) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vbvra-0005t0-Ab for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 13:16:55 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0MVJ00800M8WT900@a-mtaout22.012.net.il> for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 19:16:47 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVJ0081UMNTEMD0@a-mtaout22.012.net.il>; Thu, 31 Oct 2013 19:16:41 +0200 (IST) Date: Thu, 31 Oct 2013 19:16:26 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Stefan Monnier Message-id: <83wqktcqdx.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Stefan Monnier > Cc: Kenichi Handa , 15260@debbugs.gnu.org > Date: Wed, 30 Oct 2013 21:01:21 -0400 > > > Why do we have this confusing inconsistency? > > make_string is a bug. There's no way to know/guess if the string should > be unibyte or multibyte. So, it should be removed and replaced by calls > to either make_unibyte_string or make_multibyte_string. Here's one more gotcha I bumped into while working on this bug. Suppose the filesystem where you build Emacs uses a file-name encoding whose coding-system-category is 'charset'. Example: cpNNNN. Then, when Emacs comes up after dumping, it loads subdirs.el in each directory on load-path. To do this, it calls 'openp' to look for DIR/subdirs.el, which involves calling ENCODE_FILE on "DIR/subdirs.el", in order to pass that to 'faccessat' or 'open'. Now, if the charset that is needed to encode this file name is not yet loaded into Emacs, Emacs will try to load it. To this end, it will look along charset-map-path for the corresponding map file, and for that it will again call 'openp', recursively. That 'openp' call will again want to ENCODE_FILE with the same encoding, which will again cause Emacs to try to load the corresponding map file, etc. etc., until we exhaust the specpdl stack. I worked around this by keeping charset-map-path in unibyte form until later into the startup procedure. Is there a more elegant and less kludgey way? From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 31 14:00:30 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 18:00:30 +0000 Received: from localhost ([127.0.0.1]:55336 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VbwXj-0008Dv-Bk for submit@debbugs.gnu.org; Thu, 31 Oct 2013 14:00:28 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:34361) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VbwXd-0008Db-Jm for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 14:00:24 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MVJ00000O08ZF00@a-mtaout20.012.net.il> for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 20:00:14 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVJ000C6OO7OQD0@a-mtaout20.012.net.il>; Thu, 31 Oct 2013 20:00:08 +0200 (IST) Date: Thu, 31 Oct 2013 19:59:52 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: <831u32drtn.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: Stefan Monnier , Glenn Morris Message-id: <83sivhcodj.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <831u32drtn.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: Kenichi Handa , 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Thu, 31 Oct 2013 05:47:48 +0200 > From: Eli Zaretskii > Cc: 15260@debbugs.gnu.org > > (The job is almost done, btw, I'm in final testing.) Below is what I came up with. This survived several bootstraps, both on GNU/Linux (in- and out-of-source-tree builds) and on MS-Windows, including "make install" into a non-ASCII directory and invocation from there. To summarize the changes: . ENCODE_FILE now explicitly leaves alone unibyte strings. (It could be that it did this before as well, but I couldn't find the code which had that effect, and doing that early on is TRT anyway.) . All the *-directory and *-path variables we create at startup are forced to be unibyte strings. (Previously, they were sometimes unibyte and sometimes multibyte.) In temacs that dumps or bootstraps, they stay unibyte all the way till program exit. In a session that runs startup.el, they are decoded early in normal-top-level, after setting up the locale environment. The code which decodes these was moved much closer to the beginning of normal-top-level, as its previous place was too late, after some damage was already done. . Several left-overs from working around problems that are long gone are removed. Notable examples: (a) "set LC_ALL=C" when running 'emacs' (NOT 'temacs') in lisp/ and leim/; (b) storing (in init_buffer) the default-directory of *scratch* in the multibyte representation of the original unibyte bytes. . A few related bugs which got in the way were fixed. E.g., in make_temp_name. Please test this, as I could only do that in 2 different locales. Please pay specific attention to strings in load-path, default-directory, data-directory, etc., after Emacs comes up in an interactive session: they should all be multibyte strings (you can use multibyte-string-p to test that). If no problems pop up, I will commit this in a few days. Thanks. === modified file 'configure.ac' --- configure.ac 2013-10-27 18:57:20 +0000 +++ configure.ac 2013-10-31 16:57:18 +0000 @@ -73,30 +73,6 @@ dnl Support for --program-prefix, --prog dnl --program-transform-name options AC_ARG_PROGRAM -dnl http://debbugs.gnu.org/15260 -dnl I think we have to check, eg, both exec_prefix and bindir, -dnl because the latter by default is not yet expanded, but the user -dnl may have specified a value for it via --bindir. -dnl At first glance, _installing_ in non-ASCII seems ok, but in fact -dnl it is not; see http://debbugs.gnu.org/15260#61 -dnl Note that abs_srcdir and abs_builddir are not yet defined. :( - -dnl "`cd \"$srcdir\"`" is not portable. -dnl See autoconf manual "Shell Substitutions": -dnl "There is just no portable way to use double-quoted strings inside -dnl double-quoted back-quoted expressions (pfew!)." -temp_srcdir=`cd "$srcdir"; pwd` - -for var in "`pwd`" "$temp_srcdir" "$prefix" "$exec_prefix" \ - "$datarootdir" "$bindir" "$datadir" "$sharedstatedir" "$libexecdir"; do - - dnl configure sets LC_ALL=C early on, so this range should work. - case "$var" in - *[[^\ -~]]*) AC_MSG_ERROR([Emacs cannot be built or installed in a directory whose name contains non-ASCII characters: $var]) ;; - esac - -done - dnl It is important that variables on the RHS not be expanded here, dnl hence the single quotes. This is per the GNU coding standards, see dnl (autoconf) Installation Directory Variables === modified file 'leim/Makefile.in' --- leim/Makefile.in 2013-10-24 02:29:29 +0000 +++ leim/Makefile.in 2013-10-31 16:57:18 +0000 @@ -34,7 +34,7 @@ EMACS = ../src/emacs buildlisppath=${abs_srcdir}/../lisp # How to run Emacs. -RUN_EMACS = EMACSLOADPATH="$(buildlisppath)" LC_ALL=C \ +RUN_EMACS = EMACSLOADPATH="$(buildlisppath)" \ "${EMACS}" -batch --no-site-file --no-site-lisp MKDIR_P = @MKDIR_P@ === modified file 'lisp/Makefile.in' --- lisp/Makefile.in 2013-10-31 07:27:35 +0000 +++ lisp/Makefile.in 2013-10-31 16:57:18 +0000 @@ -115,7 +115,7 @@ COMPILE_FIRST = \ # The actual Emacs command run in the targets below. -emacs = EMACSLOADPATH="$(abs_lisp)" LC_ALL=C "$(EMACS)" $(EMACSOPT) +emacs = EMACSLOADPATH="$(abs_lisp)" "$(EMACS)" $(EMACSOPT) # Common command to find subdirectories setwins=subdirs=`find . -type d -print`; \ === modified file 'lisp/loadup.el' --- lisp/loadup.el 2013-10-08 15:11:29 +0000 +++ lisp/loadup.el 2013-10-31 16:57:18 +0000 @@ -286,6 +286,20 @@ ;For other systems, you must edit ../src/Makefile.in. (load "site-load" t) +;; Make sure default-directory is unibyte when dumping. This is +;; because we cannot decode and encode it correctly (since the locale +;; environment is not, and should not be, set up). default-directory +;; is used every time we call expand-file-name, which we do in every +;; file primitive. So the only workable solution to support building +;; in non-ASCII directories is to manipulate unibyte strings in the +;; current locale's encoding. +(if (and (or (equal (nth 3 command-line-args) "dump") + (equal (nth 4 command-line-args) "dump") + (equal (nth 3 command-line-args) "bootstrap") + (equal (nth 4 command-line-args) "bootstrap")) + (multibyte-string-p default-directory)) + (setq default-directory (string-to-unibyte default-directory))) + ;; Determine which last version number to use ;; based on the executables that now exist. (if (and (or (equal (nth 3 command-line-args) "dump") === modified file 'lisp/startup.el' --- lisp/startup.el 2013-10-30 02:45:53 +0000 +++ lisp/startup.el 2013-10-31 17:04:11 +0000 @@ -489,6 +489,63 @@ It is the default value of the variable (if command-line-processed (message "Back to top level.") (setq command-line-processed t) + + ;; Set the default strings to display in mode line for end-of-line + ;; formats that aren't native to this platform. This should be + ;; done before calling set-locale-environment, as the latter might + ;; use these mnemonics. + (cond + ((memq system-type '(ms-dos windows-nt)) + (setq eol-mnemonic-unix "(Unix)" + eol-mnemonic-mac "(Mac)")) + (t ; this is for Unix/GNU/Linux systems + (setq eol-mnemonic-dos "(DOS)" + eol-mnemonic-mac "(Mac)"))) + + (set-locale-environment nil) + ;; Decode all default-directory's (probably, only *scratch* exists + ;; at this point). default-directory of *scratch* is the basis + ;; for many other file-name variables and directory lists, so it + ;; is important to decode it ASAP. + (when locale-coding-system + (save-excursion + (dolist (elt (buffer-list)) + (set-buffer elt) + (if default-directory + (setq default-directory + (decode-coding-string default-directory + locale-coding-system t))))) + + ;; Decode all the important variables and directory lists, now + ;; that we know the locale's encoding. This is because the + ;; values of these variables are until here unibyte undecoded + ;; strings created by build_unibyte_string. data-directory in + ;; particular is used to construct many other standard directory + ;; names, so it must be decoded ASAP. + ;; Note that charset-map-path cannot be decoded here, since we + ;; could then be trapped in infinite recursion below, when we + ;; load subdirs.el, because encoding a directory name might need + ;; to load a charset map, which will want to encode + ;; charset-map-path, which will want to load the same charset + ;; map... So decoding of charset-map-path is delayed until + ;; further down below. + (dolist (pathsym '(load-path exec-path)) + (let ((path (symbol-value pathsym))) + (if (listp path) + (set pathsym (mapcar (lambda (dir) + (decode-coding-string + dir + locale-coding-system t)) + path))))) + (dolist (filesym '(data-directory doc-directory exec-directory + installation-directory + invocation-directory invocation-name + source-directory + shared-game-score-directory)) + (let ((file (symbol-value filesym))) + (if (stringp file) + (set filesym (decode-coding-string file locale-coding-system t)))))) + (let ((dir default-directory)) (with-current-buffer "*Messages*" (messages-buffer-mode) @@ -536,6 +593,16 @@ It is the default value of the variable (setq process-environment (delete (concat "PWD=" pwd) process-environment))))) + ;; Now, that other directories were searched, and any charsets we + ;; need for encoding them are already loaded, we are ready to + ;; decode charset-map-path. + (if (listp charset-map-path) + (setq charset-map-path + (mapcar (lambda (dir) + (decode-coding-string + dir + locale-coding-system t)) + charset-map-path))) (setq default-directory (abbreviate-file-name default-directory)) (let ((old-face-font-rescale-alist face-font-rescale-alist)) (unwind-protect @@ -756,18 +823,6 @@ Amongst another things, it parses the co ;;! ;; Choose a good default value for split-window-keep-point. ;;! (setq split-window-keep-point (> baud-rate 2400)) - ;; Set the default strings to display in mode line for - ;; end-of-line formats that aren't native to this platform. - (cond - ((memq system-type '(ms-dos windows-nt)) - (setq eol-mnemonic-unix "(Unix)" - eol-mnemonic-mac "(Mac)")) - (t ; this is for Unix/GNU/Linux systems - (setq eol-mnemonic-dos "(DOS)" - eol-mnemonic-mac "(Mac)"))) - - (set-locale-environment nil) - ;; Convert preloaded file names in load-history to absolute. (let ((simple-file-name ;; Look for simple.el or simple.elc and use their directory @@ -801,7 +856,7 @@ please check its value") load-history)))) ;; Convert the arguments to Emacs internal representation. - (let ((args (cdr command-line-args))) + (let ((args command-line-args)) (while args (setcar args (decode-coding-string (car args) locale-coding-system t)) @@ -1211,19 +1266,6 @@ the `--debug-init' option to view a comp (setq after-init-time (current-time)) (run-hooks 'after-init-hook) - ;; Decode all default-directory. - (if (and (default-value 'enable-multibyte-characters) locale-coding-system) - (save-excursion - (dolist (elt (buffer-list)) - (set-buffer elt) - (if default-directory - (setq default-directory - (decode-coding-string default-directory - locale-coding-system t)))) - (setq command-line-default-directory - (decode-coding-string command-line-default-directory - locale-coding-system t)))) - ;; If *scratch* exists and init file didn't change its mode, initialize it. (if (get-buffer "*scratch*") (with-current-buffer "*scratch*" === modified file 'src/buffer.c' --- src/buffer.c 2013-10-29 14:46:23 +0000 +++ src/buffer.c 2013-10-31 16:57:18 +0000 @@ -5349,13 +5349,10 @@ init_buffer (void) len++; } + /* At this moment, we still don't know how to decode the directory + name. So, we keep the bytes in unibyte form so that file I/O + routines correctly get the original bytes. */ bset_directory (current_buffer, make_unibyte_string (pwd, len)); - if (! NILP (BVAR (&buffer_defaults, enable_multibyte_characters))) - /* At this moment, we still don't know how to decode the - directory name. So, we keep the bytes in multibyte form so - that ENCODE_FILE correctly gets the original bytes. */ - bset_directory - (current_buffer, string_to_multibyte (BVAR (current_buffer, directory))); /* Add /: to the front of the name if it would otherwise be treated as magic. */ === modified file 'src/callproc.c' --- src/callproc.c 2013-08-23 17:57:07 +0000 +++ src/callproc.c 2013-10-31 16:57:18 +0000 @@ -1612,14 +1612,14 @@ init_callproc (void) Lisp_Object tem, tem1, srcdir; srcdir = Fexpand_file_name (build_string ("../src/"), - build_string (PATH_DUMPLOADSEARCH)); + build_unibyte_string (PATH_DUMPLOADSEARCH)); tem = Fexpand_file_name (build_string ("GNU"), Vdata_directory); tem1 = Ffile_exists_p (tem); if (!NILP (Fequal (srcdir, Vinvocation_directory)) || NILP (tem1)) { Lisp_Object newdir; newdir = Fexpand_file_name (build_string ("../etc/"), - build_string (PATH_DUMPLOADSEARCH)); + build_unibyte_string (PATH_DUMPLOADSEARCH)); tem = Fexpand_file_name (build_string ("GNU"), newdir); tem1 = Ffile_exists_p (tem); if (!NILP (tem1)) @@ -1646,7 +1646,7 @@ init_callproc (void) #ifdef DOS_NT Vshared_game_score_directory = Qnil; #else - Vshared_game_score_directory = build_string (PATH_GAME); + Vshared_game_score_directory = build_unibyte_string (PATH_GAME); if (NILP (Ffile_accessible_directory_p (Vshared_game_score_directory))) Vshared_game_score_directory = Qnil; #endif === modified file 'src/coding.h' --- src/coding.h 2013-10-08 06:40:09 +0000 +++ src/coding.h 2013-10-31 16:57:18 +0000 @@ -670,14 +670,16 @@ struct coding_system (code) = (s1 << 8) | s2; \ } while (0) -/* Encode the file name NAME using the specified coding system - for file names, if any. */ -#define ENCODE_FILE(name) \ - (! NILP (Vfile_name_coding_system) \ - ? code_convert_string_norecord (name, Vfile_name_coding_system, 1) \ - : (! NILP (Vdefault_file_name_coding_system) \ - ? code_convert_string_norecord (name, Vdefault_file_name_coding_system, 1) \ - : name)) +/* Encode the file name NAME using the specified coding system for + file names, if any. If NAME is a unibyte string, return NAME. */ +#define ENCODE_FILE(name) \ + (! STRING_MULTIBYTE (name) \ + ? name \ + : (! NILP (Vfile_name_coding_system) \ + ? code_convert_string_norecord (name, Vfile_name_coding_system, 1) \ + : (! NILP (Vdefault_file_name_coding_system) \ + ? code_convert_string_norecord (name, Vdefault_file_name_coding_system, 1) \ + : name))) /* Decode the file name NAME using the specified coding system === modified file 'src/emacs.c' --- src/emacs.c 2013-10-31 08:32:42 +0000 +++ src/emacs.c 2013-10-31 17:03:39 +0000 @@ -396,7 +396,7 @@ init_cmdargs (int argc, char **argv, int initial_argv = argv; initial_argc = argc; - raw_name = build_string (argv[0]); + raw_name = build_unibyte_string (argv[0]); /* Add /: to the front of the name if it would otherwise be treated as magic. */ @@ -430,7 +430,9 @@ init_cmdargs (int argc, char **argv, int /* Emacs was started with relative path, like ./emacs. Make it absolute. */ { - Lisp_Object odir = original_pwd ? build_string (original_pwd) : Qnil; + Lisp_Object odir = + original_pwd ? build_unibyte_string (original_pwd) : Qnil; + Vinvocation_directory = Fexpand_file_name (Vinvocation_directory, odir); } @@ -2204,7 +2206,7 @@ decode_env_path (const char *evarname, c p = strchr (path, SEPCHAR); if (!p) p = path + strlen (path); - element = (p - path ? make_string (path, p - path) + element = (p - path ? make_unibyte_string (path, p - path) : build_string (".")); #ifdef WINDOWSNT /* Relative file names in the default path are interpreted as @@ -2214,7 +2216,7 @@ decode_env_path (const char *evarname, c element = Fexpand_file_name (Fsubstring (element, make_number (emacs_dir_len), Qnil), - build_string (emacs_dir)); + build_unibyte_string (emacs_dir)); #endif /* Add /: to the front of the name === modified file 'src/fileio.c' --- src/fileio.c 2013-10-17 06:42:21 +0000 +++ src/fileio.c 2013-10-31 16:57:18 +0000 @@ -732,8 +732,8 @@ static unsigned make_temp_name_count, ma Lisp_Object make_temp_name (Lisp_Object prefix, bool base64_p) { - Lisp_Object val; - int len, clen; + Lisp_Object val, encoded_prefix; + int len; printmax_t pid; char *p, *data; char pidbuf[INT_BUFSIZE_BOUND (printmax_t)]; @@ -767,12 +767,11 @@ make_temp_name (Lisp_Object prefix, bool #endif } - len = SBYTES (prefix); clen = SCHARS (prefix); - val = make_uninit_multibyte_string (clen + 3 + pidlen, len + 3 + pidlen); - if (!STRING_MULTIBYTE (prefix)) - STRING_SET_UNIBYTE (val); + encoded_prefix = ENCODE_FILE (prefix); + len = SBYTES (encoded_prefix); + val = make_uninit_string (len + 3 + pidlen); data = SSDATA (val); - memcpy (data, SSDATA (prefix), len); + memcpy (data, SSDATA (encoded_prefix), len); p = data + len; memcpy (p, pidbuf, pidlen); @@ -810,7 +809,7 @@ make_temp_name (Lisp_Object prefix, bool { /* We want to return only if errno is ENOENT. */ if (errno == ENOENT) - return val; + return DECODE_FILE (val); else /* The error here is dubious, but there is little else we can do. The alternatives are to return nil, which is @@ -987,7 +986,26 @@ filesystem tree, not (expand-file-name " if (multibyte != STRING_MULTIBYTE (default_directory)) { if (multibyte) - default_directory = string_to_multibyte (default_directory); + { + unsigned char *p = SDATA (name); + + while (*p && ASCII_BYTE_P (*p)) + p++; + if (*p == '\0') + { + /* NAME is a pure ASCII string, and DEFAULT_DIRECTORY is + unibyte. Do not convert DEFAULT_DIRECTORY to + multibyte; instead, convert NAME to a unibyte string, + so that the result of this function is also a unibyte + string. This is needed during bootstraping and + dumping, when Emacs cannot decode file names, because + the locale environment is not set up. */ + name = make_unibyte_string (SSDATA (name), SBYTES (name)); + multibyte = 0; + } + else + default_directory = string_to_multibyte (default_directory); + } else { name = string_to_multibyte (name); === modified file 'src/lread.c' --- src/lread.c 2013-09-26 03:46:47 +0000 +++ src/lread.c 2013-10-31 16:57:18 +0000 @@ -1500,7 +1500,8 @@ openp (Lisp_Object path, Lisp_Object str for (tail = NILP (suffixes) ? list1 (empty_unibyte_string) : suffixes; CONSP (tail); tail = XCDR (tail)) { - ptrdiff_t fnlen, lsuffix = SBYTES (XCAR (tail)); + Lisp_Object suffix = XCAR (tail); + ptrdiff_t fnlen, lsuffix = SBYTES (suffix); Lisp_Object handler; /* Concatenate path element/specified name with the suffix. @@ -1511,7 +1512,7 @@ openp (Lisp_Object path, Lisp_Object str ? 2 : 0); fnlen = SBYTES (filename) - prefixlen; memcpy (fn, SDATA (filename) + prefixlen, fnlen); - memcpy (fn + fnlen, SDATA (XCAR (tail)), lsuffix + 1); + memcpy (fn + fnlen, SDATA (suffix), lsuffix + 1); fnlen += lsuffix; /* Check that the file exists and is not a directory. */ /* We used to only check for handlers on non-absolute file names: @@ -1521,7 +1522,18 @@ openp (Lisp_Object path, Lisp_Object str handler = Ffind_file_name_handler (filename, Qfile_exists_p); It's not clear why that was the case and it breaks things like (load "/bar.el") where the file is actually "/bar.el.gz". */ - string = make_string (fn, fnlen); + /* make_string has its own ideas on when to return a unibyte + string and when a multibyte string, but we know better. + We must have a unibyte string when dumping, since + file-name encoding is shaky at best at that time, and in + particular default-file-name-coding-system is reset + several times during loadup. We therefore don't want to + encode the file before passing it to file I/O library + functions. */ + if (!STRING_MULTIBYTE (filename) && !STRING_MULTIBYTE (suffix)) + string = make_unibyte_string (fn, fnlen); + else + string = make_string (fn, fnlen); handler = Ffind_file_name_handler (string, Qfile_exists_p); if ((!NILP (handler) || !NILP (predicate)) && !NATNUMP (predicate)) { === modified file 'src/xdisp.c' --- src/xdisp.c 2013-10-29 16:11:50 +0000 +++ src/xdisp.c 2013-10-31 16:57:18 +0000 @@ -9728,7 +9728,11 @@ message3_nolog (Lisp_Object m) putc ('\n', stderr); noninteractive_need_newline = 0; if (STRINGP (m)) - fwrite (SDATA (m), SBYTES (m), 1, stderr); + { + Lisp_Object s = ENCODE_SYSTEM (m); + + fwrite (SDATA (s), SBYTES (s), 1, stderr); + } if (cursor_in_echo_area == 0) fprintf (stderr, "\n"); fflush (stderr); @@ -9803,13 +9807,19 @@ message_with_string (const char *m, Lisp { if (m) { + /* ENCODE_SYSTEM below can GC and/or relocate the Lisp + String whose data pointer might be passed to us in M. So + we use a local copy. */ + char *fmt = xstrdup (m); + if (noninteractive_need_newline) putc ('\n', stderr); noninteractive_need_newline = 0; - fprintf (stderr, m, SDATA (string)); + fprintf (stderr, fmt, SDATA (ENCODE_SYSTEM (string))); if (!cursor_in_echo_area) fprintf (stderr, "\n"); fflush (stderr); + xfree (fmt); } } else if (INTERACTIVE) From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 31 14:05:08 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 18:05:08 +0000 Received: from localhost ([127.0.0.1]:55347 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VbwcF-0008LT-Rq for submit@debbugs.gnu.org; Thu, 31 Oct 2013 14:05:08 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:63421) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VbwcC-0008Kx-J6 for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 14:05:05 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EABK/CFFsoXfp/2dsb2JhbABEvw4Xc4IfAQVWIxALNBIUGA0kiCTBLZEKA6R6gV6DEw X-IPAS-Result: Av4EABK/CFFsoXfp/2dsb2JhbABEvw4Xc4IfAQVWIxALNBIUGA0kiCTBLZEKA6R6gV6DEw X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="36782143" Received: from 108-161-119-233.dsl.teksavvy.com (HELO pastel.home) ([108.161.119.233]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 31 Oct 2013 14:04:58 -0400 Received: by pastel.home (Postfix, from userid 20848) id 79DBC611E1; Thu, 31 Oct 2013 14:04:58 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters Message-ID: References: <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <831u32drtn.fsf@gnu.org> <83y559csrn.fsf@gnu.org> Date: Thu, 31 Oct 2013 14:04:58 -0400 In-Reply-To: <83y559csrn.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 31 Oct 2013 18:25:00 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) >> >> So, it should be removed and replaced by calls to either >> >> make_unibyte_string or make_multibyte_string. >> > I presume you think the same about build_string, then. >> Pretty much, except that I tend to think of build_string as only meant >> for use on string constants, which are all ASCII, normally, so it >> doesn't matter nearly as much. > About 20% of uses of build_string are not guaranteed to act on pure > ASCII strings. These should presumably use something like make_unibyte_string or make_multibyte_string, then. Stefan From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 31 14:09:48 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 18:09:48 +0000 Received: from localhost ([127.0.0.1]:55381 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vbwgl-0008UV-4j for submit@debbugs.gnu.org; Thu, 31 Oct 2013 14:09:47 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:58719) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vbwgj-0008UH-2H for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 14:09:45 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8EABK/CFFsoXfp/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLFAEfEhQYDSQTiAsGwS2RCgOSWpIggV6DEw X-IPAS-Result: Av8EABK/CFFsoXfp/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLFAEfEhQYDSQTiAsGwS2RCgOSWpIggV6DEw X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="36782530" Received: from 108-161-119-233.dsl.teksavvy.com (HELO pastel.home) ([108.161.119.233]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 31 Oct 2013 14:09:39 -0400 Received: by pastel.home (Postfix, from userid 20848) id 3D9A9611E1; Thu, 31 Oct 2013 14:09:39 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters Message-ID: References: <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <83wqktcqdx.fsf@gnu.org> Date: Thu, 31 Oct 2013 14:09:39 -0400 In-Reply-To: <83wqktcqdx.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 31 Oct 2013 19:16:26 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) >>>>> "Eli" == Eli Zaretskii writes: >> From: Stefan Monnier >> Cc: Kenichi Handa , 15260@debbugs.gnu.org >> Date: Wed, 30 Oct 2013 21:01:21 -0400 >> >> > Why do we have this confusing inconsistency? >> >> make_string is a bug. There's no way to know/guess if the string should >> be unibyte or multibyte. So, it should be removed and replaced by calls >> to either make_unibyte_string or make_multibyte_string. > Here's one more gotcha I bumped into while working on this bug. > Suppose the filesystem where you build Emacs uses a file-name encoding > whose coding-system-category is 'charset'. Example: cpNNNN. Then, > when Emacs comes up after dumping, it loads subdirs.el in each > directory on load-path. To do this, it calls 'openp' to look for > DIR/subdirs.el, which involves calling ENCODE_FILE on > "DIR/subdirs.el", in order to pass that to 'faccessat' or 'open'. > Now, if the charset that is needed to encode this file name is not yet > loaded into Emacs, Emacs will try to load it. To this end, it will > look along charset-map-path for the corresponding map file, and for > that it will again call 'openp', recursively. That 'openp' call will > again want to ENCODE_FILE with the same encoding, which will again > cause Emacs to try to load the corresponding map file, etc. etc., > until we exhaust the specpdl stack. So you mean that we have: - charset-map-path is a multibyte string. - the file-name encoding uses a charset that's not yet loaded. How do we get into such a state? Stefan From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 31 14:38:20 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 18:38:20 +0000 Received: from localhost ([127.0.0.1]:55426 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vbx8N-0000qN-HQ for submit@debbugs.gnu.org; Thu, 31 Oct 2013 14:38:19 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:53391) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vbx8J-0000q0-1U for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 14:38:16 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0MVJ00900QDNGB00@a-mtaout22.012.net.il> for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 20:38:08 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVJ009T8QFKEB20@a-mtaout22.012.net.il>; Thu, 31 Oct 2013 20:38:08 +0200 (IST) Date: Thu, 31 Oct 2013 20:37:52 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Stefan Monnier Message-id: <83mwlpcmm7.fsf@gnu.org> References: <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <83wqktcqdx.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Stefan Monnier > Cc: handa@gnu.org, 15260@debbugs.gnu.org > Date: Thu, 31 Oct 2013 14:09:39 -0400 > > So you mean that we have: > - charset-map-path is a multibyte string. > - the file-name encoding uses a charset that's not yet loaded. Yes. > How do we get into such a state? Not sure about the details, since I don't really understand when Emacs needs to load the charset map. Perhaps the map is needed only when we need to encode a string, not for decoding? Phenomenologically, this happened when charset-map-path was already decoded (as opposed to being a unibyte string) when this part of startup.el runs: ;; Convert preloaded file names in load-history to absolute. (let ((simple-file-name ;; Look for simple.el or simple.elc and use their directory ;; as the place where all Lisp files live. (locate-file "simple" load-path (get-load-suffixes))) lisp-dir) locate-file eventually calls 'openp', which wants to encode directories from load-path concatenated with simple.el etc. From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 31 15:25:08 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 19:25:08 +0000 Received: from localhost ([127.0.0.1]:55476 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vbxrd-00023L-HT for submit@debbugs.gnu.org; Thu, 31 Oct 2013 15:25:06 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:29119) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vbxrb-00022s-Hz for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 15:25:03 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EABK/CFFsoXfp/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kLodwBsEtkQoDpHqBXoMT X-IPAS-Result: Av4EABK/CFFsoXfp/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kLodwBsEtkQoDpHqBXoMT X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="36789424" Received: from 108-161-119-233.dsl.teksavvy.com (HELO pastel.home) ([108.161.119.233]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 31 Oct 2013 15:24:57 -0400 Received: by pastel.home (Postfix, from userid 20848) id A1341611E1; Thu, 31 Oct 2013 15:24:57 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters Message-ID: References: <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <831u32drtn.fsf@gnu.org> <83sivhcodj.fsf@gnu.org> Date: Thu, 31 Oct 2013 15:24:57 -0400 In-Reply-To: <83sivhcodj.fsf@gnu.org> (Eli Zaretskii's message of "Thu, 31 Oct 2013 19:59:52 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 15260 Cc: Glenn Morris , 15260@debbugs.gnu.org, Kenichi Handa X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) > Below is what I came up with. This survived several bootstraps, both Thanks, Eli. > +;; Make sure default-directory is unibyte when dumping. This is > +;; because we cannot decode and encode it correctly (since the locale > +;; environment is not, and should not be, set up). default-directory > +;; is used every time we call expand-file-name, which we do in every > +;; file primitive. So the only workable solution to support building > +;; in non-ASCII directories is to manipulate unibyte strings in the > +;; current locale's encoding. > +(if (and (or (equal (nth 3 command-line-args) "dump") > + (equal (nth 4 command-line-args) "dump") > + (equal (nth 3 command-line-args) "bootstrap") > + (equal (nth 4 command-line-args) "bootstrap")) > + (multibyte-string-p default-directory)) > + (setq default-directory (string-to-unibyte default-directory))) I'm not sure I understand this string-to-unibyte. This call seems to only be correct if default-directory holds the undecoded but multibyte name. Why would we have an undecided yet multibyte name? IOW, I'd expect here to either have default-directory be unibyte already, or be multibyte but encoded in some (arbitrary) encoding (in which case we can't really know how to re-encode it). Stefan From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 31 15:33:47 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 19:33:47 +0000 Received: from localhost ([127.0.0.1]:55489 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vby02-0002Hq-J0 for submit@debbugs.gnu.org; Thu, 31 Oct 2013 15:33:46 -0400 Received: from mtaout21.012.net.il ([80.179.55.169]:49305) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vby00-0002HZ-Kl for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 15:33:45 -0400 Received: from conversion-daemon.a-mtaout21.012.net.il by a-mtaout21.012.net.il (HyperSendmail v2007.08) id <0MVJ00I00SU7GY00@a-mtaout21.012.net.il> for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 21:33:37 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout21.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVJ00I5ET01DX20@a-mtaout21.012.net.il>; Thu, 31 Oct 2013 21:33:37 +0200 (IST) Date: Thu, 31 Oct 2013 21:33:22 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Stefan Monnier Message-id: <83iowdck1p.fsf@gnu.org> References: <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <831u32drtn.fsf@gnu.org> <83sivhcodj.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: rgm@gnu.org, 15260@debbugs.gnu.org, handa@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Stefan Monnier > Cc: Glenn Morris , 15260@debbugs.gnu.org, Kenichi Handa > Date: Thu, 31 Oct 2013 15:24:57 -0400 > > > Below is what I came up with. This survived several bootstraps, both > > Thanks, Eli. > > > +;; Make sure default-directory is unibyte when dumping. This is > > +;; because we cannot decode and encode it correctly (since the locale > > +;; environment is not, and should not be, set up). default-directory > > +;; is used every time we call expand-file-name, which we do in every > > +;; file primitive. So the only workable solution to support building > > +;; in non-ASCII directories is to manipulate unibyte strings in the > > +;; current locale's encoding. > > +(if (and (or (equal (nth 3 command-line-args) "dump") > > + (equal (nth 4 command-line-args) "dump") > > + (equal (nth 3 command-line-args) "bootstrap") > > + (equal (nth 4 command-line-args) "bootstrap")) > > + (multibyte-string-p default-directory)) > > + (setq default-directory (string-to-unibyte default-directory))) > > I'm not sure I understand this string-to-unibyte. > This call seems to only be correct if default-directory holds the > undecoded but multibyte name. > Why would we have an undecided yet multibyte name? This was a necessity before I removed this quirk from init_buffer: --- src/buffer.c 2013-10-29 14:46:23 +0000 +++ src/buffer.c 2013-10-31 16:57:18 +0000 @@ -5349,13 +5349,10 @@ init_buffer (void) len++; } + /* At this moment, we still don't know how to decode the directory + name. So, we keep the bytes in unibyte form so that file I/O + routines correctly get the original bytes. */ bset_directory (current_buffer, make_unibyte_string (pwd, len)); - if (! NILP (BVAR (&buffer_defaults, enable_multibyte_characters))) - /* At this moment, we still don't know how to decode the - directory name. So, we keep the bytes in multibyte form so - that ENCODE_FILE correctly gets the original bytes. */ - bset_directory - (current_buffer, string_to_multibyte (BVAR (current_buffer, directory))); /* Add /: to the front of the name if it would otherwise be treated as magic. */ After removing that, it's probably not needed anymore, since now default-directory should be a unibyte string from the very beginning. From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 31 15:41:29 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 19:41:29 +0000 Received: from localhost ([127.0.0.1]:55495 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vby7U-0002U8-TN for submit@debbugs.gnu.org; Thu, 31 Oct 2013 15:41:29 -0400 Received: from mtaout20.012.net.il ([80.179.55.166]:55343) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vby7S-0002To-F3 for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 15:41:27 -0400 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MVJ00100T6LSD00@a-mtaout20.012.net.il> for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 21:41:19 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVJ001C0TCVPV40@a-mtaout20.012.net.il>; Thu, 31 Oct 2013 21:41:19 +0200 (IST) Date: Thu, 31 Oct 2013 21:41:04 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: <83mwlpcmm7.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: monnier@iro.umontreal.ca Message-id: <83habxcjov.fsf@gnu.org> References: <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <83wqktcqdx.fsf@gnu.org> <83mwlpcmm7.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: Kenichi Handa , 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Thu, 31 Oct 2013 20:37:52 +0200 > From: Eli Zaretskii > Cc: 15260@debbugs.gnu.org > > > From: Stefan Monnier > > Cc: handa@gnu.org, 15260@debbugs.gnu.org > > Date: Thu, 31 Oct 2013 14:09:39 -0400 > > > > So you mean that we have: > > - charset-map-path is a multibyte string. > > - the file-name encoding uses a charset that's not yet loaded. > > Yes. > > > How do we get into such a state? > > Not sure about the details, since I don't really understand when Emacs > needs to load the charset map. Perhaps the map is needed only when we > need to encode a string, not for decoding? Actually, as can be seen from load_charset_map, we do different things when the map is needed for decoding and for encoding. So what probably happened was that when the file names in load-path etc. were decoded from cpNNNN, the map file was loaded and load_charset_map did whatever was necessary to set up the decoder for this encoding. Then, when we need to encode a file name using the same cpNNNN, the map file is loaded again, and load_charset_map now sets up the encoder. When the decoder was set up, charset-map-path was still in unibyte form, so the whole thing worked, because ENCODE_FILE doesn't try to encode unibyte strings. But once charset-map-path itself was decoded, the recursive call to 'openp' inside load_charset_map_from_file tried to encode it, and triggered infinite recursion. From debbugs-submit-bounces@debbugs.gnu.org Thu Oct 31 17:45:38 2013 Received: (at 15260) by debbugs.gnu.org; 31 Oct 2013 21:45:38 +0000 Received: from localhost ([127.0.0.1]:55696 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vc03e-0005oM-7j for submit@debbugs.gnu.org; Thu, 31 Oct 2013 17:45:38 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:37618 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vc03a-0005oB-Dj for 15260@debbugs.gnu.org; Thu, 31 Oct 2013 17:45:36 -0400 Received: from rgm by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1Vc03Y-0003dP-Jw; Thu, 31 Oct 2013 17:45:32 -0400 From: Glenn Morris To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> X-Spook: Maple propaganda number key Mena Nazi spies ASLET SEAL X-Ran: 40%DJvD2Q[%!I_t*INPC.XG>V}ZcZlIX1@x[|Y}>p1 (Eli Zaretskii's message of "Mon, 28 Oct 2013 18:47:32 +0200") Message-ID: User-Agent: Gnus (www.gnus.org), GNU Emacs (www.gnu.org/software/emacs/) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Spam-Score: -4.8 (----) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, Stefan Monnier , 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -4.8 (----) Eli Zaretskii wrote: > mule-cmds.el calls reset-language-environment, and language/english.el > calls set-language-info-alist; both have the effect of resetting > default-file-name-coding-system to latin-1 (!? an interesting > "default" for a Unicode-era Emacs, perhaps Handa-san could comment why > we still do that). I know nothing about this, but eg glib defaults to utf-8, which seems like a better default to me these days: https://developer.gnome.org/glib/stable/glib-Character-Set-Conversion.html#file-name-encodings From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 01 03:45:55 2013 Received: (at 15260) by debbugs.gnu.org; 1 Nov 2013 07:45:55 +0000 Received: from localhost ([127.0.0.1]:56359 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vc9QY-0004sI-KY for submit@debbugs.gnu.org; Fri, 01 Nov 2013 03:45:54 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:60447) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vc9QV-0004s2-Ec for 15260@debbugs.gnu.org; Fri, 01 Nov 2013 03:45:52 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0MVK00E00QUOW200@a-mtaout22.012.net.il> for 15260@debbugs.gnu.org; Fri, 01 Nov 2013 09:45:24 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVK00EFGQVMNKA0@a-mtaout22.012.net.il>; Fri, 01 Nov 2013 09:45:24 +0200 (IST) Date: Fri, 01 Nov 2013 09:45:09 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Glenn Morris Message-id: <83bo24d0qi.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: handa@gnu.org, monnier@iro.umontreal.ca, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Glenn Morris > Cc: Stefan Monnier , handa@gnu.org, 15260@debbugs.gnu.org > Date: Thu, 31 Oct 2013 17:45:32 -0400 > > Eli Zaretskii wrote: > > > mule-cmds.el calls reset-language-environment, and language/english.el > > calls set-language-info-alist; both have the effect of resetting > > default-file-name-coding-system to latin-1 (!? an interesting > > "default" for a Unicode-era Emacs, perhaps Handa-san could comment why > > we still do that). > > I know nothing about this, but eg glib defaults to utf-8, which seems > like a better default to me these days: Yes, probably. That's why I wrote that comment in parens. Fortunately, the final patch side-steps this issue altogether by keeping all the related file names as unibyte strings, so that the current defaults for encoding file names do not affect anything. So we can reason about the default independently of the issues in this bug. From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 01 05:28:02 2013 Received: (at 15260) by debbugs.gnu.org; 1 Nov 2013 09:28:02 +0000 Received: from localhost ([127.0.0.1]:56480 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VcB1N-0007Nd-LL for submit@debbugs.gnu.org; Fri, 01 Nov 2013 05:28:02 -0400 Received: from mtaout22.012.net.il ([80.179.55.172]:49345) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VcB1L-0007NL-4w for 15260@debbugs.gnu.org; Fri, 01 Nov 2013 05:28:00 -0400 Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0MVK00F00VLEPK00@a-mtaout22.012.net.il> for 15260@debbugs.gnu.org; Fri, 01 Nov 2013 11:27:52 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVK00F3MVME9JC0@a-mtaout22.012.net.il>; Fri, 01 Nov 2013 11:27:50 +0200 (IST) Date: Fri, 01 Nov 2013 11:27:36 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: <83iowdck1p.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: monnier@iro.umontreal.ca Message-id: <83txfwbhfb.fsf@gnu.org> References: <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <831u32drtn.fsf@gnu.org> <83sivhcodj.fsf@gnu.org> <83iowdck1p.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Thu, 31 Oct 2013 21:33:22 +0200 > From: Eli Zaretskii > Cc: 15260@debbugs.gnu.org > > > > +;; Make sure default-directory is unibyte when dumping. This is > > > +;; because we cannot decode and encode it correctly (since the locale > > > +;; environment is not, and should not be, set up). default-directory > > > +;; is used every time we call expand-file-name, which we do in every > > > +;; file primitive. So the only workable solution to support building > > > +;; in non-ASCII directories is to manipulate unibyte strings in the > > > +;; current locale's encoding. > > > +(if (and (or (equal (nth 3 command-line-args) "dump") > > > + (equal (nth 4 command-line-args) "dump") > > > + (equal (nth 3 command-line-args) "bootstrap") > > > + (equal (nth 4 command-line-args) "bootstrap")) > > > + (multibyte-string-p default-directory)) > > > + (setq default-directory (string-to-unibyte default-directory))) > > > > I'm not sure I understand this string-to-unibyte. > > This call seems to only be correct if default-directory holds the > > undecoded but multibyte name. > > Why would we have an undecided yet multibyte name? > > This was a necessity before I removed this quirk from init_buffer: > > --- src/buffer.c 2013-10-29 14:46:23 +0000 > +++ src/buffer.c 2013-10-31 16:57:18 +0000 > @@ -5349,13 +5349,10 @@ init_buffer (void) > len++; > } > > + /* At this moment, we still don't know how to decode the directory > + name. So, we keep the bytes in unibyte form so that file I/O > + routines correctly get the original bytes. */ > bset_directory (current_buffer, make_unibyte_string (pwd, len)); > - if (! NILP (BVAR (&buffer_defaults, enable_multibyte_characters))) > - /* At this moment, we still don't know how to decode the > - directory name. So, we keep the bytes in multibyte form so > - that ENCODE_FILE correctly gets the original bytes. */ > - bset_directory > - (current_buffer, string_to_multibyte (BVAR (current_buffer, directory))); > > /* Add /: to the front of the name > if it would otherwise be treated as magic. */ > > After removing that, it's probably not needed anymore, since now > default-directory should be a unibyte string from the very beginning. Would you prefer that we error out of default-directory is not a unibyte string at that point in loadup.el? From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 01 08:33:14 2013 Received: (at 15260) by debbugs.gnu.org; 1 Nov 2013 12:33:14 +0000 Received: from localhost ([127.0.0.1]:56829 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VcDuc-0004t5-0p for submit@debbugs.gnu.org; Fri, 01 Nov 2013 08:33:14 -0400 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:6277) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VcDuZ-0004sp-Qm for 15260@debbugs.gnu.org; Fri, 01 Nov 2013 08:33:12 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EABK/CFFMCrb4/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kiB4GwS2RCgOkeoFegxM X-IPAS-Result: Av4EABK/CFFMCrb4/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kiB4GwS2RCgOkeoFegxM X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="36843023" Received: from 76-10-182-248.dsl.teksavvy.com (HELO pastel.home) ([76.10.182.248]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 01 Nov 2013 08:33:05 -0400 Received: by pastel.home (Postfix, from userid 20848) id 321A16301C; Fri, 1 Nov 2013 08:33:04 -0400 (EDT) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters Message-ID: References: <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <831u32drtn.fsf@gnu.org> <83sivhcodj.fsf@gnu.org> <83iowdck1p.fsf@gnu.org> <83txfwbhfb.fsf@gnu.org> Date: Fri, 01 Nov 2013 08:33:04 -0400 In-Reply-To: <83txfwbhfb.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 01 Nov 2013 11:27:36 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) > Would you prefer that we error out of default-directory is not a > unibyte string at that point in loadup.el? I'd prefer to either not do anything, or issue a warning, or error out, yes. Stefan From debbugs-submit-bounces@debbugs.gnu.org Fri Nov 01 09:58:18 2013 Received: (at 15260) by debbugs.gnu.org; 1 Nov 2013 13:58:18 +0000 Received: from localhost ([127.0.0.1]:57504 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VcFEw-00075g-63 for submit@debbugs.gnu.org; Fri, 01 Nov 2013 09:58:18 -0400 Received: from fencepost.gnu.org ([208.118.235.10]:34056 ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VcFEt-00075X-Di for 15260@debbugs.gnu.org; Fri, 01 Nov 2013 09:58:16 -0400 Received: from fl1-110-233-32-186.iba.mesh.ad.jp ([110.233.32.186]:56906 helo=shatin) by fencepost.gnu.org with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1VcFEr-0004yg-TP; Fri, 01 Nov 2013 09:58:14 -0400 Received: from handa by shatin with local (Exim 4.80) (envelope-from ) id 1VcFEm-0000W2-4G; Fri, 01 Nov 2013 22:58:08 +0900 From: Kenichi Handa To: Stefan Monnier Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-Reply-To: (message from Stefan Monnier on Mon, 28 Oct 2013 21:35:00 -0400) Date: Fri, 01 Nov 2013 22:58:08 +0900 Message-ID: <87mwlons0f.fsf@gnu.org> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 15260 Cc: eliz@gnu.org, 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) In article , Stefan Monnier writes: > > emacs = EMACSLOADPATH="$(abs_lisp)" LC_ALL=C "$(EMACS)" $(EMACSOPT) > > ^^^^^^^^ > > Does anyone know or remember why we set LC_ALL=C while running > > commands in lisp/ (and the same in leim/)? > IIRC the issue was to avoid things like misdetecting coding-systems > because of the user's locale setting, in the files we load/compile. As far as I remember, yes. > IOW, it was to work around bugs (e.g. missing coding: cookie) and is > likely unneeded nowadays. I agree. --- Kenichi Handa handa@gnu.org From debbugs-submit-bounces@debbugs.gnu.org Mon Nov 04 12:36:06 2013 Received: (at 15260-done) by debbugs.gnu.org; 4 Nov 2013 17:36:06 +0000 Received: from localhost ([127.0.0.1]:34445 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VdO4L-0000te-62 for submit@debbugs.gnu.org; Mon, 04 Nov 2013 12:36:05 -0500 Received: from mtaout20.012.net.il ([80.179.55.166]:41843) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VdO4H-0000sV-4Y for 15260-done@debbugs.gnu.org; Mon, 04 Nov 2013 12:36:02 -0500 Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0MVR00M0022YGI00@a-mtaout20.012.net.il> for 15260-done@debbugs.gnu.org; Mon, 04 Nov 2013 19:35:54 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVR00MYW27UEO20@a-mtaout20.012.net.il>; Mon, 04 Nov 2013 19:35:54 +0200 (IST) Date: Mon, 04 Nov 2013 19:35:50 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: <83sivhcodj.fsf@gnu.org> X-012-Sender: halo1@inter.net.il To: monnier@iro.umontreal.ca, rgm@gnu.org Message-id: <83ob6083yh.fsf@gnu.org> References: <83y55ih6g3.fsf@gnu.org> <81ppqums9o.fsf@fencepost.gnu.org> <83hac5h1ha.fsf@gnu.org> <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <831u32drtn.fsf@gnu.org> <83sivhcodj.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260-done Cc: 15260-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > Date: Thu, 31 Oct 2013 19:59:52 +0200 > From: Eli Zaretskii > Cc: 15260@debbugs.gnu.org > > If no problems pop up, I will commit this in a few days. No further comments, and I got fed up with resolving merge conflicts every day, so I committed the changes, and I'm marking this bug done. Thanks to everybody for their feedback and support. From debbugs-submit-bounces@debbugs.gnu.org Mon Nov 04 12:37:25 2013 Received: (at 15260) by debbugs.gnu.org; 4 Nov 2013 17:37:25 +0000 Received: from localhost ([127.0.0.1]:34454 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VdO5b-0000xp-6X for submit@debbugs.gnu.org; Mon, 04 Nov 2013 12:37:23 -0500 Received: from mtaout23.012.net.il ([80.179.55.175]:49240) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VdO5X-0000xQ-Bv for 15260@debbugs.gnu.org; Mon, 04 Nov 2013 12:37:20 -0500 Received: from conversion-daemon.a-mtaout23.012.net.il by a-mtaout23.012.net.il (HyperSendmail v2007.08) id <0MVR00M00278I100@a-mtaout23.012.net.il> for 15260@debbugs.gnu.org; Mon, 04 Nov 2013 19:37:13 +0200 (IST) Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout23.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MVR00MOI29XAXB0@a-mtaout23.012.net.il>; Mon, 04 Nov 2013 19:37:10 +0200 (IST) Date: Mon, 04 Nov 2013 19:37:06 +0200 From: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters In-reply-to: X-012-Sender: halo1@inter.net.il To: Stefan Monnier Message-id: <83mwlk83wd.fsf@gnu.org> References: <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <831u32drtn.fsf@gnu.org> <83sivhcodj.fsf@gnu.org> <83iowdck1p.fsf@gnu.org> <83txfwbhfb.fsf@gnu.org> X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: 15260 Cc: 15260@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Eli Zaretskii List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.0 (+) > From: Stefan Monnier > Cc: 15260@debbugs.gnu.org > Date: Fri, 01 Nov 2013 08:33:04 -0400 > > > Would you prefer that we error out of default-directory is not a > > unibyte string at that point in loadup.el? > > I'd prefer to either not do anything, or issue a warning, or error > out, yes. I eventually opted for erroring out, mostly to be able to catch any unforseen problems and use cases I missed. From debbugs-submit-bounces@debbugs.gnu.org Mon Nov 04 13:38:33 2013 Received: (at 15260-done) by debbugs.gnu.org; 4 Nov 2013 18:38:33 +0000 Received: from localhost ([127.0.0.1]:34526 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VdP2n-0002WS-0i for submit@debbugs.gnu.org; Mon, 04 Nov 2013 13:38:33 -0500 Received: from ironport2-out.teksavvy.com ([206.248.154.182]:47322) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VdP2k-0002WE-54 for 15260-done@debbugs.gnu.org; Mon, 04 Nov 2013 13:38:30 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EABK/CFFFxLzd/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kiB4GwS2RCgOIYZwZgV6DFQ X-IPAS-Result: Av4EABK/CFFFxLzd/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLNBIUGA0kiB4GwS2RCgOIYZwZgV6DFQ X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="37063942" Received: from 69-196-188-221.dsl.teksavvy.com (HELO pastel.home) ([69.196.188.221]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 04 Nov 2013 13:38:23 -0500 Received: by pastel.home (Postfix, from userid 20848) id AFCDA60D75; Mon, 4 Nov 2013 13:38:22 -0500 (EST) From: Stefan Monnier To: Eli Zaretskii Subject: Re: bug#15260: cannot build in a directory with non-ascii characters Message-ID: References: <83bo2dgq36.fsf@gnu.org> <837gd1gpcz.fsf@gnu.org> <8361slgnhk.fsf@gnu.org> <83zjpwfp3l.fsf@gnu.org> <83mwlug0cd.fsf@gnu.org> <83bo2af4hh.fsf@gnu.org> <8361shfil7.fsf@gnu.org> <834n81fdp8.fsf@gnu.org> <83zjpseo1g.fsf@gnu.org> <838uxaei43.fsf@gnu.org> <831u32drtn.fsf@gnu.org> <83sivhcodj.fsf@gnu.org> <83ob6083yh.fsf@gnu.org> Date: Mon, 04 Nov 2013 13:38:22 -0500 In-Reply-To: <83ob6083yh.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 04 Nov 2013 19:35:50 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 15260-done Cc: rgm@gnu.org, 15260-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.3 (/) > Thanks to everybody for their feedback and support. Thank you, Eli, Stefan From unknown Fri Jun 20 07:11:17 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Tue, 03 Dec 2013 12:24:03 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator