GNU bug report logs - #19412
24.3; ido-write-file sometimes writes to a different directory than it says it will

Previous Next

To reply to this bug, email your comments to 19412 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Don Morrison <dfm <at> ringing.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.3; ido-write-file sometimes writes to a different directory than
 it says it will
Date: Fri, 19 Dec 2014 15:21:20 -0500

This bug report will be sent to the Bug-GNU-Emacs mailing list
and the GNU bug tracker at debbugs.gnu.org.  Please check that
the From: line contains a valid email address.  After a delay of up
to one day, you should receive an acknowledgment at that address.

Please write in English if possible, as the Emacs maintainers
usually do not have translators for other languages.

Please describe exactly what actions triggered the bug, and
the precise symptoms of the bug.  If you can, give a recipe
starting from `emacs -Q':



Create a file, ~/mumble.frotz, containing some text.

emacs -Q

M-x ido-mode

M-x ido-everywhere

C-x C-f ~/mumble.frotz

C-x C-w /tmp/ C-f

You are now sitting at a prompt that appears to be saying if confirmed
it will write the file into /tmp/, with the file name mumble.frotz
implied.

Hit the carriage return key to confirm it.

Note that it is trying to write it into ~/mumble.frotz, not /tmp/mumble.frotz




If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
/usr/share/emacs/24.3/etc/DEBUG.


In GNU Emacs 24.3.1 (x86_64-pc-linux-gnu, GTK+ Version 3.10.7)
 of 2014-03-07 on lamiak, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11501000
System Description: Linux Mint 17 Qiana

Configured using:
 `configure '--build' 'x86_64-linux-gnu' '--build' 'x86_64-linux-gnu'
 '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib'
 '--localstatedir=/var/lib' '--infodir=/usr/share/info'
 '--mandir=/usr/share/man' '--with-pop=yes'
 '--enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.3/site-lisp:/usr/share/emacs/site-lisp'
 '--with-crt-dir=/usr/lib/x86_64-linux-gnu' '--with-x=yes'
 '--with-x-toolkit=gtk3' '--with-toolkit-scroll-bars'
 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fstack-protector
 --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall'
 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro'
 'CPPFLAGS=-D_FORTIFY_SOURCE=2''

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Fundamental

Minor modes in effect:
  shell-dirtrack-mode: t
  ido-everywhere: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<help-echo> <help-echo> M-x i d o - e v e r y w h e
r e <return> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> m o d e <return> M-x i d o - e v e r y
w h e r e <return> C-x C-f m u m <return> C-x C-w /
t m p / C-f <return> C-g <help-echo> M-x r e p o r
t - e m a c s - b u g <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Ido mode enabled
Ido-Everywhere mode enabled
mumble.frotz has auto save data; consider M-x recover-this-file
Quit
Type C-x 1 to delete the help window.

Load-path shadows:
None found.

Features:
(shadow sort mail-extr help-mode emacsbug message rfc822 mml easymenu
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils
tramp-cache tramp tramp-compat auth-source eieio byte-opt bytecomp
byte-compile cconv gnus-util mm-util mail-prsvr password-cache
tramp-loaddefs shell pcomplete comint ansi-color ring format-spec advice
help-fns cl-lib advice-preload cus-start cus-load ido time-date tooltip
ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face macroexp files text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty emacs)

Message #10 received at 19412 <at> debbugs.gnu.org (full text, mbox):

From: "Ryan C. Thompson" <rct <at> thompsonclan.org>
To: 19412 <at> debbugs.gnu.org
Subject: Re: 24.3; ido-write-file sometimes writes to a different directory
 than, it says it will
Date: Sun, 3 Nov 2019 14:48:15 -0800

I believe I have discovered the cause of this bug. It can be reproduced 
by evaluating the following code and then hitting RET:

(read-file-name-default "Write file: " "/tmp/" "~/mumble.frotz" nil nil nil)

This will return "~/mumble.frotz" rather than "/tmp/". Ido triggers this 
issue by causing "read-file-name-default" to be called with arguments 
like the above when triggered to fall back to non-ido completion as 
described in the inital report. The crux of the issue is that the 
initial directory, "/tmp/", is treated as *not* user-entered, so 
pressing RET on it returns the default filename instead (which is 
populated from the buffer file name).

This is not trivial to fix, because ido isn't calling 
"read-file-name-default" directly. In the example given, it's 
let-binding "default-directory" to "/tmp/" and then doing:

(call-interactively 'write-file)

And then the interactive form calls:

(read-file-name "Write file: ")

which then picks up DIR and DEFAULT-FILENAME from "default-directory" 
and "buffer-file-name" respectively. So this isn't a case of just fixing 
a function call somewhere. One possible solution would be to also 
let-bind "buffer-file-name" to nil, in which case DEFAULT-FILENAME gets 
set to DIR. That would work for the case of "write-file", but I don't 
know if it would work for other functions that read file names.

Lastly, I'm guessing that the original reporter ran into this issue 
because they were using C-f RET to select "/tmp/" within ido completion, 
since RET would just select the first file or subdirectory on the list. 
They should be using C-j to do that.

I might add a hack to my ido-completing-read-plus package to fix this 
edge case, if I can figure out a reasonably clean way to fix it.

Message #13 received at 19412 <at> debbugs.gnu.org (full text, mbox):

From: "Ryan C. Thompson" <rct <at> thompsonclan.org>
To: 19412 <at> debbugs.gnu.org
Subject: Re: 24.3; ido-write-file sometimes writes to a different directory
 than, it says it will
Date: Mon, 4 Nov 2019 06:52:19 -0800

[Message part 1 (text/plain, inline)]

I think I have found a fix for this issue. A patch for ido.el is 
attached. The solution is essentially to simulate re-typing the current 
ido input into the fallback command's prompt rather than modifying 
dynamic variables to trick the fallback command into starting in the 
right place.

Note that this is NOT thoroughly tested yet. It seems to work for the 
specific case described in this bug (ido-write-file), but I need to test 
it for some time to make sure it isn't breaking other cases at the same 
time. I will use this fix in my Emacs for some time and report back.

[0001-Ensure-correct-behavior-in-ido-file-fallback-complet.patch (text/plain, attachment)]

Message #16 received at 19412 <at> debbugs.gnu.org (full text, mbox):

From: "Ryan C. Thompson" <rct <at> thompsonclan.org>
To: 19412 <at> debbugs.gnu.org
Subject: Re: 24.3; ido-write-file sometimes writes to a different directory
 than, it says it will
Date: Mon, 4 Nov 2019 07:55:46 -0800

On further testing, I've determined that this patch does not handle all 
the relevant code paths. At least ido-file-internal, ido-read-file-name, 
and ido-read-directory-name all need to be handled, each in a slightly 
different way. I'll work on it.

In any case, please DO NOT install the current patch as is.

Message #19 received at 19412 <at> debbugs.gnu.org (full text, mbox):

From: "Ryan C. Thompson" <rct <at> thompsonclan.org>
To: 19412 <at> debbugs.gnu.org
Subject: Re: 24.3; ido-write-file sometimes writes to a different directory
 than, it says it will
Date: Wed, 11 Mar 2020 12:46:44 -0400

[Message part 1 (text/plain, inline)]

Ok, I think I have a working patch for this issue. I patched 
ido-file-internal, ido-read-file-name, and ido-read-directory-name, and 
I think that's all the code paths that need to be fixed. Now the test 
case described in the original report produces the correct result for 
me: writing to /tmp/mumble.frotz.

However, I should note that this is still relatively untested. I will 
test it out and try to make sure it doesn't cause any unexpected issues 
before I recommend merging it.

[0001-Fix-default-directory-handling-in-ido-file-fallback-.patch (text/plain, attachment)]

Message #22 received at 19412 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: "Ryan C. Thompson" <rct <at> thompsonclan.org>
Cc: 19412 <at> debbugs.gnu.org
Subject: Re: bug#19412: 24.3; ido-write-file sometimes writes to a different
 directory than, it says it will
Date: Wed, 12 Aug 2020 09:44:22 -0700

"Ryan C. Thompson" <rct <at> thompsonclan.org> writes:

> Ok, I think I have a working patch for this issue. I patched ido-file-internal,
> ido-read-file-name, and ido-read-directory-name, and I think that's all the code
> paths that need to be fixed. Now the test case described in the original report
> produces the correct result for me: writing to /tmp/mumble.frotz.
>
> However, I should note that this is still relatively untested. I will test it
> out and try to make sure it doesn't cause any unexpected issues before I
> recommend merging it.

Any updates here?  Did you give it more testing?

Best regards,
Stefan Kangas

Message #25 received at 19412 <at> debbugs.gnu.org (full text, mbox):

From: "Ryan C. Thompson" <rct <at> thompsonclan.org>
To: 28513 <at> debbugs.gnu.org
Cc: 19412 <at> debbugs.gnu.org
Subject: Re: bug#28513
Date: Sun, 10 Jan 2021 18:07:23 -0500

Hello again,

I finally got around to testing this patch. I've been using it for over 
a week now and I haven't observed any unexpected behaviors or issues as 
a result. And of course, it does indeed resolve the bug as reported 
here. I believe my patch is ready to merge. Again, for more details on 
how my patch works, see my investigation in #19412.

Regards,

Ryan Thompson

On 1/1/21 4:51 PM, Ryan C. Thompson wrote:
> Hello,
>
> I believe this bug (#28513) is a duplicate of #19412. In the 
> discussion for #19412 I have investigated the problem and proposed a 
> fix that doesn't require special-casing anything. However, due to 
> being quite busy since then I have not had the time to properly test 
> my fix. Briefly, the fix is to pass all the original args to the 
> fallback function unchanged and then use "minibuffer-with-setup-hook" 
> to simulate deleting the initial input and then typing whatever the 
> user currently had typed into ido before they triggered the fallback.
>
> I have rebased the patch from that thread onto the current master and 
> attached it. To reiterate, this patch should be tested before merging.
>
> Regards,
>
> Ryan
>

Message #28 received at 19412 <at> debbugs.gnu.org (full text, mbox):

From: "Ryan C. Thompson" <rct <at> thompsonclan.org>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 19412 <at> debbugs.gnu.org
Subject: Re: bug#19412: 24.3; ido-write-file sometimes writes to a different
 directory than, it says it will
Date: Sun, 10 Jan 2021 18:12:20 -0500

Hi Stefan,

It's been a while, but I've fixed up my patch and given it some testing, 
and it seems to work on for me. However, in the meantime, this issue has 
recently been "fixed" by special-casing write-file in ido.el, as seen in 
#28513. So if you want to install my patch now, you'll need to install 
the version attached to that thread. That version reverts the other fix, 
since of course they are not compatible, and would be redundant even if 
they were.

Regards,

Ryan Thompson

On 8/12/20 12:44 PM, Stefan Kangas wrote:
> "Ryan C. Thompson" <rct <at> thompsonclan.org> writes:
>
>> Ok, I think I have a working patch for this issue. I patched ido-file-internal,
>> ido-read-file-name, and ido-read-directory-name, and I think that's all the code
>> paths that need to be fixed. Now the test case described in the original report
>> produces the correct result for me: writing to /tmp/mumble.frotz.
>>
>> However, I should note that this is still relatively untested. I will test it
>> out and try to make sure it doesn't cause any unexpected issues before I
>> recommend merging it.
> Any updates here?  Did you give it more testing?
>
> Best regards,
> Stefan Kangas

Message #31 received at 19412 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: "Ryan C. Thompson" <rct <at> thompsonclan.org>
Cc: 19412 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#19412: 24.3; ido-write-file sometimes writes to a different
 directory than, it says it will
Date: Mon, 11 Jan 2021 15:14:50 +0100

"Ryan C. Thompson" <rct <at> thompsonclan.org> writes:

> It's been a while, but I've fixed up my patch and given it some
> testing, and it seems to work on for me. However, in the meantime,
> this issue has recently been "fixed" by special-casing write-file in
> ido.el, as seen in #28513. So if you want to install my patch now,
> you'll need to install the version attached to that thread. That
> version reverts the other fix, since of course they are not
> compatible, and would be redundant even if they were.

It looks like a more thorough fix.  However:

+          (minibuffer-with-setup-hook
+              (:append
+               (lambda ()
+                 ;; Clear out whatever started in the minibuffer and
+                 ;; replace it with what the user had already entered
+                 ;; into ido.
+                 (delete-minibuffer-contents)
+                 (insert (abbreviate-file-name ido-current-directory))))
+            (call-interactively this-command))))

I'd be worried that this would step on other modifications the user may
be doing from the minibuffer setup.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #34 received at 19412 <at> debbugs.gnu.org (full text, mbox):

From: "Ryan C. Thompson" <rct <at> thompsonclan.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 19412 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#19412: 24.3; ido-write-file sometimes writes to a different
 directory than, it says it will
Date: Mon, 11 Jan 2021 09:28:06 -0500

On 1/11/21 9:14 AM, Lars Ingebrigtsen wrote:
> "Ryan C. Thompson" <rct <at> thompsonclan.org> writes:
>
>> It's been a while, but I've fixed up my patch and given it some
>> testing, and it seems to work on for me. However, in the meantime,
>> this issue has recently been "fixed" by special-casing write-file in
>> ido.el, as seen in #28513. So if you want to install my patch now,
>> you'll need to install the version attached to that thread. That
>> version reverts the other fix, since of course they are not
>> compatible, and would be redundant even if they were.
> It looks like a more thorough fix.  However:
>
> +          (minibuffer-with-setup-hook
> +              (:append
> +               (lambda ()
> +                 ;; Clear out whatever started in the minibuffer and
> +                 ;; replace it with what the user had already entered
> +                 ;; into ido.
> +                 (delete-minibuffer-contents)
> +                 (insert (abbreviate-file-name ido-current-directory))))
> +            (call-interactively this-command))))
>
> I'd be worried that this would step on other modifications the user may
> be doing from the minibuffer setup.

The only case where this would step on other minibuffer setup code is 
when that code makes modifications to the initial input in the 
minibuffer, since those modifications would be deleted and replaced. 
However, in this case I'd argue that is the correct behavior. The point 
of this code is to fall back from ido completion to standard emacs 
completion while preserving the current input. That means that any setup 
hook that modifies the initial contents of the minibuffer has *already* 
run at the start of ido completion and should not run *again* here. 
Effectively, we want to pretend that we are continuing the same 
completion session with a different completion system, even though we 
are actually starting a new completion session. And to do that, we need 
to preserve the user's current input verbatim when falling back.

Message #37 received at 19412 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: "Ryan C. Thompson" <rct <at> thompsonclan.org>
Cc: 19412 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#19412: 24.3; ido-write-file sometimes writes to a different
 directory than, it says it will
Date: Mon, 11 Jan 2021 19:43:37 +0100

"Ryan C. Thompson" <rct <at> thompsonclan.org> writes:

> And to do that, we need to preserve the user's current input
> verbatim when falling back.

Sounds reasonable, but who knows what people are doing in the setup
hooks there?  So I worry about making a change like this, perhaps overly
much.

I guess the only way to find out whether it breaks a lot of stuff or
not is to try it out and get feedback.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #40 received at 19412 <at> debbugs.gnu.org (full text, mbox):

From: "Ryan C. Thompson" <rct <at> thompsonclan.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 19412 <at> debbugs.gnu.org, Stefan Kangas <stefan <at> marxist.se>
Subject: Re: bug#19412: 24.3; ido-write-file sometimes writes to a different
 directory than, it says it will
Date: Mon, 11 Jan 2021 13:50:46 -0500

On 1/11/21 1:43 PM, Lars Ingebrigtsen wrote:
> "Ryan C. Thompson" <rct <at> thompsonclan.org> writes:
>
>> And to do that, we need to preserve the user's current input
>> verbatim when falling back.
> Sounds reasonable, but who knows what people are doing in the setup
> hooks there?  So I worry about making a change like this, perhaps overly
> much.
>
> I guess the only way to find out whether it breaks a lot of stuff or
> not is to try it out and get feedback.
>
Yes, it's definitely an invasive change, and more properly characterized 
as a work-around than a true fix, so the worry isn't unwarranted. I'm 
pretty sure the code is correct (or as close to correct as possible 
given the "API" limitations we're working with), but it's always 
possible there's some edge case I haven't anticipated.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.