GNU bug report logs - #26011
26.0.50; tramp should respect large-file-warning-threshold

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 26011 in the body.
You can then email your comments to 26011 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Robert Marshall <robert.marshall <at> codethink.co.uk>
To: bug-gnu-emacs <at> gnu.org
Subject: 26.0.50; tramp should respect large-file-warning-threshold
Date: Tue, 07 Mar 2017 14:21:05 +0000

If from dired you attempt to copy a file to a tramp ssh dired (I have
dired-dwim-target set to t) and that file is very large, emacs will pause
for some time and eventually stop with:

tramp-file-name-handler: Memory exhausted--use C-x s then exit and
restart Emacs

If tramp is going to open the file and it is large I think it should
warn the user (respecting large-file-warning-threshold?) rather than
going ahead without confirmation and erroring with an alarming message!

Robert

In GNU Emacs 26.0.50 (build 4, x86_64-unknown-linux-gnu, GTK+ Version 2.24.25)
 of 2017-03-06 built on ct-lt-579
Repository revision: 0fae08d0072f74d97ca70b91a4d46d8d28a03952
Windowing system distributor 'The X.Org Foundation', version 11.0.11604000
System Description:	Debian GNU/Linux 8.7 (jessie)


Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS GSETTINGS NOTIFY GNUTLS LIBXML2
FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS GTK2 X11

Important settings:
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

Message #8 received at 26011 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Robert Marshall <robert.marshall <at> codethink.co.uk>
Cc: 26011 <at> debbugs.gnu.org
Subject: Re: bug#26011: 26.0.50;
 tramp should respect large-file-warning-threshold
Date: Tue, 07 Mar 2017 17:57:17 +0100

Robert Marshall <robert.marshall <at> codethink.co.uk> writes:

Hi Robert,

> If from dired you attempt to copy a file to a tramp ssh dired (I have
> dired-dwim-target set to t) and that file is very large, emacs will pause
> for some time and eventually stop with:
>
> tramp-file-name-handler: Memory exhausted--use C-x s then exit and
> restart Emacs
>
> If tramp is going to open the file and it is large I think it should
> warn the user (respecting large-file-warning-threshold?) rather than
> going ahead without confirmation and erroring with an alarming message!

Well, this happens in case Tramp inserts the file into a temporary
buffer.  What about this patch:

--8<---------------cut here---------------start------------->8---
diff --git a/lisp/tramp-sh.el b/lisp/tramp-sh.el
index 071ef79..8561962 100644
--- a/lisp/tramp-sh.el
+++ b/lisp/tramp-sh.el
@@ -2147,6 +2147,11 @@ file names."
 First arg OP is either `copy' or `rename' and indicates the operation.
 FILENAME is the source file, NEWNAME the target file.
 KEEP-DATE is non-nil if NEWNAME should have the same timestamp as FILENAME."
+  ;; Check, whether file is too large.  Emacs checks in `insert-file-1'
+  ;; and `find-file-noselect', but that's not called here.
+  (abort-if-file-too-large
+   (tramp-compat-file-attribute-size (file-attributes (file-truename filename)))
+   (symbol-name op) filename)
   ;; We must disable multibyte, because binary data shall not be
   ;; converted.  We don't want the target file to be compressed, so we
   ;; let-bind `jka-compr-inhibit' to t.  `epa-file-handler' shall not
--8<---------------cut here---------------end--------------->8---

> Robert

Best regards, Michael.

Message #11 received at 26011 <at> debbugs.gnu.org (full text, mbox):

From: Robert Marshall <robert.marshall <at> codethink.co.uk>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 26011 <at> debbugs.gnu.org
Subject: Re: bug#26011: 26.0.50;
 tramp should respect large-file-warning-threshold
Date: Wed, 08 Mar 2017 12:46:57 +0000

Michael Albinus <michael.albinus <at> gmx.de> writes:

> Robert Marshall <robert.marshall <at> codethink.co.uk> writes:
>
> Hi Robert,
>
>> If from dired you attempt to copy a file to a tramp ssh dired (I have
>> dired-dwim-target set to t) and that file is very large, emacs will pause
>> for some time and eventually stop with:
>>
>> tramp-file-name-handler: Memory exhausted--use C-x s then exit and
>> restart Emacs
>>
>> If tramp is going to open the file and it is large I think it should
>> warn the user (respecting large-file-warning-threshold?) rather than
>> going ahead without confirmation and erroring with an alarming message!
>
> Well, this happens in case Tramp inserts the file into a temporary
> buffer.  What about this patch:
>
> diff --git a/lisp/tramp-sh.el b/lisp/tramp-sh.el
> index 071ef79..8561962 100644
> --- a/lisp/tramp-sh.el
> +++ b/lisp/tramp-sh.el
> @@ -2147,6 +2147,11 @@ file names."
>  First arg OP is either `copy' or `rename' and indicates the operation.
>  FILENAME is the source file, NEWNAME the target file.
>  KEEP-DATE is non-nil if NEWNAME should have the same timestamp as FILENAME."
> +  ;; Check, whether file is too large.  Emacs checks in `insert-file-1'
> +  ;; and `find-file-noselect', but that's not called here.
> +  (abort-if-file-too-large
> +   (tramp-compat-file-attribute-size (file-attributes (file-truename filename)))
> +   (symbol-name op) filename)
>    ;; We must disable multibyte, because binary data shall not be
>    ;; converted.  We don't want the target file to be compressed, so we
>    ;; let-bind `jka-compr-inhibit' to t.  `epa-file-handler' shall not
>

Yes that's better thank you, you still get the transient error if I
continue (my file was 4Gig) but I guess it's too late to do anything
else by the time it gets there


Robert

Message #16 received at 26011-done <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Robert Marshall <robert.marshall <at> codethink.co.uk>
Cc: 26011-done <at> debbugs.gnu.org
Subject: Re: bug#26011: 26.0.50;
 tramp should respect large-file-warning-threshold
Date: Wed, 08 Mar 2017 17:38:50 +0100

Robert Marshall <robert.marshall <at> codethink.co.uk> writes:

Hi Robert,

> Yes that's better thank you, you still get the transient error if I
> continue (my file was 4Gig) but I guess it's too late to do anything
> else by the time it gets there

Once you confirm to copy/rename a huge file, its contents is loaded into
a temporary buffer. And yes, if it's too large, memory is exhausted.
There's nothing Tramp could know in advance, and prevent it.

If you handle regularly such huge files, Tramp's "scp" method is
superiour to "ssh".

I've pushed the patch to both Emacs and Tramp repositories, closing the bug.

> Robert

Best regards, Michael.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.