GNU bug report logs - #56342
TRAMP (sh) issues way too many commands, thus being very slow over high-ping networks

Previous Next

Full log

View this message in rfc822 format

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Pogonyshev <pogonyshev <at> gmail.com>
Cc: 56342 <at> debbugs.gnu.org
Subject: bug#56342: TRAMP (sh) issues way too many commands, thus being very slow over high-ping networks
Date: Sat, 02 Jul 2022 17:58:25 +0200

Paul Pogonyshev <pogonyshev <at> gmail.com> writes:

Hi Paul,

> 1) check if connection is alive (`echo are you awake');
> 2) test if the file exists;
> 3) creating a temporary file for the chunk to be inserted; I guess it
> tries until it finds an unused filename, e.g. here it seems to be done
> after `test -e /tmp/tramp.OD3cCu', which doesn't exist;
> 4) 'touch' on the temporary file, presumably to create it;
> 5) 'chmod' on the temporary, presumably so that other users cannot
> read it;
> 6) copying the requested chunk from the full file into the temporary
> (using `dd');
> 7) finding the real name of the temporary with `readlink';
> 8) finding attributes of the temporary with `stat';
> 9) gzipping the temporary for transmition from the remote to the local
> machine;
> 10) testing if the temporary is a directory (WTF?);
> 11) removing the temporary.
>
> I guess it should be obvious that this is a bit too much for one
> `insert-file-contents' call.

In general, I agree. However, some of the commands are caused by
primitive file operations, like file-exists-p. Tramp cannot know what
will be the next call, and it doesn't have all the opportunities to
optimize, compared with the overall picture you see in the eleven steps.

> Suggested improvements:
>
> * TRAMP should issue just one `stat' command to find out most of the
> things about a file: whether it exists, if it is a directory, its real
> name when dereferencing links and whatever stats it is used to find
> now; from `$ stat --help' this seems to be possible. In other words,
> TRAMP shouldn't use simple commands like `test -e': any ping, even
> nominal, will negate any gains from using a tad faster command.
> Instead, if it needs to find anything about a file, it should ask the
> remote about as many things as possible in one go: it is very likely
> that the additional information will be needed soon and even if not,
> this is basically free compared to ping anyway.

Not all remote hosts carry a stat command, and not all existing stat's
are GNU compatible. But yes, if possible, Tramp shall gather as much
information in one run, and cache the results for further use.

I will see what could be done. Will come back with a proposal next days
(note that this will be for Emacs 29, ie git master).

> * TRAMP code should prefer the approach "try do something and handle
> resulting errors" where possible. For example, don't check if the file
> exists, try to read it right away and handle failures properly. Code
> like `(when (file-exists-p ...) do-something)' adds an unnecessary
> command call and creates a racing condition anyway. Also, error-free
> requests should be more frequent, so they should be the main
> optimization goal. I'm not sure if it is applicable to TRAMP itself
> and doesn't come from a higher level, though.

Indeed, this is not Tramp's responsibility. Tramp is a stupid
library. If there is a call for file-exists-p, it must return the
answer. It doesn't know what will be the next request. So I'm rather
pesimistic that Tramp can improve here.

Best regards, Michael.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.