GNU bug report logs - #16361
[wishlist] improve freshness checking in compile cache

Previous Next

Full log

View this message in rfc822 format

From: Zefram <zefram <at> fysh.org>
To: 16361 <at> debbugs.gnu.org
Subject: bug#16361: compile cache confused about file identity
Date: Wed, 13 May 2015 12:07:39 +0100

Mark H Weaver wrote:
>You could make the same complaint about 'make', 'rsync', or any number
>of other programs.

Not really.  make does use this type of freshness check, but it's used
in a specific situation where the freshness issue is immediately obvious
and is part of the program's visible primary concern.  That's quite
unlike guile's compile cache, which as the name suggests is a cache.
It's meant to be unobtrusive, and the cache semantics are not a direct
part of the transaction that is ostensibly taking place, of running
a program that happens to be written in Scheme.  Those circumstances,
of running an arbitrary program, are much broader than circumstances in
which make's freshness checks become relevant.  make also gets a pass
from having always worked this way, whereas guile used to not cache
compilations.  rsync, by contrast, does not use this type of freshness
checking; I believe it uses a hash mechanism.

>                    It's true that a cryptographic hash would be more
>robust, but it would also be considerably more expensive in the common
>case where the .go file is already in the cache.
>
>I don't think it's worth paying this cost every time

OK, you can rule that suggestion out, but I think you have erred in
jumping from that to wontfix on the general problem.  You have not
addressed my prior suggestion of identifying programs by exact match on
device, inode number, and mtime.  (File size could also be included.)
This freshness check is very cheap, because it's just a few fixed-size
fields from the stat structure, and you're already necessarily doing a
stat on the program file.  Using the identifying fields as the cache
key even saves you a stat on the cached file.  Although not quite as
effective as a hash comparison, it would be a huge practical improvement
over the current filename-and-inexact-mtime comparison.

-zefram

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.