GNU bug report logs - #54241
[PATCH 0/4] 'github' importer gracefully handles rate limiting

Previous Next

Full log

View this message in rfc822 format

From: Maxime Devos <maximedevos <at> telenet.be>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 54241 <at> debbugs.gnu.org, Nicolas Goaziou <mail <at> nicolasgoaziou.fr>
Subject: [bug#54241] [PATCH 0/4] 'github' importer gracefully handles rate limiting
Date: Sat, 05 Mar 2022 10:44:48 +0100

[Message part 1 (text/plain, inline)]

Ludovic Courtès schreef op vr 04-03-2022 om 21:45 [+0100]:
> My concern is that we’d end up caching one or two little files in
> ~/.cache for each candidate package, and (rate limit aside) the overhead
> of dealing with the cache might outweigh the benefits.  I’d rather use
> ‘http-fetch/cached’ for bigger files, like in (guix cve).
> 
> WDYT?

If the overhead of caching little files is a concern, then perhaps a
SQLite (or GDBM) database could be used instead of the filesystem-based
cache?  The number of packages in Guix was about 150 000 IIRC, if we
assume something around the magnitude of 200 bytes per package, then
we end up with about 29 MiB for the entirity of Guix.  And there might
be some opportunities for compression, reducing this number.

Something like this could be left for later though.

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.