GNU bug report logs - #52338
Crawler bots are downloading substitutes

Previous Next

Package: guix;

Reported by: Leo Famulari <leo <at> famulari.name>

Date: Mon, 6 Dec 2021 21:22:01 UTC

Severity: normal

Done: Mathieu Othacehe <othacehe <at> gnu.org>

Bug is archived. No further changes may be made.

Full log

View this message in rfc822 format

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 52338 <at> debbugs.gnu.org
Subject: bug#52338: Crawler bots are downloading substitutes
Date: Thu, 09 Dec 2021 14:27:38 +0100

Hello Leo,

> +           (nginx-location-configuration
> +             (uri "/robots.txt")
> +             (body
> +               (list
> +                 "add_header  Content-Type  text/plain;"
> +                 "return 200 \"User-agent: *\nDisallow: /nar/\n\";"))))))

Nice, the bots are also accessing the Cuirass web interface, do you
think it would be possible to extend this snippet to prevent it?

Thanks,

Mathieu
This bug report was last modified 3 years and 209 days ago.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.