Subject: Re: bug#52338: Crawler bots are downloading substitutes

From: Mathieu Othacehe <[email-masked]>
Date: Thu, 09 Dec 2021 14:27:38 +0100

Hello Leo,

> +           (nginx-location-configuration
> +             (uri "/robots.txt")
> +             (body
> +               (list
> +                 "add_header  Content-Type  text/plain;"
> +                 "return 200 \"User-agent: *\nDisallow: /nar/\n\";"))))))

Nice, the bots are also accessing the Cuirass web interface, do you
think it would be possible to extend this snippet to prevent it?

Thanks,

Mathieu