Allow Software Heritage IP addresses

Software Heritage is similar to the Internet Archive (which is allowed)
but is focused on archiving source code from forges
instead of pages from websites.

Software Heritage processes do not use generic web crawling,
but instead forge-specific and VCS-specific tools that
are designed to use as little resources as possible,
especially using incremental pulls, using/adding APIs
and ignoring repositories that have not changed.

See-also: https://github.com/TecharoHQ/anubis/pulls/276
See-also: https://www.softwareheritage.org/
See-also: https://www.softwareheritage.org/software-heritage-faq/
See-also: https://docs.softwareheritage.org/user/faq/
See-also: https://gitlab.softwareheritage.org/swh
See-also: https://gitlab.softwareheritage.org/swh/infra/add-forge-now-requests/-/merge_requests/4

Author: pabs3

data/crawlers/_allow-good.yaml

@@ -8,5 +8,6 @@
 - import: (data)/crawlers/marginalia.yaml
 - import: (data)/crawlers/mojeekbot.yaml
 - import: (data)/crawlers/commoncrawl.yaml
+- import: (data)/crawlers/software-heritage.yaml
 - import: (data)/crawlers/wikimedia-citoid.yaml
 - import: (data)/crawlers/yandexbot.yaml

data/crawlers/software-heritage.yaml

@@ -0,0 +1,4 @@
+- name: software-heritage
+  action: ALLOW
+  # https://docs.softwareheritage.org/user/faq/#which-ip-address-range-should-we-mark-as-safe-in-our-anti-bot-protection-systems
+  remote_addresses: [ "128.93.166.0/26" ]

docs/docs/CHANGELOG.md

@@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+- Allow requests from Software Heritage
 - Expose [pprof endpoints](https://pkg.go.dev/net/http/pprof) on the metrics listener to enable profiling Anubis in production.
 - fix: prevent nil pointer panic in challenge validation when threshold rules match during PassChallenge (#1463)
 - Instruct reverse proxies to not cache error pages.