shodan_idb module sends requests too fast and gets temp-banned from internetdb API, hanging bbot for an hour

[modemlaugh]

Describe the bug
What happened vs what was expected?

The shodan_idb module sends too many requests too fast to shodan's internetdb API, causing a temporary one hour ban of the client WAN IP (public IP). Bbot hangs/waits for about an hour until the ban is lifted and requests start working again.

Using the commands in the "how to reproduce" section, I was able to send just over 600 requests in rapid succession before getting my WAN IP temporarily banned. I tried this three times, each on a stopped/started AWS instance with a fresh IP and the results were consistent.

Expected behavior: Bbot should rate limit requests to the shodan internetdb API to one request per second in order to avoid the temporary IP ban.

Regarding whether or not bbot should time out in the event of a 429 response and temporary IP ban, I think waiting it out for an hour is the better default behavior vs giving up and moving on. I would rather my scan be complete and take longer vs parts of it failing silently and giving sub-optimal results. Maybe if bbot throws an INFO warning like "Hey, shodan internetdb blocked us. Sounds like something is wrong with the rate limiting - consider enabling debugging. Re-run with shodan_idb.timeout=true module parameter to enable timeout or 'kill shodan_idb' to give up."

Notes and observations:

• By the time shodan's API throws a 429 response, it is too late. The client WAN IP is already temp-banned. Best to just avoid it by self-throttling requests.
• The rate limit must be globally applied, perhaps via some kind of queue. If there were 20 threads self-throttling their requests, they could still trip the rate limit because they could potentially send 20 requests within one second in total.
• Shodan's internetdb API seems to temp-ban around the low 600-someodd request mark. Might consider adding a shodan_idb.throttling=[true|false] (default: true) switch so users can disable it for shorter batches.
• As mentioned, the ban lasts for about an hour. This can be seen in the debug.log by grepping for 'shodan_idb'.
• Based on some googling, it sounds like the client accessing the API needs to self-throttle with 1-second delays between requests to avoid getting temp-banned. I could not find official shodan documentation on this, but it's probably out there somewhere.

How to Reproduce
In order to induce the IP BAN, run the following from a fresh WAN IP via proxy or whatnot, or wait an hour to be unbanned. If you get HUP'ed out of your ssh session, try increasing the sleep a bit. Running this many simultaneous curl commands can max out the host CPU cores.

# set up temp dir
mkdir -p idb && cd idb/ && rm request_* -f

# slam the shodan internetdb api
for i in {1..2000}; do sleep 0.01; curl --silent --write-out " : HTTP %{response_code}\n" https://internetdb.shodan.io/ >>"request_$(mktemp --dry-run XXXXXX)" &; done

# analyze results
cat request_* |sort |uniq --count

Note that the IP ban can be also be induced by running a bbot command against a large target set (20000 hosts were about what I was scanning), enabling the shodan_idb module, and then waiting a while.

At this point, validate that you are banned by curl'ing the idb API, which should result in a 429 error "Rate limit exceeded. Contact sales@shodan.io if you need a higher rate limit."
curl https://internetdb.shodan.io/

Now try any bbot scan that uses the shodan_idb module and watch it hang for an hour until the IP ban is lifted.

If you enabled debug logging, you can view the error in detail as follows:
grep -P "shodan_idb.+status code 429" debug.log

BBOT Command

For the following command, I had a set of CIDR blocks in my targets.txt file that equaled to about 20000 target IP addresses.
time bbot --targets targets.txt --output-dir . --name bbot-$(date +%Y%m%d)-safe-sub-active --flags safe,subdomain-enum,active --config modules.massdns.max_resolvers=5000 --config web.spider_depth=2 web.spider_distance=2 web.spider_links_per_page=25 web.debug=true --output-modules asset_inventory,csv,txt,stdout,json,subdomains,web_report --allow-deadly

OS, BBOT Installation Method + Version
OS: Kali Linux 2025.1 on AWS ec2 instance

• Note that kali is built on debian

Installation method:

sudo apt update && sudo apt install pipx
pipx ensurepath
pipx install bbot

BBOT Config

description: bbot-20250418-safe-sub-active
config:
  modules:
    shodan_dns:
      api_key: <redacted>
  web:
    spider_depth: 2
    spider_distance: 2
    spider_links_per_page: 25
    debug: true
flags:
- active
- safe
- subdomain-enum
output_modules:
- asset_inventory
- csv
- json
- stdout
- subdomains
- txt
- web_report
scan_name: bbot-20250418-safe-sub-active
output_dir: .

Logs/Screenshots
The following is a snippet of the shodan_idb events from the debug.log of a scan that had the described problem. You can see the requests working (404 no info), followed by the 429 exceeded rate limit response, following by bbot just sitting there waiting for an hour, followed by it starting to work again after the IP ban is lifted. Note that the IPs have been redacted and sanitized to 1.2.3.4.

2025-04-17 04:36:55,306 [TRACE] bbot.modules.shodan_idb logger.py:151 API response to https://internetdb.shodan.io/1.2.3.4 failed with status code 404: {"detail":"No information available"}
2025-04-17 04:36:55,370 [TRACE] bbot.modules.shodan_idb logger.py:151 API response to https://internetdb.shodan.io/1.2.3.4 failed with status code 404: {"detail":"No information available"}
2025-04-17 04:36:55,399 [TRACE] bbot.modules.shodan_idb logger.py:151 API response to https://internetdb.shodan.io/1.2.3.4 failed with status code 404: {"detail":"No information available"}
2025-04-17 04:36:55,460 [TRACE] bbot.modules.shodan_idb logger.py:151 API response to https://internetdb.shodan.io/1.2.3.4 failed with status code 404: {"detail":"No information available"}
2025-04-17 04:36:55,482 [TRACE] bbot.modules.shodan_idb logger.py:151 API response to https://internetdb.shodan.io/1.2.3.4 failed with status code 429: {"error": "Rate limit exceeded. Contact sales@shodan.io if you need a higher rate limit."}
2025-04-17 04:36:56,817 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0), httpx(2:3:0), fingerprintx(0:2:0), sslcert(0:2:0)
2025-04-17 04:37:11,819 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0), httpx(0:2:0)
2025-04-17 04:37:26,820 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0), httpx(0:2:0)
2025-04-17 04:37:41,822 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:37:56,824 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:38:11,825 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:38:26,827 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:38:41,828 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:38:56,830 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:39:11,831 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:39:26,833 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:39:41,835 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:39:56,838 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:40:11,839 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:40:26,840 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:40:41,842 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:40:56,843 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:41:11,844 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:41:26,846 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:41:41,847 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:41:56,849 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:42:11,850 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:42:26,852 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:42:41,854 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:42:56,855 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:43:11,856 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:43:26,857 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:43:41,859 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:43:56,861 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:44:11,862 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:44:26,865 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:44:41,866 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:44:56,867 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:45:11,868 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:45:26,869 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:45:41,870 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:45:56,872 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:46:11,873 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:46:26,874 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:46:41,875 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:46:56,877 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:47:11,879 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:47:26,880 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:47:41,881 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:47:56,882 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:48:11,884 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:48:26,885 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:48:41,888 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:48:56,889 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:49:11,890 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:49:26,892 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:49:41,893 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:49:56,894 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:50:11,896 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:50:26,897 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:50:41,899 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:50:56,900 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:51:11,901 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:51:26,904 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:51:41,906 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:51:56,907 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:52:11,908 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:52:26,909 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:52:41,910 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:52:56,912 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:53:11,914 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:53:26,915 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:53:41,918 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:53:56,920 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:54:11,922 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:54:26,924 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:54:41,924 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:54:56,926 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:55:11,928 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:55:26,929 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:55:41,930 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:55:56,931 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:56:11,933 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:56:26,934 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:56:41,935 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:56:56,936 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:57:11,938 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:57:26,939 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:57:41,940 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:57:56,943 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:58:11,945 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:58:26,946 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:58:41,949 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:58:56,950 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:59:11,952 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:59:26,953 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:59:41,954 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 04:59:56,955 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:00:11,957 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:00:26,958 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:00:41,959 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:00:56,961 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:01:11,963 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:01:26,965 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:01:41,967 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:01:56,968 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:02:11,970 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:02:26,972 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:02:41,972 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:02:56,974 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:03:11,975 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:03:26,976 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:03:41,979 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:03:56,980 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:04:11,982 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:04:26,983 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:04:41,985 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:04:56,987 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:05:11,988 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:05:26,989 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:05:41,992 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:05:56,993 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:06:11,994 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:06:26,996 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:06:41,997 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:06:56,999 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:07:12,000 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:07:27,001 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:07:42,003 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:07:57,003 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:08:12,006 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:08:27,009 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:08:42,010 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:08:57,012 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:09:12,013 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:09:27,014 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:09:42,016 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:09:57,018 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:10:12,019 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:10:27,021 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:10:42,023 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:10:57,024 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:11:12,026 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:11:27,027 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:11:42,028 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:11:57,029 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:12:12,031 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:12:27,032 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:12:42,033 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:12:57,034 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:13:12,036 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:13:27,037 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:13:42,039 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:13:57,041 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:14:12,042 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:14:27,043 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:14:42,045 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:14:57,046 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:15:12,048 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:15:27,050 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:15:42,051 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:15:57,052 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:16:12,053 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:16:27,054 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:16:42,056 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:16:57,057 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:17:12,058 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:17:27,060 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:17:42,061 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:17:57,062 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:18:12,064 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:18:27,065 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:18:42,067 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:18:57,070 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:19:12,071 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:19:27,072 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:19:42,073 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:19:57,074 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:20:12,076 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:20:27,078 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:20:42,079 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:20:57,080 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:21:12,081 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:21:27,082 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:21:42,083 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:21:57,085 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:22:12,086 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:22:27,087 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:22:42,088 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:22:57,090 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:23:12,092 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:23:27,094 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:23:42,096 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:23:57,097 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:24:12,098 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:24:27,100 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:24:42,101 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:24:57,102 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:25:12,103 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:25:27,104 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:25:42,105 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:25:57,107 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:26:12,108 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:26:27,109 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:26:42,111 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:26:57,113 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:27:12,114 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:27:27,116 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:27:42,117 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:27:57,120 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:28:12,121 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:28:27,123 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:28:42,125 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:28:57,126 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:29:12,127 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:29:27,129 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:29:42,131 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:29:57,132 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:30:12,133 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:30:27,134 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:30:42,135 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:30:57,136 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:31:12,138 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:31:27,139 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:31:42,141 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:31:57,142 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:32:12,143 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:32:27,144 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:32:42,145 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:32:57,146 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:33:12,148 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:33:27,150 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:33:42,152 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:33:57,155 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:34:12,156 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:34:27,158 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:34:42,159 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:34:57,160 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:35:12,161 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:35:27,164 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:35:42,164 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:35:57,166 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:36:12,168 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:36:27,170 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:36:42,171 [INFO] bbot.scanner scanner.py:1175 bbot-20250416-safe-sub-active: Modules running (incoming:processing:outgoing) shodan_idb(12,944:1:0)
2025-04-17 05:36:55,558 [TRACE] bbot.modules.shodan_idb logger.py:151 API response to https://internetdb.shodan.io/1.2.3.4 failed with status code 404: {"detail":"No information available"}
2025-04-17 05:36:55,619 [TRACE] bbot.modules.shodan_idb logger.py:151 API response to https://internetdb.shodan.io/1.2.3.4 failed with status code 404: {"detail":"No information available"}
2025-04-17 05:36:55,638 [TRACE] bbot.modules.shodan_idb logger.py:151 API response to https://internetdb.shodan.io/1.2.3.4 failed with status code 404: {"detail":"No information available"}
2025-04-17 05:36:55,655 [TRACE] bbot.modules.shodan_idb logger.py:151 API response to https://internetdb.shodan.io/1.2.3.4 failed with status code 404: {"detail":"No information available"}
2025-04-17 05:36:55,715 [TRACE] bbot.modules.shodan_idb logger.py:151 API response to https://internetdb.shodan.io/1.2.3.4 failed with status code 404: {"detail":"No information available"}
2025-04-17 05:36:55,737 [TRACE] bbot.modules.shodan_idb logger.py:151 API response to https://internetdb.shodan.io/1.2.3.4 failed with status code 404: {"detail":"No information available"}

[TheTechromancer]

I've raised the verbosity of the rate limit message to INFO, so you can see whenever a rate limit is hit. I've also added a global config option so you can crank up the number of API retries by setting -c web.api_retries=999999. By default it will retry 2 times.

Finally, I modified the shodan module to self-limit itself to a max of 1 request per second. If you're able, please test it out on the fix-shodan-ratelimit branch.

One thing is still a mystery, which is how shodan_idb managed to get stuck processing one event. That's very strange and I wasn't able to reproduce it even with a permanent 429 ban. It times out after 90 seconds (1 original request + 2 retries with 30 second backoff), then moves on to the next event. Not sure what's happening there.

[TheTechromancer]

I added a shodan-specific retries option which can override the global web.retries. You can use it like this:

bbot -t evilcorp.com -m shodan_idb -c modules.shodan_idb.retries=999999

I believe there was also a bug where 404s were being retried, which would have increased the number of requests sent to the API. This has been fixed as well.

[modemlaugh]

Thanks for implementing the changes over a weekend.

What is the recommended way to install from source? I can clone down the repo and switch branches on a throwaway instance, but what is your procedure for installing dependencies? I'm not a seasoned python programmer yet so bear with me. :)

I'll test it out on that target list that was causing the problems in the first place with debugging turned on and see if I can reproduce the hang.

Edit: I did check the docs but they don't cover installing from source.

[modemlaugh]

So it looks like there may be a new bug to do with the retry setting. If I specify modules.shodan_idb.retries=n, it retries 404 errored requests that many times. Details as follows.

Per your guidance, I uninstalled the stable version of bbot and then reinstalled bbot from this fix branch as follows :

# uninstall bbot, remove cruft
pipx uninstall bbot
rm ~/.bbot/ ~/.ansible/ ~/.cache/ ~/.config/bbot/ -rf

# install fix branch
pipx install git+https://github.com/blacklanternsecurity/bbot@fix-shodan-ratelimit

Running the scan with the retries setting set to 999999:

bbot --targets targets.txt --output-dir . --name bbot-20250422-safe-sub-active --flags safe,subdomain-enum,active --config modules.massdns.max_resolvers=5000 --config modules.shodan_idb.retries=999999 web.spider_depth=2 web.spider_distance=2 web.spider_links_per_page=25 web.debug=true --output-modules asset_inventory,csv,txt,stdout,json,subdomains,web_report --allow-deadly

I let it run overnight and found a bunch of rotated debug logs the next morning. After gunzip'ing the logs and grepping through them, we can see that it is retrying the 404 error responses. It seems logical that it should be skipping those as 404 just means that shodan had no info on that IP, and only retrying responses other than 404 or 200.

# notes: the ip addresses in the output have been sanitized, line comments were added
grep shodan_idb debug.log* |grep "status code 404" |grep -oP "(?<=shodan\.io/)[^\s]+(?=\s)" |sort |uniq -c  
 999997 1.2.3.4    # this one had a couple 502 bad gateway errors in addition to the 999997 404 errors
  95903 3.4.5.6    # this one was still retrying when I killed the bbot scan
 999999 2.3.4.5    # this one retried 999999 times

[TheTechromancer]

Thanks @modemlaugh; I found the bug. it should be fixed now. Can you try again?

[modemlaugh]

Yes - I just kicked off the same scan. I'll check on it later this evening.

[modemlaugh]

Looking good so far. The scan completed without errors. I'm trying it again without the --config modules.shodan_idb.retries=999999 switch to see if it produces the same results. I'll let you know how it turns out.

[modemlaugh]

@TheTechromancer I think we are good to close this one out and merge. The scan I ran last night completed successfully without shodan_idb errors in the debug log.

[TheTechromancer]

Thanks for your testing @modemlaugh it's much appreciated 👍

Fixed in #2413