Crawl Sessions & Logs

Current Status
Crawling Active
Session: auto_crawl
Started: 2025-12-10 20:58
Progress: 0 servers, 0 resources
Queue Status

254877

Pending

45721

Processing

208460

Completed

263273

Failed
Crawl Sessions
Session Name Status Duration Results Errors Started Actions
auto_crawl Running Running... 0 servers
0 resources 		0 2025-12-10 20:58
auto_crawl Completed 00:39:01 162 servers
162 resources 		264 2025-12-03 16:02
auto_crawl Completed 00:36:40 171 servers
171 resources 		231 2025-11-02 00:55
auto_crawl Completed 01:02:43 166 servers
166 resources 		276 2025-10-28 09:50
auto_crawl Completed 03:03:25 383 servers
382 resources 		951 2025-10-27 22:18
auto_crawl Completed 10:45:46 190 servers
190 resources 		1724 2025-10-22 14:05
auto_crawl Running Running... 0 servers
0 resources 		0 2025-10-22 02:40
auto_crawl Running Running... 0 servers
0 resources 		0 2025-10-21 21:38
auto_crawl Completed 02:45:35 174 servers
174 resources 		621 2025-10-21 15:25
auto_crawl Completed 05:22:09 295 servers
295 resources 		3081 2025-10-18 18:44
auto_crawl Running Running... 0 servers
0 resources 		0 2025-10-17 22:34
auto_crawl Running Running... 0 servers
0 resources 		0 2025-10-17 19:46
auto_crawl Running Running... 0 servers
0 resources 		0 2025-10-17 17:14
auto_crawl Completed 04:44:35 175 servers
175 resources 		1183 2025-10-17 10:12
auto_crawl Running Running... 0 servers
0 resources 		0 2025-10-17 05:14
auto_crawl Running Running... 0 servers
0 resources 		0 2025-10-16 15:42
auto_crawl Completed 00:05:13 61 servers
61 resources 		504 2025-10-16 15:33
auto_crawl Completed 06:27:44 172 servers
172 resources 		1409 2025-10-15 17:15
auto_crawl Completed 04:55:54 227 servers
227 resources 		1140 2025-10-15 12:05
auto_crawl Running Running... 0 servers
0 resources 		0 2025-10-15 00:11
Recent Logs
ERROR 2025-12-10 21:08
Error crawling URL: Blocked by robots.txt
http://en.wikipedia.org/wiki/PDP-10
ERROR 2025-12-10 21:08
Error crawling URL: DNS resolution failed: [Errno -2] Name or service not known
git://git.symbolics.com:9418/
ERROR 2025-12-10 21:08
Error crawling URL: Blocked by robots.txt
http://en.wikipedia.org/wiki/PDP-10
ERROR 2025-12-10 21:08
Error crawling URL: Blocked by robots.txt
http://en.wikipedia.org/wiki/Digital_Equipment_Corporation
ERROR 2025-12-10 21:08
Error crawling URL: DNS resolution failed: [Errno -2] Name or service not known
git://git.common-lisp.info:9418/
ERROR 2025-12-10 21:08
Error crawling URL: DNS resolution failed: [Errno -2] Name or service not known
git://git.symbolics.com:9418/
WARNING 2025-12-10 21:05
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://tftp.example.com:69
WARNING 2025-12-10 21:05
Network error: Connection error: URL fetch timeout after 30 seconds
shoutcast://radio.plaza.one:8000
ERROR 2025-12-10 21:05
Error crawling URL: Connection error: HTTPConnectionPool(host='forum.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f15c4344690>: Failed to resolve 'forum.i2p' ([Errno -2] Name or service not known)"))
http://forum.i2p
ERROR 2025-12-10 21:05
Error crawling URL: Connection error: HTTPConnectionPool(host='echelon.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f15803b1c10>: Failed to resolve 'echelon.i2p' ([Errno -3] Temporary failure in name resolution)"))
http://echelon.i2p
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='identiguy.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f1524549b90>: Failed to resolve 'identiguy.i2p' ([Errno -2] Name or service not known)"))
http://identiguy.i2p
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='stats.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f15806f6a90>: Failed to resolve 'stats.i2p' ([Errno -3] Temporary failure in name resolution)"))
http://stats.i2p
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://10.0.0.100:69
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPSConnectionPool(host='facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion', port=443): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f1521197650>: Failed to resolve 'facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion' ([Errno -2] Name or service not known)"))
https://facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://192.168.1.100:69
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
docker://registry-1.docker.io:5000
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error timed out>
ethereum://mainnet.infura.io:8545
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://firmware.local:69
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://pxe.local:69
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
docker://registry.hub.docker.com:5000
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://boot.local:69
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='3g2upl4pq6kufc4m.onion', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f1524ea3290>: Failed to resolve '3g2upl4pq6kufc4m.onion' ([Errno -2] Name or service not known)"))
http://3g2upl4pq6kufc4m.onion
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error [Errno 111] Connection refused>
eth://localhost:8545
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 400: Bad Request
ethereum://arb1.arbitrum.io:443
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 400: Bad Request
ethereum://bsc-dataseed.binance.org:443
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error [Errno -3] Temporary failure in name resolution>
web3://web3.example.com:8545
ERROR 2025-12-10 21:04
Error crawling URL: DHT node not responding to ping
bittorrent://bt2.archive.org:6969
ERROR 2025-12-10 21:04
Error crawling URL: DHT node not responding to ping
bittorrent://bt1.archive.org:6969
ERROR 2025-12-10 21:04
Error crawling URL: DHT node not responding to ping
dht://router.utorrent.com:6881
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error [Errno 111] Connection refused>
eth://localhost:8545
ERROR 2025-12-10 21:04
Error crawling URL: Connection timeout
multiaddr:/ip4/147.75.77.187/tcp/4001/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 400: Bad Request
ethereum://eth-mainnet.alchemyapi.io:443
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='localhost', port=5001): Max retries exceeded with url: /api/v0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1524e020d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
http://localhost:5001/api/v0
ERROR 2025-12-10 21:04
Error crawling URL: No info hash in magnet link
magnet:?xt=urn:btih:26de44c9ec3a69b28fc27d0cd87c7c5e61e3cc4f&dn=debian-11.5.0-amd64-netinst.iso
ERROR 2025-12-10 21:04
Error crawling URL: No info hash in magnet link
magnet:?xt=urn:btih:3b245504cf5f11bbdbe1201cea6a6bf45aee1bc0&dn=ubuntu-22.04.1-desktop-amd64.iso
ERROR 2025-12-10 21:04
Error crawling URL: No info hash in magnet link
magnet:?xt=urn:btih:88594AAACBDE40EF3E2510C47374EC0AA396C08E&dn=archlinux-2023.01.01-x86_64.iso
ERROR 2025-12-10 21:04
Error crawling URL: [Errno -5] No address associated with hostname
grpc://demo.grpc.io:50051
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='localhost', port=5001): Max retries exceeded with url: /api/v0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f15a05bc210>: Failed to establish a new connection: [Errno 111] Connection refused'))
http://localhost:5001/api/v0
ERROR 2025-12-10 21:04
Error crawling URL: Connection timeout
grpc://grpc.io:9090
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
snmp://10.0.0.1:161/public
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
snmp://192.168.1.1:161/public
ERROR 2025-12-10 21:04
Error crawling URL: Connection refused
p2p://12D3KooWEqnTdgqHnkkwarSrJjeMP2ZJiADWLYADaNvUb6SQNyPF
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://cloudflare-ipfs.com
ERROR 2025-12-10 21:04
Error crawling URL: No gRPC or HTTP/2 support detected
grpc://grpc-test.sandbox.googleapis.com:443
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://dweb.link
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://gateway.ipfs.io
ERROR 2025-12-10 21:04
Error crawling URL: [Errno -5] No address associated with hostname
syslog://logs.company.com:514
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://ipfs.io
ERROR 2025-12-10 21:04
Error crawling URL: No gRPC or HTTP/2 support detected
grpc://buf.build:443
ERROR 2025-12-10 21:04
Error crawling URL: [Errno -5] No address associated with hostname
syslog://syslog.example.com:514