Crawl Sessions & Logs

Current Status
Crawling Active
Session: auto_crawl
Started: 2025-12-10 20:58
Progress: 0 servers, 0 resources
Queue Status

254877

Pending

45721

Processing

208460

Completed

263273

Failed
Crawl Sessions
Session Name Status Duration Results Errors Started Actions
auto_crawl Completed 00:06:15 33 servers
32 resources 		37 2025-09-07 15:37
auto_crawl Completed 00:06:31 43 servers
42 resources 		57 2025-09-07 15:37
auto_crawl Completed 00:04:50 57 servers
55 resources 		45 2025-09-07 15:32
auto_crawl Completed 00:04:54 58 servers
51 resources 		39 2025-09-07 15:32
auto_crawl Completed 00:12:14 631 servers
622 resources 		87 2025-09-07 15:19
auto_crawl Completed 00:12:19 373 servers
369 resources 		63 2025-09-07 15:19
auto_crawl Completed 00:09:01 420 servers
408 resources 		118 2025-09-07 15:10
auto_crawl Completed 00:09:23 194 servers
188 resources 		79 2025-09-07 15:10
auto_crawl Completed 00:08:27 255 servers
236 resources 		150 2025-09-07 15:01
auto_crawl Completed 00:11:06 226 servers
212 resources 		101 2025-09-07 14:58
auto_crawl Completed 00:29:50 2,580 servers
2,414 resources 		358 2025-09-07 14:28
auto_crawl Completed 00:29:55 1,417 servers
1,355 resources 		175 2025-09-07 14:28
auto_crawl Completed 00:00:25 21 servers
15 resources 		8 2025-09-07 14:28
auto_crawl Completed 00:03:03 172 servers
167 resources 		12 2025-09-07 14:25
auto_crawl Completed 00:01:46 85 servers
84 resources 		6 2025-09-07 14:23
auto_crawl Completed 00:01:49 174 servers
154 resources 		24 2025-09-07 14:23
auto_crawl Completed 00:32:32 3,104 servers
3,061 resources 		102 2025-09-07 13:50
auto_crawl Completed 00:02:08 196 servers
193 resources 		18 2025-09-07 13:48
auto_crawl Completed 00:02:22 147 servers
144 resources 		21 2025-09-07 13:48
server_info_test Completed 00:01:09 18 servers
18 resources 		4 2025-09-07 01:04
Recent Logs
ERROR 2025-12-10 21:08
Error crawling URL: Blocked by robots.txt
http://en.wikipedia.org/wiki/PDP-10
ERROR 2025-12-10 21:08
Error crawling URL: DNS resolution failed: [Errno -2] Name or service not known
git://git.symbolics.com:9418/
ERROR 2025-12-10 21:08
Error crawling URL: Blocked by robots.txt
http://en.wikipedia.org/wiki/PDP-10
ERROR 2025-12-10 21:08
Error crawling URL: Blocked by robots.txt
http://en.wikipedia.org/wiki/Digital_Equipment_Corporation
ERROR 2025-12-10 21:08
Error crawling URL: DNS resolution failed: [Errno -2] Name or service not known
git://git.common-lisp.info:9418/
ERROR 2025-12-10 21:08
Error crawling URL: DNS resolution failed: [Errno -2] Name or service not known
git://git.symbolics.com:9418/
WARNING 2025-12-10 21:05
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://tftp.example.com:69
WARNING 2025-12-10 21:05
Network error: Connection error: URL fetch timeout after 30 seconds
shoutcast://radio.plaza.one:8000
ERROR 2025-12-10 21:05
Error crawling URL: Connection error: HTTPConnectionPool(host='forum.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f15c4344690>: Failed to resolve 'forum.i2p' ([Errno -2] Name or service not known)"))
http://forum.i2p
ERROR 2025-12-10 21:05
Error crawling URL: Connection error: HTTPConnectionPool(host='echelon.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f15803b1c10>: Failed to resolve 'echelon.i2p' ([Errno -3] Temporary failure in name resolution)"))
http://echelon.i2p
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='identiguy.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f1524549b90>: Failed to resolve 'identiguy.i2p' ([Errno -2] Name or service not known)"))
http://identiguy.i2p
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='stats.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f15806f6a90>: Failed to resolve 'stats.i2p' ([Errno -3] Temporary failure in name resolution)"))
http://stats.i2p
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://10.0.0.100:69
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPSConnectionPool(host='facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion', port=443): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f1521197650>: Failed to resolve 'facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion' ([Errno -2] Name or service not known)"))
https://facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://192.168.1.100:69
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
docker://registry-1.docker.io:5000
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error timed out>
ethereum://mainnet.infura.io:8545
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://firmware.local:69
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://pxe.local:69
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
docker://registry.hub.docker.com:5000
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://boot.local:69
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='3g2upl4pq6kufc4m.onion', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f1524ea3290>: Failed to resolve '3g2upl4pq6kufc4m.onion' ([Errno -2] Name or service not known)"))
http://3g2upl4pq6kufc4m.onion
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error [Errno 111] Connection refused>
eth://localhost:8545
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 400: Bad Request
ethereum://arb1.arbitrum.io:443
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 400: Bad Request
ethereum://bsc-dataseed.binance.org:443
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error [Errno -3] Temporary failure in name resolution>
web3://web3.example.com:8545
ERROR 2025-12-10 21:04
Error crawling URL: DHT node not responding to ping
bittorrent://bt2.archive.org:6969
ERROR 2025-12-10 21:04
Error crawling URL: DHT node not responding to ping
bittorrent://bt1.archive.org:6969
ERROR 2025-12-10 21:04
Error crawling URL: DHT node not responding to ping
dht://router.utorrent.com:6881
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error [Errno 111] Connection refused>
eth://localhost:8545
ERROR 2025-12-10 21:04
Error crawling URL: Connection timeout
multiaddr:/ip4/147.75.77.187/tcp/4001/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 400: Bad Request
ethereum://eth-mainnet.alchemyapi.io:443
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='localhost', port=5001): Max retries exceeded with url: /api/v0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1524e020d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
http://localhost:5001/api/v0
ERROR 2025-12-10 21:04
Error crawling URL: No info hash in magnet link
magnet:?xt=urn:btih:26de44c9ec3a69b28fc27d0cd87c7c5e61e3cc4f&dn=debian-11.5.0-amd64-netinst.iso
ERROR 2025-12-10 21:04
Error crawling URL: No info hash in magnet link
magnet:?xt=urn:btih:3b245504cf5f11bbdbe1201cea6a6bf45aee1bc0&dn=ubuntu-22.04.1-desktop-amd64.iso
ERROR 2025-12-10 21:04
Error crawling URL: No info hash in magnet link
magnet:?xt=urn:btih:88594AAACBDE40EF3E2510C47374EC0AA396C08E&dn=archlinux-2023.01.01-x86_64.iso
ERROR 2025-12-10 21:04
Error crawling URL: [Errno -5] No address associated with hostname
grpc://demo.grpc.io:50051
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='localhost', port=5001): Max retries exceeded with url: /api/v0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f15a05bc210>: Failed to establish a new connection: [Errno 111] Connection refused'))
http://localhost:5001/api/v0
ERROR 2025-12-10 21:04
Error crawling URL: Connection timeout
grpc://grpc.io:9090
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
snmp://10.0.0.1:161/public
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
snmp://192.168.1.1:161/public
ERROR 2025-12-10 21:04
Error crawling URL: Connection refused
p2p://12D3KooWEqnTdgqHnkkwarSrJjeMP2ZJiADWLYADaNvUb6SQNyPF
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://cloudflare-ipfs.com
ERROR 2025-12-10 21:04
Error crawling URL: No gRPC or HTTP/2 support detected
grpc://grpc-test.sandbox.googleapis.com:443
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://dweb.link
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://gateway.ipfs.io
ERROR 2025-12-10 21:04
Error crawling URL: [Errno -5] No address associated with hostname
syslog://logs.company.com:514
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://ipfs.io
ERROR 2025-12-10 21:04
Error crawling URL: No gRPC or HTTP/2 support detected
grpc://buf.build:443
ERROR 2025-12-10 21:04
Error crawling URL: [Errno -5] No address associated with hostname
syslog://syslog.example.com:514