Crawl Sessions & Logs

Current Status
Crawling Active
Session: auto_crawl
Started: 2025-12-10 20:58
Progress: 0 servers, 0 resources
Queue Status

254877

Pending

45721

Processing

208460

Completed

263273

Failed
Crawl Sessions
Session Name Status Duration Results Errors Started Actions
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 18:02
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 18:02
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 18:00
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 18:00
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 17:59
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 17:58
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 17:57
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 17:57
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 17:55
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 17:55
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 17:53
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 17:53
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 17:51
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 17:51
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 17:49
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 17:49
auto_crawl Completed 04:42:18 745 servers
740 resources 		141 2025-09-08 13:07
auto_crawl Completed 00:19:56 200 servers
200 resources 		61 2025-09-08 12:45
auto_crawl Completed 00:00:48 9 servers
9 resources 		5 2025-09-08 12:44
auto_crawl Completed 00:12:12 103 servers
103 resources 		31 2025-09-08 12:30
Recent Logs
ERROR 2025-12-10 21:08
Error crawling URL: Blocked by robots.txt
http://en.wikipedia.org/wiki/PDP-10
ERROR 2025-12-10 21:08
Error crawling URL: DNS resolution failed: [Errno -2] Name or service not known
git://git.symbolics.com:9418/
ERROR 2025-12-10 21:08
Error crawling URL: Blocked by robots.txt
http://en.wikipedia.org/wiki/PDP-10
ERROR 2025-12-10 21:08
Error crawling URL: Blocked by robots.txt
http://en.wikipedia.org/wiki/Digital_Equipment_Corporation
ERROR 2025-12-10 21:08
Error crawling URL: DNS resolution failed: [Errno -2] Name or service not known
git://git.common-lisp.info:9418/
ERROR 2025-12-10 21:08
Error crawling URL: DNS resolution failed: [Errno -2] Name or service not known
git://git.symbolics.com:9418/
WARNING 2025-12-10 21:05
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://tftp.example.com:69
WARNING 2025-12-10 21:05
Network error: Connection error: URL fetch timeout after 30 seconds
shoutcast://radio.plaza.one:8000
ERROR 2025-12-10 21:05
Error crawling URL: Connection error: HTTPConnectionPool(host='forum.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f15c4344690>: Failed to resolve 'forum.i2p' ([Errno -2] Name or service not known)"))
http://forum.i2p
ERROR 2025-12-10 21:05
Error crawling URL: Connection error: HTTPConnectionPool(host='echelon.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f15803b1c10>: Failed to resolve 'echelon.i2p' ([Errno -3] Temporary failure in name resolution)"))
http://echelon.i2p
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='identiguy.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f1524549b90>: Failed to resolve 'identiguy.i2p' ([Errno -2] Name or service not known)"))
http://identiguy.i2p
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='stats.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f15806f6a90>: Failed to resolve 'stats.i2p' ([Errno -3] Temporary failure in name resolution)"))
http://stats.i2p
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://10.0.0.100:69
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPSConnectionPool(host='facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion', port=443): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f1521197650>: Failed to resolve 'facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion' ([Errno -2] Name or service not known)"))
https://facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://192.168.1.100:69
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
docker://registry-1.docker.io:5000
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error timed out>
ethereum://mainnet.infura.io:8545
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://firmware.local:69
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://pxe.local:69
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
docker://registry.hub.docker.com:5000
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://boot.local:69
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='3g2upl4pq6kufc4m.onion', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f1524ea3290>: Failed to resolve '3g2upl4pq6kufc4m.onion' ([Errno -2] Name or service not known)"))
http://3g2upl4pq6kufc4m.onion
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error [Errno 111] Connection refused>
eth://localhost:8545
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 400: Bad Request
ethereum://arb1.arbitrum.io:443
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 400: Bad Request
ethereum://bsc-dataseed.binance.org:443
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error [Errno -3] Temporary failure in name resolution>
web3://web3.example.com:8545
ERROR 2025-12-10 21:04
Error crawling URL: DHT node not responding to ping
bittorrent://bt2.archive.org:6969
ERROR 2025-12-10 21:04
Error crawling URL: DHT node not responding to ping
bittorrent://bt1.archive.org:6969
ERROR 2025-12-10 21:04
Error crawling URL: DHT node not responding to ping
dht://router.utorrent.com:6881
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error [Errno 111] Connection refused>
eth://localhost:8545
ERROR 2025-12-10 21:04
Error crawling URL: Connection timeout
multiaddr:/ip4/147.75.77.187/tcp/4001/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 400: Bad Request
ethereum://eth-mainnet.alchemyapi.io:443
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='localhost', port=5001): Max retries exceeded with url: /api/v0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1524e020d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
http://localhost:5001/api/v0
ERROR 2025-12-10 21:04
Error crawling URL: No info hash in magnet link
magnet:?xt=urn:btih:26de44c9ec3a69b28fc27d0cd87c7c5e61e3cc4f&dn=debian-11.5.0-amd64-netinst.iso
ERROR 2025-12-10 21:04
Error crawling URL: No info hash in magnet link
magnet:?xt=urn:btih:3b245504cf5f11bbdbe1201cea6a6bf45aee1bc0&dn=ubuntu-22.04.1-desktop-amd64.iso
ERROR 2025-12-10 21:04
Error crawling URL: No info hash in magnet link
magnet:?xt=urn:btih:88594AAACBDE40EF3E2510C47374EC0AA396C08E&dn=archlinux-2023.01.01-x86_64.iso
ERROR 2025-12-10 21:04
Error crawling URL: [Errno -5] No address associated with hostname
grpc://demo.grpc.io:50051
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='localhost', port=5001): Max retries exceeded with url: /api/v0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f15a05bc210>: Failed to establish a new connection: [Errno 111] Connection refused'))
http://localhost:5001/api/v0
ERROR 2025-12-10 21:04
Error crawling URL: Connection timeout
grpc://grpc.io:9090
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
snmp://10.0.0.1:161/public
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
snmp://192.168.1.1:161/public
ERROR 2025-12-10 21:04
Error crawling URL: Connection refused
p2p://12D3KooWEqnTdgqHnkkwarSrJjeMP2ZJiADWLYADaNvUb6SQNyPF
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://cloudflare-ipfs.com
ERROR 2025-12-10 21:04
Error crawling URL: No gRPC or HTTP/2 support detected
grpc://grpc-test.sandbox.googleapis.com:443
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://dweb.link
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://gateway.ipfs.io
ERROR 2025-12-10 21:04
Error crawling URL: [Errno -5] No address associated with hostname
syslog://logs.company.com:514
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://ipfs.io
ERROR 2025-12-10 21:04
Error crawling URL: No gRPC or HTTP/2 support detected
grpc://buf.build:443
ERROR 2025-12-10 21:04
Error crawling URL: [Errno -5] No address associated with hostname
syslog://syslog.example.com:514