Crawl Sessions & Logs

Current Status
Crawling Active
Session: auto_crawl
Started: 2025-12-10 20:58
Progress: 0 servers, 0 resources
Queue Status

254877

Pending

45721

Processing

208460

Completed

263273

Failed
Crawl Sessions
Session Name Status Duration Results Errors Started Actions
auto_crawl Completed 02:02:25 0 servers
0 resources 		10 2025-09-09 15:35
auto_crawl Completed 12:56:52 7,146 servers
7,144 resources 		127 2025-09-08 19:51
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 19:49
auto_crawl Completed 00:18:40 0 servers
0 resources 		522 2025-09-08 19:31
auto_crawl Completed 00:00:36 0 servers
0 resources 		166 2025-09-08 19:30
auto_crawl Completed 00:09:25 11 servers
11 resources 		39 2025-09-08 19:20
auto_crawl Completed 00:06:08 4 servers
4 resources 		37 2025-09-08 19:10
auto_crawl Completed 00:01:28 0 servers
0 resources 		31 2025-09-08 18:58
auto_crawl Completed 00:04:28 6 servers
6 resources 		19 2025-09-08 18:50
auto_crawl Completed 00:13:25 10 servers
10 resources 		55 2025-09-08 18:41
auto_crawl Completed 00:06:51 0 servers
0 resources 		33 2025-09-08 18:33
auto_crawl Completed 00:06:58 0 servers
0 resources 		33 2025-09-08 18:33
auto_crawl Completed 00:11:47 20 servers
20 resources 		39 2025-09-08 18:20
auto_crawl Completed 00:17:05 15 servers
15 resources 		71 2025-09-08 18:15
auto_crawl Completed 00:06:53 2 servers
2 resources 		45 2025-09-08 18:08
auto_crawl Completed 00:07:05 1 servers
1 resources 		22 2025-09-08 18:08
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 18:06
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 18:06
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 18:04
auto_crawl Running Running... 0 servers
0 resources 		0 2025-09-08 18:04
Recent Logs
ERROR 2025-12-10 21:08
Error crawling URL: Blocked by robots.txt
http://en.wikipedia.org/wiki/PDP-10
ERROR 2025-12-10 21:08
Error crawling URL: DNS resolution failed: [Errno -2] Name or service not known
git://git.symbolics.com:9418/
ERROR 2025-12-10 21:08
Error crawling URL: Blocked by robots.txt
http://en.wikipedia.org/wiki/PDP-10
ERROR 2025-12-10 21:08
Error crawling URL: Blocked by robots.txt
http://en.wikipedia.org/wiki/Digital_Equipment_Corporation
ERROR 2025-12-10 21:08
Error crawling URL: DNS resolution failed: [Errno -2] Name or service not known
git://git.common-lisp.info:9418/
ERROR 2025-12-10 21:08
Error crawling URL: DNS resolution failed: [Errno -2] Name or service not known
git://git.symbolics.com:9418/
WARNING 2025-12-10 21:05
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://tftp.example.com:69
WARNING 2025-12-10 21:05
Network error: Connection error: URL fetch timeout after 30 seconds
shoutcast://radio.plaza.one:8000
ERROR 2025-12-10 21:05
Error crawling URL: Connection error: HTTPConnectionPool(host='forum.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f15c4344690>: Failed to resolve 'forum.i2p' ([Errno -2] Name or service not known)"))
http://forum.i2p
ERROR 2025-12-10 21:05
Error crawling URL: Connection error: HTTPConnectionPool(host='echelon.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f15803b1c10>: Failed to resolve 'echelon.i2p' ([Errno -3] Temporary failure in name resolution)"))
http://echelon.i2p
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='identiguy.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f1524549b90>: Failed to resolve 'identiguy.i2p' ([Errno -2] Name or service not known)"))
http://identiguy.i2p
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='stats.i2p', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f15806f6a90>: Failed to resolve 'stats.i2p' ([Errno -3] Temporary failure in name resolution)"))
http://stats.i2p
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://10.0.0.100:69
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPSConnectionPool(host='facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion', port=443): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f1521197650>: Failed to resolve 'facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion' ([Errno -2] Name or service not known)"))
https://facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://192.168.1.100:69
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
docker://registry-1.docker.io:5000
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error timed out>
ethereum://mainnet.infura.io:8545
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://firmware.local:69
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://pxe.local:69
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
docker://registry.hub.docker.com:5000
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
tftp://boot.local:69
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='3g2upl4pq6kufc4m.onion', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7f1524ea3290>: Failed to resolve '3g2upl4pq6kufc4m.onion' ([Errno -2] Name or service not known)"))
http://3g2upl4pq6kufc4m.onion
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error [Errno 111] Connection refused>
eth://localhost:8545
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 400: Bad Request
ethereum://arb1.arbitrum.io:443
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 400: Bad Request
ethereum://bsc-dataseed.binance.org:443
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error [Errno -3] Temporary failure in name resolution>
web3://web3.example.com:8545
ERROR 2025-12-10 21:04
Error crawling URL: DHT node not responding to ping
bittorrent://bt2.archive.org:6969
ERROR 2025-12-10 21:04
Error crawling URL: DHT node not responding to ping
bittorrent://bt1.archive.org:6969
ERROR 2025-12-10 21:04
Error crawling URL: DHT node not responding to ping
dht://router.utorrent.com:6881
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: <urlopen error [Errno 111] Connection refused>
eth://localhost:8545
ERROR 2025-12-10 21:04
Error crawling URL: Connection timeout
multiaddr:/ip4/147.75.77.187/tcp/4001/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 400: Bad Request
ethereum://eth-mainnet.alchemyapi.io:443
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='localhost', port=5001): Max retries exceeded with url: /api/v0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1524e020d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
http://localhost:5001/api/v0
ERROR 2025-12-10 21:04
Error crawling URL: No info hash in magnet link
magnet:?xt=urn:btih:26de44c9ec3a69b28fc27d0cd87c7c5e61e3cc4f&dn=debian-11.5.0-amd64-netinst.iso
ERROR 2025-12-10 21:04
Error crawling URL: No info hash in magnet link
magnet:?xt=urn:btih:3b245504cf5f11bbdbe1201cea6a6bf45aee1bc0&dn=ubuntu-22.04.1-desktop-amd64.iso
ERROR 2025-12-10 21:04
Error crawling URL: No info hash in magnet link
magnet:?xt=urn:btih:88594AAACBDE40EF3E2510C47374EC0AA396C08E&dn=archlinux-2023.01.01-x86_64.iso
ERROR 2025-12-10 21:04
Error crawling URL: [Errno -5] No address associated with hostname
grpc://demo.grpc.io:50051
ERROR 2025-12-10 21:04
Error crawling URL: Connection error: HTTPConnectionPool(host='localhost', port=5001): Max retries exceeded with url: /api/v0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f15a05bc210>: Failed to establish a new connection: [Errno 111] Connection refused'))
http://localhost:5001/api/v0
ERROR 2025-12-10 21:04
Error crawling URL: Connection timeout
grpc://grpc.io:9090
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
snmp://10.0.0.1:161/public
WARNING 2025-12-10 21:04
Network error: Connection error: URL fetch timeout after 30 seconds
snmp://192.168.1.1:161/public
ERROR 2025-12-10 21:04
Error crawling URL: Connection refused
p2p://12D3KooWEqnTdgqHnkkwarSrJjeMP2ZJiADWLYADaNvUb6SQNyPF
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://cloudflare-ipfs.com
ERROR 2025-12-10 21:04
Error crawling URL: No gRPC or HTTP/2 support detected
grpc://grpc-test.sandbox.googleapis.com:443
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://dweb.link
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://gateway.ipfs.io
ERROR 2025-12-10 21:04
Error crawling URL: [Errno -5] No address associated with hostname
syslog://logs.company.com:514
ERROR 2025-12-10 21:04
Error crawling URL: HTTP 404: Not Found
ipfs://ipfs.io
ERROR 2025-12-10 21:04
Error crawling URL: No gRPC or HTTP/2 support detected
grpc://buf.build:443
ERROR 2025-12-10 21:04
Error crawling URL: [Errno -5] No address associated with hostname
syslog://syslog.example.com:514