Hakrawler
Fast web crawler for endpoint discovery
Recon & OSINT8 commands
#crawler#endpoint#recon#golang
Installation
1Install via Go
2Verify installation
# Using Go
go install github.com/hakluke/hakrawler@latest
# Verify
hakrawler -hBasic Usage
Crawl a target website and discover endpoints
# Basic crawl via stdin
echo "https://example.com" | hakrawler
# Deep crawl with subdomains
echo "https://example.com" | hakrawler -subs -depth 3
# Output to file
echo "https://example.com" | hakrawler -out results.txt
# With custom user agent
echo "https://example.com" | hakrawler -u "Mozilla/5.0"Command Reference
8 commands
1Include subdomains
-subs2Crawl depth
-depth3Output file
-out4Custom user agent
-u5Number of threads
-t6Request timeout
-timeout7Skip TLS verification
-insecure8Show help
-hWhen to Use
1
Initial reconnaissance for endpoint discovery2
Finding JavaScript files and API endpoints3
Mapping site structure before manual testing4
Automated content discovery in bug bounty5
Collecting URLs for further analysisNotes & Tips
1
Designed for use in pipelines with other tools2
Lightweight and fast compared to traditional crawlers3
Output can be piped directly to tools like gf or qsreplace4
Does not render JavaScript — only parses responsesCommon Errors & Solutions
No URLs found
Increase depth with -depth flag or check target accessibility
TLS errors
Use -insecure flag to skip certificate verification
Too many redirects
The tool follows redirects by default; check target configuration