🤖 robots.txt Detailed Analyzer
Just enter a hostname to fetch /robots.txt and analyze Allow / Disallow rules per User-agent group, Sitemap, Crawl-delay, and the permission status for major search engine and AI crawlers.
100% Free
No signup
Server-side
No logs / DB
Rate-limited
5 languages
Dark mode
🔗 Related Tools
- ・LLMO Check — AI crawler permissions and llms.txt
- ・Basic SEO Check
- ・Structured Data
- ・URL Total Check