Robots.txt Generator
Build crawler rules with explicit user agents, allow/disallow paths and sitemap URLs.
0 characters
Ready when you are
Add your input, adjust the options and run the tool.
—
No input saved
How to use this tool
01
Add your input
Paste text or choose a file. Try the example to get started.
02
Choose your options
Build crawler rules with explicit user agents, allow/disallow paths and sitemap URLs.
03
Inspect and export
Review the processing notes, then copy or download the result. Continue with another tool if needed.