SEO & Marketing

robots.txt Generator - Crawler Rules

Build a basic robots.txt with User-agent, Allow, Disallow, and Sitemap rules. Useful for launch-time crawl policy setup.

Quick switch→Sitemap Generator

robots.txtSEOCrawlers

Quick switch:Sitemap Generator Meta Tag Generator

Favorite this tool

No favorite storage mode has been selected.

Quick actions

Back to SEO & Marketing Browse all tools

Copy this tool URL or open the system share sheet on a supported device.

Use cases

Write a starter robots.txt for a new site
Block crawlers from admin or draft paths
Declare a Sitemap URL in robots.txt

How to use

Enter the User-agent and Allow/Disallow paths
Add a Sitemap URL if needed
Copy the output text

Example input and output

Input

User-agent: * / Disallow: /admin / Sitemap: https://uuup.net/sitemap.xml

Output

robots.txt text

FAQ

Can I add multiple Disallow rules?

Yes. Enter one path per line to generate multiple disallow rules quickly.

Is robots.txt access control?

No. It is advisory for well-behaved crawlers and does not replace authentication or permissions.

Tools

Related tools

Continue with another translated tool for a nearby task.

Sitemap Generator

sitemap.xml Generator - URL List Sitemap

Available

Turn a multiline URL list into a basic sitemap.xml. Useful for small sites preparing a page inventory for search engines. Processing stays in your browser.

Sitemap SEO XML

Open tool

Meta Tag Generator

Meta Tag Generator - Basic SEO and Open Graph