WHITE LIFELoading...
Generate robots.txt files for your website SEO. Free browser utility for founders, marketers, and operators who need fast, no-login workflows.
Generate a robots.txt file with allow/disallow rules, sitemap declaration, and crawl delay settings. Control how search engines index your website.
A text file at your site root (example.com/robots.txt) that tells search engine crawlers which pages they can and cannot access. It is a suggestion, not a security measure.
Specifies which bot the rules apply to. Use * for all bots, or target specific ones like Googlebot, Bingbot, GPTBot. Each bot section can have different rules.
Disallow prevents crawling of specified paths. Allow explicitly permits crawling (useful to override a broader Disallow). An empty Disallow allows everything.
Declare your XML sitemap URL so search engines can discover all your pages. Always use the full absolute URL (https://example.com/sitemap.xml).
Asks bots to wait N seconds between requests. Useful for low-resource servers. Google ignores this directive — use Google Search Console instead.
Share this tool
/// Next best path
No signup required for the tool. Stripe-secured checkout only appears when you choose a paid path.
Get the 7-step AI business guide and use this tool inside a full launch path.
OPEN PATH →2X DAILYFollow the New York morning and evening marketing brief for new angles and moves.
OPEN PATH →FULL SYSTEMSee ebooks, bundles, and the Daily News path without guessing what fits next.
OPEN PATH →Trusted WHITE LIFE route: free guide → daily news → pricing → account fulfillment.