Your Cart
$ 0.00
View/Edit Cart
Checkout Now

Robots.txt Generator

Robots Txt Generator is an convenient-to-use tool to create correct Robots.Txt directives on your site with no trouble copy and tweak Robots.Txt documents from other web sites.

Now, as a search engine spiders crawl a internet site, they start crawling by means of analyzing a robots.Txt file residing on the root domain.

Select a User Agent
Add directories/files you DO NOT want indexed
Directory/Page.html >
Three Examples:  1: Directory/myPage.html
2: Directory/
3: myPage.html