Traffic Blazer Robots.txt Generator Screen Shot
Robots.txt Generator
Close

The Traffic Blazer Robots.txt Generator feature enables you to manage bandwidth consumption and control the way search engine spiders crawl and/or index your Web site by allowing you to generate a robots.txt file for your Web site or to create Robots Meta tags for specific pages on the site. A robots.txt file enables you to command all or certain spiders to steer clear of any part or all pages and files on a given Web site. Once uploaded to the root directory of your Web site, visiting spiders will open and read the robots.txt file and implement its directions before accessing the affiliated Web site. The robots.txt file must be placed in the server's root directory. If you cannot access the root directory, you can instead use the Robots Meta tag format to direct the behavior of visiting spiders.

<< HTML Analysis Automated Submission >>


<< HTML Analysis Automated Submission >>