New Robots.txt Syntax Checker: a validator for robots.txt files

Robots.txt Checker

Robots.txt files (often erroneously called robot.txt, in singular) are created by webmasters to mark (disallow) files and directories of a web site that search engine spiders (and other types of robots) should not access.

This robots.txt checker is a “validator” that analyzes the syntax of a robots.txt file to see if its format is valid as established by Robot Exclusion Standard (please read the documentation and the tutorial to learn the basics) or if it contains errors.

Simple usage: How to check your robots.txt file format? Just insert the full URL (Example: http://www.yourdomain.com/robots.txt) of the robots.txt file you want to analyze and hit Enter

Powerful: The checker finds syntax errors, “logic” errors, mistyped words and it gives you useful optimization tips

Accurate: The validation process takes in account both Robots Exclusion Standard rules and spider-specific (Google, Inktomi, etc.) extensions.

Source: New Robots.txt Syntax Checker: a validator for robots.txt files

• How to mount ISO files in Windows Vista
• 10 Useful free CSS tools and services
• Save your time with WinMerge
• How to use different header for categories and pages in WordPress
• 14 Effective Practical Security Tips for WordPress
• Syntax of All Windows, Bash, SQL Server Command Line Tools
• Ultimate Wordpress SEO Tools and Plugins Resources from WordCampUK
• How to download any song in mp3 format from Blip.fm

About Ruhani Rabin

Ruhani Rabin is the original owner and author of this blog. He also reviews web 2.0 startups at Tech2all.com. Largely interested in web 2.0 apps & Social Media. Currently the Vice President of MOL Access Portal (MOL is owner of Friendster.com). Also he is Web 2.0 & Social Media Researcher and a Total fun Geek!.. There you have it ;)

Twitter Profile: http://twitter.com/ruhanirabin