How to use the Robots.txt File to block search engines
Robots.txt files control search engine crawler access to certain areas of your site.
Robots.txt files control search engine crawler access to certain areas of your site.