J Jammer VIP Member VIP ★★★★★★★★★★ Impact 20 Oct 2, 2003 603 views 3 replies #1 I saw this in another post and was wondering what it means. I've also seen it in a few of my error logs,but never knew what it was. Thanks! Click to expand...
I saw this in another post and was wondering what it means. I've also seen it in a few of my error logs,but never knew what it was. Thanks!
RJ Domain BuyerTop Member PRO VIP ★★★★★★★★★★ Impact 3,032 Oct 2, 2003 #2 Search engines like Google look for this file on your website when indexing your site. You can create a robots.txt file to give the search engine instructions for indexing your site, for example excluding certain pages or directories. Here's a whole site about it http://www.robotstxt.org/ Click to expand...
Search engines like Google look for this file on your website when indexing your site. You can create a robots.txt file to give the search engine instructions for indexing your site, for example excluding certain pages or directories. Here's a whole site about it http://www.robotstxt.org/
TheWatcher Founder, MAJ.comTop Member VIP ★★★★★★★★★★ Impact 992 Oct 3, 2003 #4 Nice link RJ. Thanks, TW Click to expand...