HTTP Error codes...

[Lyte]

I don't understand what Awstat is telling me here in the "HTTP Error codes" column. It reads...

HTTP Error codes* -- Hits -- Percent -- Bandwidth

404 Document Not Found -- 109 -- 79.5% -- 0
403 Forbidden -- 23 -- 16.7% -- 0

When I click on the 404 link this is the information I'm given...

URL -- Hits

/robots.txt -- 53

What does this mean??

Thanks!

Lyte

 

[theparrot]

you dont have a robots.txt I guess.

 

[Kodeking]

robots.txt is the file that google and other search engines look at to see what pages they are allowed to index on your site. They like to honor your request to not add certain pages or directories to the search engine. If there are confidential pages that the public should not be able to search, you can simply add a line to the robots.txt file to keep search engines from adding those pages to their index.

http://www.searchengineworld.com/robots/robots_tutorial.htm
http://www.outfront.net/tutorials_02/adv_tech/robots.htm