is it necessary?

[seojig]

hello,
is it necessary to upload robot.txt file or not as far as SEO concern?
i have some site and i didn't uploaded it.i need your suggestion that should i upload it or not.

 

[Keral_Patel]

If it is there then it is okay and if it is not there then also it is okay. :D

But if it is there and it is blank then it is just like as if there is no robots.txt for the robots.

If there is something written in it then it depends on what it is written in it.

 

[traderonline]

robots.txt can be used to give certain commands to SEs.

check this link for more details : http://en.wikipedia.org/wiki/Robots.txt

 

[John84]

i dont think it effects SEO if you dont add one. however, i have gotten into the habit of making one just in case. kinda like title tags on my hyperlinks and meta descriptions and meta keywords. not important now but it could be in the next algorithm change.

 

[solidghost]

Certain parts of your site which you don't want Search Engines to crawl, you should use Robots.txt to block off access.

And if you are worried about duplicate content (due to feeds or searches or something), you should also use it.

 

[squatter]

Certain parts of your site which you don't want Search Engines to crawl, you should use Robots.txt to block off access.

And if you are worried about duplicate content (due to feeds or searches or something), you should also use it.

He had no content, as he didn't upload the site yet.

 

[shockie]

i dont think it effects SEO if you dont add one. however, i have gotten into the habit of making one just in case. kinda like title tags on my hyperlinks and meta descriptions and meta keywords. not important now but it could be in the next algorithm change.

i also make a robots.txt just in case, but i would disagree with saying that title and meta tags arent important :P

 

[I-Got-IT!]

It dosent matter if you do not add one, but make it a habit to if u want =)

 

[Dotmainer]

not SEO relevant: i would upload it, because to stop 404 Errors in your logfiles, when SE don't find it.

 

[enlytend]

Yahoo's bot used to leave without indexing anything if it couldn't find robots.txt. I'm not sure if that's still the case but it's easy enough to have a 3 line default file just to be on the safe side.