How do I stop search engines from indexing my page?

[TiNiB]

hi

how do i stop search engines from indexing my page.

i have heard that there is a bit of code to stop this??

thx

 

[deadserious]

You can add something like this to a text file:

User-agent: *
Disallow: /

Save it as robots.txt and upload it to your main directory. That would tell the robots to not index any directory below /.

You can also disallow more than one or certain directories by doing something like:

User-agent: *
Disallow: /downloads/
Disallow: /cgi-bin/

Don't put any secret directories in that file because someone can just type in http://yourdomain.ext/robotx.txt and see what's in it.

User-agent: * is the the name of the bot. Using * is a wild card for all bots.

You could also do something like:

User-agent: GoogleBot

To disallow certain spiders/bots. You can usually find the names of the bots on the search engines website.

More info here: http://www.robotstxt.org/wc/exclusion.html#robotstxt

 

[TiNiB]

thanks!

 

[deadserious]

YW!

 

[DarkDevil]

What sort of website are you running that you wouldn't want people to find it online? just wondering.

 

[TiNiB]

my online cv.

it has personal info on it and its only for employer.

 

[DarkDevil]

Ah. Ok, it satisfies my curiousity. I'm hoping your taking some other steps in preventing people from finding it.