Robots

[peaches017]

I run a website called CollegeACB.com. Pretty much every page on the site runs on CollegeACB.com/sb.php

I do want to be indexed for the homepage, but not for anything else on sb.php.

My robots .txt can be found at http://www.collegeacb.com/robots.txt

I believe it is still indexing pages, despite my efforts. Is my robots.txt configured incorrectly, or is there code I can put on every page to make sure it isn't indexed?

Peter

 

[Keral_Patel]

If possible then put the code like

<META name="ROBOTS" content="NOINDEX,NOFOLLOW">

In the header part of sb.php

 

[angilina]

How about trying this code?

User-Agent: *
Disallow: /sb.php*$

Sitemap: http://www.collegeacb.com/sitemap.xml

Google support these symbols "*$", and they will stop Google from indexing all pages that start with sb.php

 

[angilina]

How about trying this code?

User-Agent: *
Disallow: /sb.php*$

Sitemap: http://www.collegeacb.com/sitemap.xml

Google support these symbols "*$", and they will stop Google from indexing all pages that start with sb.php

 

[peaches017]

I don't think it worked. An example of a link that still shows up in google:

"joey lesner collegeacb" without quotes

 

[Litgen]

I don't think it worked. An example of a link that still shows up in google:

"joey lesner collegeacb" without quotes

I agree, that will not work

 

[peaches017]

I agree, that will not work

Ya :/

 

[enlytend]

Any chance those were indexed before you created the robots.txt entry? If so, you need to request removal.

 

[peaches017]

Any chance those were indexed before you created the robots.txt entry? If so, you need to request removal.

Yes, Google won't re-look at my site and realize they should remove them? I'll investigate.