Need .htaccess code to prevent Bing from indexing proxied pages

[vectro]

Bing is going crazy trying to index proxied pages from PHPRoxy. With Glype, it's easy to exclude browse.php in robots.txt. However, PHPRoxy doesn't use browse.php. It uses index.php. For example: /index.php?q=aHR0cDovL21pbmRjdHJs

I know it's possible to include user agents in .htaccess code. How can I use .htaccess to prevent Bing from crawling index.php?= (but without blocking index.php entirely) no matter what string comes after the = symbol?

 

[vectro]

I finally found a solution to this one. It works great for Glype.

Place this in .htaccess:

RewriteEngine On
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?example.com/ [NC]
RewriteRule ^browse\.php$ http://example.com/403.shtml [R,NC]

Just replace example.com with your actual domain name.

 

[kleszcz]

Following thread is also what you're looking for?

http://www.namepros.com/free-resources/182288-all-proxy-site-owners-look-here.html

 

[vectro]

Following thread is also what you're looking for?

http://www.namepros.com/free-resources/182288-all-proxy-site-owners-look-here.html

That solution only works on PHProxy and is built into the script. My solution is for Glype and is server-side instead of script-side. In certain cases, like this, I believe a server-side solution to be more robust.

On my proxy hosting servers, I recently implemented mod_security rules to keep search engine bots off of proxied pages in both Glype and PHProxy. That way the crawlers can see home pages and individual pages, just not large query strings off of browse.php and index.php.