Unknown Bots eating up my valuable bandwidth!! How to stop!!

efil · 06-09-2009, 10:35 AM

My awstats shows Unknown robot(identified by empty user agent string) 35662 hits 1.04GB bandwidth usage.
Thats a huge amount of bandwidth used by unknown bots.

What I wanted to confirm was are these of any use?? If not how do I stop them from eating up my valuable bandwidth.

Tried to check out the logs. And it shows some ip which belong to some Worldcall Telecom Ltd pakistan similarly also shows my own host ip as repeated continous hits.

What does this mean??

need help!

HappyDay · 06-09-2009, 11:53 AM

Use your .htaccess to ban these bad bots. But before you do this you need to find out if they are worthless.
Ban a single IP:
deny from "IP goes here"

????: NamePros.com http://www.namepros.com/web-hosting-discussion/588889-unknown-bots-eating-up-my-valuable.html
or

deny from "IP with /cidr"

or via user agent e.g short example.:

Quote:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^Alexibot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Yeti/0.01 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xenu's [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus
RewriteRule ^.* - [F,L]

Jack · 06-09-2009, 02:02 PM

You should look in to creating a robots.txt - you'll have to google to find an example that meets your needs.

HappyDay · 06-09-2009, 02:11 PM

Bad bots don´t follow the rules in robots.txt
So what was the IP you found in your log files ?

sdsinc · 06-09-2009, 02:28 PM

Ban the IP in your .htaccess file

SimpleMan · 06-09-2009, 04:23 PM

There's frankly no telling what the bot was up to; chances are it's not something good, but it could be a mistake/bug. Regardless, the easy way out is simply to ban the offending IP(s). .htaccess is a quick way to do this, and that has already been laid out for you. It's also one of the easiest on resources.

.Andy · 06-09-2009, 10:23 PM

Talk to your host they can look at logs and see what exactly is the problem. You probably can block there ip.

rumsfo · 06-10-2009, 02:28 AM

Originally Posted by sdsinc

Ban the IP in your .htaccess file

They might appear with another one. Maybe it worth to read about their activity to think over the measures to be taken?

efil · 06-10-2009, 10:01 AM

Mailed my host with this issue. Waiting for reply. But what bothers me is why does my own hosts IP appear in that list and repeated hits?? And the hits point out to images in my directory.

~ServerPoint~ · 06-11-2009, 01:50 AM

What is the bot name I wonder?

efil · 06-11-2009, 11:49 AM

tracked out these two. Can anyone verify if they are useful. Checked out in the blacklist at What Is My IP Address? - Is my IP address blacklisted? and they appear to be listed.

119.152.40.205
????: NamePros.com http://www.namepros.com/showthread.php?t=588889
83.132.77.29

---------- Post added at 11:19 PM ---------- Previous post was at 10:47 PM ----------

Can writing something like this in htacess help

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

<Limit GET POST>
order deny,allow
deny from 212.100.249.69
deny from 119.152.40.205
deny from 83.132.77.29
</Limit>

Aesthetic · 06-12-2009, 12:18 AM

Could you tell which host are you using?

SiberForum · 06-12-2009, 12:59 AM

It seems that you need first block the IPs to stop them temporary and then you will need in any case to work on some kind of filters and probably will need to hire some designers

SeanPreston · 06-12-2009, 01:29 AM

I woyld suggest warning your host, if there worth being with they will sort it out for you.

Sean

efil · 06-12-2009, 05:09 AM

Had contacted the host. They replied that it isant an hosting issue or issiue with other sites on their hosting. They say that thats a issue related to the robot.txt file and advised to create a robot.txt file to stop the unwanted bots from acessing the site.

john23 · 08-19-2009, 05:15 AM

Simply use your .htaccess to stop these bots, if its not working; call your hosting company and talk to some technical person, instead of customer service. I did face couple of problems like this in past, i talk to hosting and CS guy said this not hosting, issue, after two days i called again and talk to some technical supervisor, then they solve that issue.

06-09-2009, 10:35 AM	THREAD STARTER #1 (permalink)
efil NamePros Member Join Date: Nov 2007 Location: http://www.goashost.com Posts: 96 Follow @goashost	Unknown Bots eating up my valuable bandwidth!! How to stop!! My awstats shows Unknown robot(identified by empty user agent string) 35662 hits 1.04GB bandwidth usage. Thats a huge amount of bandwidth used by unknown bots. What I wanted to confirm was are these of any use?? If not how do I stop them from eating up my valuable bandwidth. Tried to check out the logs. And it shows some ip which belong to some Worldcall Telecom Ltd pakistan similarly also shows my own host ip as repeated continous hits. What does this mean?? need help! __________________ Goa Online entertaiment Health Blog Alternative Medicines Top Videos

06-09-2009, 02:02 PM	#3 (permalink)
Jack NP Veteran Join Date: Oct 2005 Location: UK Posts: 474 Follow @jackheskett	You should look in to creating a robots.txt - you'll have to google to find an example that meets your needs. __________________ ~Jack

06-09-2009, 02:11 PM	#4 (permalink)
HappyDay NamePros Regular Join Date: Mar 2009 Location: Germany Posts: 525	Bad bots don´t follow the rules in robots.txt So what was the IP you found in your log files ? Last edited by HappyDay; 06-09-2009 at 02:16 PM.

06-09-2009, 02:28 PM	#5 (permalink)
sdsinc Domains my Dominion Join Date: Aug 2005 Location: Web 1.0 Posts: 9,558	Ban the IP in your .htaccess file __________________ NameNewsletter.com - free lists of available domain names ZoneFiles.net (beta) - ccTLD and gTLD droplists

06-10-2009, 10:01 AM	THREAD STARTER #9 (permalink)
efil NamePros Member Join Date: Nov 2007 Location: http://www.goashost.com Posts: 96 Follow @goashost	Mailed my host with this issue. Waiting for reply. But what bothers me is why does my own hosts IP appear in that list and repeated hits?? And the hits point out to images in my directory. __________________ Goa Online entertaiment Health Blog Alternative Medicines Top Videos

06-09-2009, 04:23 PM	#6 (permalink)
SimpleMan NamePros Expert Join Date: Oct 2004 Location: South Carolina Posts: 5,050	There's frankly no telling what the bot was up to; chances are it's not something good, but it could be a mistake/bug. Regardless, the easy way out is simply to ban the offending IP(s). .htaccess is a quick way to do this, and that has already been laid out for you. It's also one of the easiest on resources.

06-09-2009, 10:23 PM	#7 (permalink)
.Andy NamePros Regular Join Date: Feb 2007 Posts: 664	Talk to your host they can look at logs and see what exactly is the problem. You probably can block there ip.

06-11-2009, 01:50 AM	#10 (permalink)
~ServerPoint~ NamePros Regular Join Date: Nov 2007 Posts: 990	What is the bot name I wonder? __________________ ServerPoint.com - a true hosting company offering online presence solutions since 1998 Web Hosting, colocation, dedicated servers, Virtual Private Server (VPS) hosting *Wholly owned multi homed network, servers and facilities*

06-11-2009, 11:49 AM	THREAD STARTER #11 (permalink)
efil NamePros Member Join Date: Nov 2007 Location: http://www.goashost.com Posts: 96 Follow @goashost	tracked out these two. Can anyone verify if they are useful. Checked out in the blacklist at What Is My IP Address? - Is my IP address blacklisted? and they appear to be listed. 119.152.40.205 ????: NamePros.com http://www.namepros.com/showthread.php?t=588889 83.132.77.29 ---------- Post added at 11:19 PM ---------- Previous post was at 10:47 PM ---------- Can writing something like this in htacess help <IfModule mod_rewrite.c> RewriteEngine On RewriteBase / RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule . /index.php [L] </IfModule> <Limit GET POST> order deny,allow deny from 212.100.249.69 deny from 119.152.40.205 deny from 83.132.77.29 </Limit> __________________ Goa Online entertaiment Health Blog Alternative Medicines Top Videos

06-12-2009, 12:18 AM	#12 (permalink)
Aesthetic Account Closed Join Date: Oct 2006 Posts: 311	Could you tell which host are you using?

06-12-2009, 12:59 AM	#13 (permalink)
SiberForum Senior Member Join Date: Dec 2007 Posts: 1,456	It seems that you need first block the IPs to stop them temporary and then you will need in any case to work on some kind of filters and probably will need to hire some designers __________________ Canadian .CA Domain Name Registration and Web Site Hosting Sibername.com is an ICANN/CIRA/Eurid Accredited Registrar.

06-12-2009, 01:29 AM	#14 (permalink)
SeanPreston Senior Member Join Date: Nov 2005 Location: UK Posts: 1,278	I woyld suggest warning your host, if there worth being with they will sort it out for you. Sean

06-12-2009, 05:09 AM	THREAD STARTER #15 (permalink)
efil NamePros Member Join Date: Nov 2007 Location: http://www.goashost.com Posts: 96 Follow @goashost	Had contacted the host. They replied that it isant an hosting issue or issiue with other sites on their hosting. They say that thats a issue related to the robot.txt file and advised to create a robot.txt file to stop the unwanted bots from acessing the site. __________________ Goa Online entertaiment Health Blog Alternative Medicines Top Videos

08-19-2009, 05:15 AM	#16 (permalink)
john23 New Member Join Date: Jul 2009 Posts: 12	Simply use your .htaccess to stop these bots, if its not working; call your hosting company and talk to some technical person, instead of customer service. I did face couple of problems like this in past, i talk to hosting and CS guy said this not hosting, issue, after two days i called again and talk to some technical supervisor, then they solve that issue. __________________ Cheap Web Hosting \|\| Dedicated Server \|\| Web Hosting

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
propane.us and others closing soon on Afternic!	CKoelzer	Domains For Sale - External Auctions	0	04-06-2007 11:25 AM
Decisioncenter.com and others closing soon on Afternic!	CKoelzer	Domains For Sale - External Auctions	0	03-19-2007 08:55 AM
eBookFinder.com (and many others) Closing Soon on Afternic!	AWest	Domains For Sale - External Auctions	0	01-22-2007 02:18 PM
RestaurantOnWheels.com $75 reserve met, and many more!	AWest	For Sale / Advertising Board	0	11-21-2006 10:09 AM
Going, Going on Afternic	billinchina	For Sale / Advertising Board	47	03-31-2005 08:45 AM