Need help from experts

SpaceshipSpaceship
SpaceshipSpaceship
SpaceshipSpaceship
Watch

earn4sure

Established Member
Impact
0
Hi,

I hope anybody help my friend out of this problem.

My friend's domain has 700 MB/per month bandwidth Limit. Now when he checked 2 days back there is mostly bandwidth used his search engines.Please see below given status.

11 different robots* Hits Bandwidth Last visit
MSNBot 3191 66.75 MB 30 Nov 2005 - 02:39
Unknown robot (identified by 'crawl') 1008 380.99 MB 27 Nov 2005 - 07:54
Googlebot 296 106.83 MB 30 Nov 2005 - 01:04
Inktomi Slurp 167 2.33 MB 29 Nov 2005 - 23:22
Unknown robot (identified by 'spider') 154 3.16 MB 28 Nov 2005 - 19:47
AskJeeves 111 3.61 MB 30 Nov 2005 - 01:58
Alexa (IA Archiver) 60 1.42 MB 27 Nov 2005 - 22:25
LinkWalker 19 252.01 KB 25 Nov 2005 - 07:45
WISENutbot 17 368.72 KB 26 Nov 2005 - 07:11
Walhello appie 8 148.45 KB 23 Nov 2005 - 20:32
Unknown robot (identified by 'robot') 1 24.19 KB 19 Nov 2005 - 00:45

So please tell me how he prevent/save his bandwidth from unwanted search engines.

Regards.

:(
 
Last edited:
0
Close
The views expressed on this page by users and staff are their own, not those of NamePros.
GoDaddyGoDaddy
in the robots.txt he can disable robots from going to specific pages. It will dramatically drop his bandwidth. If the bots can only view his index, then he will be in good shape.
 
0
Close
How to make robots.txt ? He use google adsense (adsense publisher). So is there any problem after stop spiders.
 
0
Close
0
Close
be adviced that blocking some spiders can mean less visitors from search engines
 
0
Close
not if its like an admin section or something. Also config.php files don't need to be searched.
 
0
Close
so what if they are searched?
they don't output any html code
it only apears a blank page
 
0
Close
what??
 
0
Close
You do not want to block search engine bots, thats how you get indexed and ranked. It looks like Google and MSN are doing a full index of your site, which is a good thing, and is definitely worth the bandwidth. It won't always be as heavy after the initial indexing.
 
0
Close
I had a similar problem sometime ago, especially googleboot seemed liked my site a lot.

I found in the webmasters section of google site a reference to that problem.
You just write to them and tell them you have such problem.
They will reply in a week or so asking you to provide details,(so if you provide these data first you may reduce the time for answer)
They are able in some way to reduce the robot visits, at least it worked for me.
Hope it help
 
0
Close
Thank u all. I think thats my wrong decision to block search engine to visit my website. Thanks to all members for your proper help.

:)
 
0
Close
CatchedCatched

We're social

Escrow.com
Spaceship
Domain Recover
CryptoExchange.com

NamePros updates

Filters for activity feeds
Your news feed, Latest activity, and the activity tab on profiles now have filter buttons: All, Posts, Quick replies, and Other. Your...
AIVikings
Catchy
  • The sidebar remains visible by scrolling at a speed relative to the page’s height.
Back