Need little Help

[earn4sure]

Hi,

I hope anybody help my friend out of this problem.

My friend' website has limit of 700 MB/per month bandwidth Limit. Now when he checked 2 days back there is mostly bandwidth used his search engines.Please see below given status.

11 different robots* Hits Bandwidth Last visit
MSNBot 3191 66.75 MB 30 Nov 2005 - 02:39
Unknown robot (identified by 'crawl') 1008 380.99 MB 27 Nov 2005 - 07:54
Googlebot 296 106.83 MB 30 Nov 2005 - 01:04
Inktomi Slurp 167 2.33 MB 29 Nov 2005 - 23:22
Unknown robot (identified by 'spider') 154 3.16 MB 28 Nov 2005 - 19:47
AskJeeves 111 3.61 MB 30 Nov 2005 - 01:58
Alexa (IA Archiver) 60 1.42 MB 27 Nov 2005 - 22:25
LinkWalker 19 252.01 KB 25 Nov 2005 - 07:45
WISENutbot 17 368.72 KB 26 Nov 2005 - 07:11
Walhello appie 8 148.45 KB 23 Nov 2005 - 20:32
Unknown robot (identified by 'robot') 1 24.19 KB 19 Nov 2005 - 00:45

So please tell me how i prevent his bandwidth from unwanted search engines.

Regards.

 

[Eric]

Ehh well. You could possibly use robots.txt .

User-agent: *
Disallow: /

Save as robots.txt and put in the main root folder. All "well behaved" bots should obey the robots.txt file.

However not all do... Anyway, the above will block ALL bots, if you just want to block certain ones ....

User-agent: botname
Disallow: /

User-agent: botname2
Disallow: /

etc..

Hope this helps.

-Eric

 

[earn4sure]

He use google adsense(He is google adsense publisher). So how he only allow google adsense user and msn user ?(which line he has to add in robots.txt) ?