2 Possibilities here:
1: It seems like the Yahoo Slurp bot has issues... serious ones, I think.
Take a look at this:
These are my site's top visitors. The address that ends with 208 is me. All the others until you reach the one ending in 183 are the Yahoo Slurp bot (according to IP traces [here]).
The bot has gobbled up about 20 gigs of bandwidth in just the last 20 days. I don't get it. This is [my site]... As you can see, there just isn't much going on there.
2: I'm wondering if it isn't perhaps a sneaky site stalker using fake IP's. I suspect a certain person that I used to be in contact with, but haven't been able to tell for certain... IF this were the case, I would think they're using some program like UpdatePatrol or NeoDownloader to keep track of changed pages/dirs, and download whatever's new or modified.
The activity coming from these IP's has increased gradually in the past months... starting from around 2 gigs per month, and growing quickly in the past couple months to 8 gigs, 12 gigs, and now 20... though site content has NOT grown that much at all (making me skeptical of whether bots would do this, unless the Yahoo bot is simply known to be this anal).
Any ideas? Does the Yahoo bot download your files (like rar files, etc), or what? I recently added a robots.txt to block the *real* Yahoo bot entirely, and then I'm going to see if activity from these IP's changes.
Thanks.
1: It seems like the Yahoo Slurp bot has issues... serious ones, I think.
Take a look at this:
These are my site's top visitors. The address that ends with 208 is me. All the others until you reach the one ending in 183 are the Yahoo Slurp bot (according to IP traces [here]).
The bot has gobbled up about 20 gigs of bandwidth in just the last 20 days. I don't get it. This is [my site]... As you can see, there just isn't much going on there.
2: I'm wondering if it isn't perhaps a sneaky site stalker using fake IP's. I suspect a certain person that I used to be in contact with, but haven't been able to tell for certain... IF this were the case, I would think they're using some program like UpdatePatrol or NeoDownloader to keep track of changed pages/dirs, and download whatever's new or modified.
The activity coming from these IP's has increased gradually in the past months... starting from around 2 gigs per month, and growing quickly in the past couple months to 8 gigs, 12 gigs, and now 20... though site content has NOT grown that much at all (making me skeptical of whether bots would do this, unless the Yahoo bot is simply known to be this anal).
Any ideas? Does the Yahoo bot download your files (like rar files, etc), or what? I recently added a robots.txt to block the *real* Yahoo bot entirely, and then I'm going to see if activity from these IP's changes.
Thanks.
