Chris2412
Established Member
- Impact
- 1
I am a n00b developer and I would like to talk about bots and how they crawl a web page.
I tried searching keyword “bot” on the forum but I got nill, just a bunch of random results. I’m sure this is a thread covering this so a link provided will do just fine.
So, Google has “bots” that crawl your page. It looks for keywords, phrases, ect for cataloging purposes. Which is good, because you want your website cataloged in their search engine.
But there are other kind of bots, too. Yes?
Some of these bots are evil pawns sent out to do- what exactly?
Eat your bandwidth?
I read a simple php code a year or two ago, that basically makes bot’s sleep (or time out).
I am not even close to HTML 5 yet so perhaps I am getting way ahead of myself.
There’s no harm in asking. Perhaps I can bump this thread as I get more knowledge and have additional questions regarding coding.
I tried searching keyword “bot” on the forum but I got nill, just a bunch of random results. I’m sure this is a thread covering this so a link provided will do just fine.
So, Google has “bots” that crawl your page. It looks for keywords, phrases, ect for cataloging purposes. Which is good, because you want your website cataloged in their search engine.
But there are other kind of bots, too. Yes?
Some of these bots are evil pawns sent out to do- what exactly?
Eat your bandwidth?
I read a simple php code a year or two ago, that basically makes bot’s sleep (or time out).
I am not even close to HTML 5 yet so perhaps I am getting way ahead of myself.
There’s no harm in asking. Perhaps I can bump this thread as I get more knowledge and have additional questions regarding coding.