Search Engines - How they work...

[Keith Allen]

I am curious as to how search engines work. How do they get all the results they have when someone uses the search?

Whats the best way to get that kind of a search script/site on a budget?

I've seen a few scripts, but they are the type where you add the links and descriptions yourself, and it searches for the description rather than the site...if that made any sense...

I want a google/yahoo style search script where it actually searches for the information on the pages rather than a link and description I had to add manually.

 

[cerebus]

Short answer is you need alot of bandwith and computing power to manage a proper web search engine so that it isn't possible on a tight budget.

The two core components of any search engine such as Google are the spider and the indexer. The spider crawls web pages and follow links across the web and the indexer indexes the information in structures to make it easier and faster to search.

To even crawl a decent ammount of pages too get enough information for a good searchable index requires loads and loads of bandwith. Then to index alot of information into structures that are easily searchable requires alot of knowledge and alot of storage and CPU resources.

 

[Mark]

Best to build a simple one with Your content - Then add anothers content as well ....

One of the best bets is to use the Google API .... They'll crawl your site and give you all the additional content in the world - lol - used it for a year or so ....