NamePros
Welcome, Guest! Ready to make a name for yourself in the domain business? We welcome both the hobbyist and professional domainer to join the discussion as part of the NamePros community.

Click here to create your profile to start earning reputation for posting, and trader ratings for buying & selling in our free e-marketplace. Build your trader rating with each successful sale. Our system has tracked over 100,000 sales and counting!
FAQ & TOS Register Search Today's Posts Mark Forums Read

Go Back   NamePros.com > Website Development Discussion Forums > Programming > CODE
Reload this Page Simple Search Engine

CODE This forum is for posting code snippets and example scripts that aren't quite tutorials, but could be useful for others. You may post code snippets and/or completed scripts that you've written and want to share here.

Advanced Search
7 members in live chat ~  


Closed Thread
 
LinkBack Thread Tools
Old 09-29-2006, 02:05 PM THREAD STARTER               #1 (permalink)
Senior Member
 
Barrucadu's Avatar
Join Date: Aug 2005
Location: East Yorkshire, England
Posts: 2,689
Barrucadu is a splendid one to beholdBarrucadu is a splendid one to beholdBarrucadu is a splendid one to beholdBarrucadu is a splendid one to beholdBarrucadu is a splendid one to beholdBarrucadu is a splendid one to beholdBarrucadu is a splendid one to behold
 




Simple Search Engine


Demo

What this does is search a pages contents for a keyword, if the keyword is found, the uri is added to an array, and every page on the array is scanned for links, which are then scanned for the keyword, etc.
????: NamePros.com http://www.namepros.com/code/242552-simple-search-engine.html
The array returned is a user-definable size, and it includes options to limit the spider to only crawl specific domains.

It's a bit slow because it indexes all the pages on the fly, if you made it store them in a database, and then search through the database it would be so much faster, but this is just a simple bit of code for people. Also: because it can be slow, I wouldnt recommend setting the max number of pages to much more than 100, thats what the demo is set to and you can see how slow it is (over a minute and a half, lol)

ToDo:
  • robots.txt support
  • <meta> robots support
  • speed it up a bit
Attached Files
File Type: zip Spider.zip (1.5 KB, 18 views)
Barrucadu is offline  
Old 10-04-2006, 09:51 PM   #2 (permalink)
 
kleszcz's Avatar
Join Date: Jul 2006
Posts: 4,609
kleszcz Has achieved greatnesskleszcz Has achieved greatnesskleszcz Has achieved greatnesskleszcz Has achieved greatnesskleszcz Has achieved greatnesskleszcz Has achieved greatnesskleszcz Has achieved greatnesskleszcz Has achieved greatnesskleszcz Has achieved greatnesskleszcz Has achieved greatnesskleszcz Has achieved greatness
 



Marrow Donor Program Multiple Sclerosis
Yeah, I think additional caching function would make it awesome search engine script. Looking forward to stable version.
kleszcz is offline  
Old 10-20-2006, 01:16 PM   #3 (permalink)
NamePros Regular
Join Date: Oct 2006
Location: Inside your firmware.......
Posts: 217
boebi will become famous soon enoughboebi will become famous soon enough
 



Yeah, if would have some more speed, and some better stuff (like better design,caching),
i would run it on one of my site
boebi is offline  
Old 11-01-2006, 08:01 AM   #4 (permalink)
NamePros Member
Join Date: Mar 2006
Location: Belgium
Posts: 77
wolvenraider is an unknown quantity at this point
 



Originally Posted by boebi
Yeah, if would have some more speed, and some better stuff (like better design,caching),
i would run it on one of my site
Actually I don't see the problem with the layout, it can be easily edited in the demo.php cpage, a simple fieldset would make it good. But that's probably my point of view as a website creator
wolvenraider is offline  
Closed Thread


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools


 
All times are GMT -7. The time now is 03:03 PM.

Domain name forum recommended by Domaining.com Powered by: vBulletin® Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.
Search Engine Friendly URLs by vBSEO 3.6.0 Ad Management plugin by RedTyger