If anyone can reccomend a script, and I use it, free or paid ($100 or there about) script I will send you $15 via paypal.
I have used a php search script before, entered a url and it crawls the whole site, thats not what I need.
Heres what I need from it:
I need to be able to enter a URL in an admin panel and for the script to crawl the site, eg:
If i entered the following URL to crawl:
http://www.pimpingpages.com/myspace-layouts/
It would only index pages such as:
http://www.pimpingpages.com/myspace-layouts/abstract/sea-blue-splatter/
http://www.pimpingpages.com/myspace-layouts/abstract/roam-the-world/
http://www.pimpingpages.com/myspace-layouts/fashion/ckin2u-him/
and not the actual category pages, not the preview pages, not the main page or the site homepage, contact page etc.
Maybe a script that would allow me to set some option like crawl http://domain.com/folder/[a-z_0-9]/ and only crawl pages matching them options.
The results then stored in a mysql or postgresql database, I would want to do this with about 20 sites, each having between 200-5000 pages.
Just a note that the site mentioned above is just a site I found, it has nothing to do with myspace what im wanting.
A user enters a query and is given relevant results. I dont want the category pages showing up in the results because they also mention the search query.
PS. Im also open to people giving me a quote to code this. Need it asap, within a week.
Thanks,
Paul
I have used a php search script before, entered a url and it crawls the whole site, thats not what I need.
Heres what I need from it:
I need to be able to enter a URL in an admin panel and for the script to crawl the site, eg:
If i entered the following URL to crawl:
http://www.pimpingpages.com/myspace-layouts/
It would only index pages such as:
http://www.pimpingpages.com/myspace-layouts/abstract/sea-blue-splatter/
http://www.pimpingpages.com/myspace-layouts/abstract/roam-the-world/
http://www.pimpingpages.com/myspace-layouts/fashion/ckin2u-him/
and not the actual category pages, not the preview pages, not the main page or the site homepage, contact page etc.
Maybe a script that would allow me to set some option like crawl http://domain.com/folder/[a-z_0-9]/ and only crawl pages matching them options.
The results then stored in a mysql or postgresql database, I would want to do this with about 20 sites, each having between 200-5000 pages.
Just a note that the site mentioned above is just a site I found, it has nothing to do with myspace what im wanting.
A user enters a query and is given relevant results. I dont want the category pages showing up in the results because they also mention the search query.
PS. Im also open to people giving me a quote to code this. Need it asap, within a week.
Thanks,
Paul






