In search of a product search Script?

[hangman1919]

Not sure if the title makes sense, but I'm in need of script/technology software that works like amazon.com by retrieving products from other ecommerce or related sites.

The idea is for the script to search the web and find products based on the search information. Once the data is retrieved it should list the appropriate product with all the available websites/vendors and price (like amazon), and like amazon we would like to maintain all transactions in our system opposed to sending them to the websites like shopping.com does.

I'm hoping that someone here could point me in the right direction. I could provide more deatils if necessary, and rep will be rovided for those who help.

Thanks ahead of time for the look and replies.

 

[kceleb9]

Hi - what you are talking about could be many things - Getting product feeds from major sites? Remote API access - all the way to screen scraping.

In any case - you are not looking for a simple script here - i hate to break it to you - but this is a complicated piece of software - with need for configuration - and probably continuous updates - to make sure this is retrieving information in the way you want it to.

I say it is a system - since you will need your own DB to track rules / etc. for retrieving from various sites.

It is likely to be something customized - though you might work with existing stuff, and adapt it a bit. Screen scraping stuff needs to be flexible enough to change as sites change their format - and how data is presented or can be retrieved.

 

[hangman1919]

Thank you for a much better understanding. I figured it woud be a bit more complex and not a cookie cutter application. Is there any software out their that already has some of these features available?

Thanks again.

 

[kceleb9]

Well - for search - a very very good engine is Lucene - one of the best embeddable search capabilities around.

For screen scraping & retrieving from other web pages - HTMLUnit is not a bad choice.

These are both in the Java realm - I am not a PHP expert - but there may be other offerings in that space as well.