"Digg.com type" duplicate checker

[Mattias]

I run the site dumbest(,)info and I am having some troubles with duplicate information being submitted. As of now I am currently reviewing every contribution manually and it is working fine for now but I am worried as the site database grows this will be virtually impossible to check manually. So my question basically is, how has the Digg duplicate checker been constructed? Any ideas on how I can build a similiar system for my site?

 

[Brujah]

That was fun to visit. Did you write the script? You could of course just compare the full text of a submission to see if it's entered. You might try a soundex and present the submitter with the option to decide if it's already been entered, or maybe if the soundex matches enough you can have it send you the info instead. PHP has a similar_text() function that might be useful.

 

[Mattias]

Brujah, you made my day! I had no clue about the similar_text() function, it seems to be exactly what I am looking for. I was thinking of developing a function that compared two strings after similarity in percent and this does EXACTLY that. Will try it out ASAP! Thanks again