Duplicates?

[apower]

i have a somewhat larg txt file that has song list in it however there are tons of Duplicates is there an easy way to get rid of the Duplicates?

 

[shilpamg]

what exactly you are asking i m not getting. please be more specific.

 

[weblord]

how it was formatted? can you copy/paste some sample entries?

 

[Unknown]

i have a somewhat larg txt file that has song list in it however there are tons of Duplicates is there an easy way to get rid of the Duplicates?

If its a raw text file thats organized in some way (marking for beginning of each song list/line break after each song) your best bet would be to run a script of some sort on it which wouldn't be too hard to code.

 

[nasaboy007]

<?php
$filename = "songs.txt";
$handle = fopen($filename, "r");
$contents = fread($handle, filesize($filename));
fclose($handle);

$list = explode('\n', $contents);
$result = array_unique($list);

foreach($result as $key=>$value) {
echo $value . "<br />";
}
?>

I just wrote up this quick PHP script assuming its a flat text file with the songs separated by new lines. It'll output it to browser window so just copypaste the results, they should be unique'd.

 

[neosource]

if it's under 10,000 lines long you can try this online script..

http://my-i-p.com/Remove_Duplicate_Lines_from_Text.html

Note: It doesn't trim leading/trailing whitespaces though, so if there's any whitespaces on the duplicates it won't sort or remove them.

I have a similar javascript script I've written that I use for cleaning & sorting keywords etc, though I'd probably need to tweak it to suit before I could post it publicly. So I found this site quickly as it may do the trick..

hope it helps,

Cheers,

Joe

 

[neosource]

hi apower,

if the file is less than 10,000 lines long you can use the following online script:

http://my-i-p.com/Remove_Duplicate_Lines_from_Text.html

Note: It doesn't sort or remove leading/trailing spaces on the lines..

I have a javascript form that I've created that does sort/strip leading/trailing spaces, though it needs some tweaks before I'd make it public.. I use it for sorting keyword lists etc.

Hope that helps.

Cheers,

Joe

 

[andr69]

It's the simple algorithm from school. You can write it on any programming language.