How do I delete redundant duplicates in 1 table

[Anon]

I need to delete redundant duplicates within the same table...
This table stores email adresses, name etc etc.
I can`t set the email field to be "uniqe" since I am loading huge txt files that i recieve from my boss. And sometimes he has duplicate emails in them.

Can someone help me?

[Anon]

I'm pretty sure there is an easy way to do this with awk, but I don't know the command. I'm sure docs on the net will explain how.

[Anon]

here's the classic technique, per joe celko. it uses a temporary working table.

//use DISTINCT to kill dups
insert into WorkingTable
select distinct * from MessedUpTable

//clean out messed up table
delete from MessedUpTable

//load cleaned up data
insert into MessedUpTable
select * from WorkingTable

//loose temp table
drop table WorkingTable

[Anon]

I found out how I can do it. And what you described there was a difficult way to do it.
All you have to do is run this SQL command:
SELECT postid FROM adresses GROUP BY epost HAVING count(epost) > 1 ;

This outputs all the rows that are redundant duplicates... All I have to do now is to delete the rows with these id`s!

Thanks for the help ppl... 🙂