thoughts as follows
- get a list of records which have duplicates
SELECT DISTINCTROW(id), COUNT(id) FROM table WHERE 1
- Add a temp field into the db (don't have to, just makes things easier). Then go through the list of returned id's, and for each one run this
SELECT * FROM table WHERE id='$id' ORDER BY DATE ASC LIMIT 1
this will take the oldest record for each duplicate (only one record per duplicate because of LIMIT 1)
you can change this last sql into a delete statement to automatically delete the oldest of each duplicate, or you can have i mark the temp field and delete later after an audit.
as it only deletes the oldest of each dupe, you may want to check how many rows are returned by the first sql (mysql_num_rows) and run it a few times until there are no more dupes.
try on test data first. Let me know if this works, i'd be really astounded!
adam