In one system, to check for duplicate videos, I generated an MD5 hash of the uploaded file and stored it in the database with the rest of the information about the video. If an uploaded file had the same hash as a video already in the database it was considered a duplicate.
This was mainly to prevent accidental duplications, or multiple people accidentally uploading the same video: pretty much any change to the file so that it wasn't bit-for-bit identical would mean it would have a different hash (even if the playback looked exactly the same). But that was a change that would have to be done deliberately by the uploader.