If you already have the information about referrals being stored in a database, then it's real easy. But from your question, I'm assuming you don't.
So from the page that you expect people to link to, you would want to start recording $_SERVER['HTTP_REFERER']. Note that this could easily be abused if you don't check IP addresses, so you'll probably want to only record one referral per ip address.
So, assuming you had the table set up with columns 'ip' and 'referrer', you could do something like this...
$ip = $_SERVER['REMOTE_ADDR'];
$refer = mysql_real_escape_string($_SERVER['HTTP_REFER']);
$alreadyLogged = mysql_num_rows(mysql_query("select * from table_name where ip='$ip' and referrer='$referrer'"));
if(!$alreadyLogged) mysql_query("insert into table_name (ip,referrer) values ('$ip','$referrer'") or die(mysql_error());
Then, you should be able to pull out the top 5 referrers with a query like this:
select referrer,count(ip) as referrals from table_name group by referrer order by referrals desc
That should work. Let me know how it goes.
EDIT: Oh, by the way. You might also want to filter out referrals from your own website, from search engines, or other sites that you don't want to link back to. You can use ereg() for that.