From mysql.com:
DELETE FROM somelog
WHERE user = 'jcole'
ORDER BY timestamp
LIMIT 1
Their query works just fine using the ORDER BY and LIMIT. Maybe your problem is something else. I'm not sure on this, but maybe if you use ORDER BY you need to use WHERE along with it. I'm not positive; it would require some playing around with. Are you using mysql_error() when you execute your query?
$query = mysql_query("DELETE FROM table_name WHERE id != 'a' ORDER BY column_name LIMIT 1") or die(mysql_error());
I added the WHERE (you can try without it). Your id field would be numbers, so they would never equal 'a'. This is just incase you do need the WHERE clause along with it, but you may be able to get away with not using it. Just make sure to replace table_name and column_name with their respective entries. mysql_error() will give you a more descriptive error message.
Cgraz