if (mysql_num_rows(mysql_query("select * from table where name = '$name' limit 1"))) {
print("Name already in use!");
}
"limit 1" will make MySQL stop searching the table once it finds a first record (which is OK since you don't want to know more than that).
Don't forget to addSlashes() or mysql_escape_string() all PHP variables you use in SQL queries.