You've almost got it. It's really a 2 part solution, and you've almost got the first part. In a nut shell what you are trying to accomplish, is make one page act as many different pages. Inside the code you have, you are generating html for different pages, these pages don't exist, so you are getting the error.
What you want to do, is link to the same page, but add GET variables to the query string, to tell the page when it reloads, with what data it should load with.
Current URL
http://domain.com/entry_title
The way you currently have it, you are telling the link to go to a URL for the entries title, and not an html page. Unless you have 'entry_title' as a file, it will generate an error.
http://domain.com/page.php?entry_id=1
You want to generate something like that url, where it's linking to an actual PHP page, and you are supplying a point of reference for it to pull data. page.php would need someway to load the entry details based on the id, then build the appropriate display.
So you code would need to look something like this, and page.php would need the logic to sort out what to do with entry_id.
<?php
include ("connect.php");
$query ="SELECT * FROM weblog ORDER BY entrydate DESC LIMIT 10";
$result= mysql_query($query, $connection);
while ($a_row = mysql_fetch_assoc($result)) {
print "<p><a href=\"page.php?entry_id=".$a_row['entryid'] ."\">".$a_row['entrytitle'] ."</a></p>";
}
?>
This way the user is displayed the title, but when they click on it it goes back to page.php with the entries id as a reference, so you can do some more logic and build the display for that entry.
Sorry I can't add more at this point in time, I hope that get's you on your away, or enough information to research more. If you're still lost post another comment, I can try and elaborate more later today.