I have a database driven site where the title, the content include, and other elements are pulled from a MySQL database, based on a variable in the page's URL. I was concerned that search engine robots would not read the title or the content of the included file and give it a low ranking.
I did a test with a free link-checker called Xenu (available on download.com). As far as I understand, Xenu is very similar to a search engine robot.
I ran the program and all of the pages were listed properly with their titles and embedded links checked. It appears as though Xenu correctly pulled the information from the database.
Could anyone tell me if this is an accurate testing method or recommend another way to test how search engine robots behave with database driven sites?
I read: http://www.phpbuilder.com/columns/tim19990117.php3 which discusses search engine robots and database driven sites, but I'd like to avoid altering Apache.