Search engines and PHP

azurian

Does anyone know if the search engines can spider PHP pages?

Thanks

jpmoriarty

most will - but bear in mind that they'll just see what you would see if you loaded up the page. They won't (I don't think) catch anything that needs variables and such, since they won't submit those variables to the page. So if your pages work by doing something like

include_once($_GET[page].".php");

then it won't see the contents of that page, since it won't supply a GET variable when it's spidering.

azurian

Thanks John

deurwaarder

I know, old topic, but long live search!

So: spider/ bots read the HTML part of a php page. what about content between <?php ?>

Like:

<?php echo "John bought some bread"; ?>

do they read that?

I’m just a bit worried, because all my pages are php. i'm on a learning trip regarding search engines and i'm finding out that there is allot to it!

Weedpacket

They see whatever anyone else would see if they hit it with their browser. Spiders are web clients just like any other and they work in exactly the same way as other web clients such as browsers. They ask the server for a page and read what the server sends back.

deurwaarder

Weedpacket wrote:
They see whatever anyone else would see if they hit it with their browser. Spiders are web clients just like any other and they work in exactly the same way as other web clients such as browsers. They ask the server for a page and read what the server sends back.

Yes, that is what i wanted to hear. thanks.

Btw, i see you quite often, and am always fascinated by the quote in your sig. older visions and predictions regarding "internet"

MarkR

Web spiders can spider PHP pages.

But they won't go infinitely deep into your site (obviously), especially if you use query strings, especially if you have many similar pages.

Tips:

Make each page have JUST ONE URL. So always be totally consistent when you refer to it. So if you have blog.php?id=1 don't also have blog.php?id=1&useless=42 or other URLs pointing to the same item. Also watch for stuff like blog.php?cat=1&id=2 and blog.php?id=2&cat=1 which are DIFFERENT URLs.

Search engines have limited resources to spider your site - ensure that they use them wisely.

Use robots.txt to ban robots from pages that are of no interest to them. This is particularly relevant if you have something like logon.php?from=somepage.php linked from somepage.php - You don't want to double the number of pages on your site with links to logon.php with different query strings.

Normal pages to ban robots from are:
- Logon pages
- Register pages
- Contact forms
- Anything else which requires interaction and is therefore of no interest to a bot.

Send back sensible html.
Make page titles relevant and different on each page! This is really important.
Use real links, not javascript dropdowns or other malarky (Except for areas of your site which search engines are forbidden from anyway)
Make headings real headings, not <div> etc
Make your HTML well-formed and valid, where possible. Search engines can handle invalid HTML to a point, but may less resilient than browsers.

Mark