Using PHP to pull a remote file, then measure it's http calls (images, etc).

jexley

Hey kids,
I'm new around here but have been using PHP to beef up several online tools that I'm actively using in my modest SEO/Consulting business.

I've got a client that's not unique in their concern for the download times their US-based customers experience as compared to those here in Australia. Or even over in the Eastern States as we are in Perth.

I've written some scripts that will pull the source code for their pages and is able to measure their size and download speed, and can parse through for <img> tags and CSS includes, but there are a few JavaScript calls and whatnot that I would prefer not to parse for.

So, my (long-winded) question is:
Does anyone out there know of how to pull down an HTML file and use PHP to detect which external HTTP calls that HTML file is making?

Thanks heaps, this site's already been great for me in figuring out quite a bit to get me here, so I'm looking forward to throwing more questions around.

Cheers,
-Judd

johanafm

Does it have to be written in PHP? There are working solutions that does just this allready, e.g. Firefox with the Firebug extension.

Open firebug, then load the page and you will see not only all requests, but also their results such as HTTP status, time spent waiting for response and how long it actually takes to receive data.

jexley

Yes, unfortunately.

That's a solid idea though, and I'll get that plugin for my own use, but I need to build it so that it's web-enabled and I can give a login to my clients to that they can check their own sites through the backend of my site.

johanafm

I'd suggest using PHP DOM

$dom = new DOMDocument();
$f = file_get_contents('...');
$dom->loatHTML($f);

$calls = array();

$imgs = $dom->getElementsById('img');
foreach($imgs as $img) {
	$calls[] = $img->attributes->getNamedItem('src');
}

I'm no expert, so I may be missing things you need to deal with, but at the very least they include:
link elements: href attribute
script elements: src attribute
img elements: src attribute

jexley

Cheers for that. That's what I'm using now actually.

Thing is, there's things in there like the AC_Content JS include, somewhat standard, Flash Display code to get around the Flash issues from a couple of years ago.

Depending on how it's loaded, there's quite a bit of stuff that Javascript can pull in without showing up in any "src" attribute. Same with CSS.

That's why I was hoping that somebody'd have some code that said:
"Read this HTML page like a browser would and tell me what you're downloading."

Thanks heaps for all the replies though.

johanafm

There is? I've never looked closely at it, but I got the impression the only thing it does is a document.write of an object (which also results in a request), but that will show up in your document anyway, so that should be covered.