Page Caching

Anon

Maybe someone can help me -

I have a script that parses a news page, I want to know if its possible to detect if it's changed (size, last modified?) and if so, parse the page, if not show a cached version.

Anyone got any ideas?

Anon

There is a problem though: if the news page is itself dynamic, then the content may have changed, but the code itself, and therefore any file-based 'last modified' check won't work the way you want it to (ie filemtime() )

Brett

Anon

I was thinking I could check filesize, but you can't check the size of a remote page... Any ideas?

Anon

I wrote a smiple app for my bookmarks, unfortunatly it doesn't work on all sites.

first what I tried doing was using fopen() to open the website, then do an md5 hash of the site, compare it to a preivous md5 of the site, if they dont match then we have a change. unfortunatly this cause my script to hang on what I beielve was web servers only supporting HTTP/1.0 not HTTP/1.1

so then I rewrote the app to connect to the remote site using fsockopen() on port 80, echo the header "HTTP www.none.com/index.html 1.1" or whatever the site was. and read the repley and md5 hash it, but I found this also wasnt 100% reliable, and unfortunatly I never had time to look into it more..

Chris Lee

Anon

Why not do this:

Have your script check to see whether the local copy is over a certain age, if it is then it requests a new one; here strategies branch: if the parsing is resource hungry, check the 'new' version against the old one to see whether you need to parse it. If you don't mind the occasional overhead of parsing the file, then just go ahead and parse it.

If the file isn't over a certain age (or if it is and the most recent version you snagged from the remote site is no different), it shows the cached version, clearly indicating that the version is cached and how old it is [ or the last time you checked ], providing the user with the opportunity to snag a new version.

[ Of course you might want to add some extra code here to make sure someone can't DoS you by constantly requesting a large file which you then have to parse ]

Is this appropriate for your site?

HTH

Anon

That is what I had in mind, I just need to figure out the PHP behind it..