Revalidation, why doesn't it work.

[Anon]

I've written a tiny bit of code to demonstrate this,

Basically, even if I set all the headers, last modifed, etag etc... a conditional request (If modified since) still seems to return the identical page even if it shouldn't sent anything because it's not changed.

see:-

http://www.ircache.net/cgi-bin/cacheability.py?query=http%3A%2F%2Fwww.fsck.co.uk%2Ftesting%2Fetag-demo.php&descend=on

for a demo!

[Anon]

Have you tried to set the expiration to sometime last year? This should force the page to expire as soon as it is served, making the cache get a new one every time. At least in theory.

Jim Hawley

[Anon]

I want to ensure that the page is not downloaded unless absolutely necessary... not ensure that every request sends unnecessary data!

I expected that Apache would handle the conditional request if I supplied the correct headers, but it appears not.... unless I've missed somthing.

I don't really relish having to write a conditional request handler myself....

The transaction should look like this:-

GET /testing/etag.php HTTP/1.0
Host: www.fsck.co.uk
User-Agent: CacheabilityEngine/1.30 <http://www.mnot.net/cacheability/>
Accept: /
Pragma: no-cache
Cache-Control: no-cache
If-Modified-Since: Sat, 02 Dec 2000 15:33:14 GMT
Connection: close

HTTP/1.1 304 Not Modified
Date: Sat, 02 Dec 2000 15:35:44 GMT
Server: Apache/1.3.14 (Unix) PHP/4.0.3pl1
Connection: close
ETag: "1c720-28-3a29163a"

NOT, like this:-

GET /testing/etag.php HTTP/1.0
Host: www.fsck.co.uk
User-Agent: CacheabilityEngine/1.30 <http://www.mnot.net/cacheability/>
Accept: /
Pragma: no-cache
Cache-Control: no-cache
If-Modified-Since: Fri, 04 Feb 2000 00:56:36 GMT
Connection: close

HTTP/1.1 200 OK
Date: Sat, 02 Dec 2000 15:26:26 GMT
Server: Apache/1.3.14 (Unix) PHP/4.0.3pl1
X-Powered-By: PHP/4.0.3pl1
Expires: Fri, 04 Feb 2001 00:56:36 GMT
Last-Modified: Fri, 04 Feb 2000 00:56:36 GMT
Cache-Control: public, must-revalidate
Etag: 17627681623866
Content-Length: 40
Connection: close
Content-Type: text/html

<hr>
This is a test
<hr>

[Anon]

If-modified-since is not handled the same when the page is static and when it's dynamic. For a static page, the web server is able by itself to say if the date has changed (it simply checks the file's date). For a dynamic page, it can't do the same (it would have no sense) so it executes the script. Therefore, you have to do the check yourself at the beginning of your script and return a 304 header if necessary, i.e, if you estimate that the result page hasn't changed since the previous date. Only you can perform this check; the date could be, for instance, the timestamp of the last record inserted into a table containing some news.

Other possibility : if you know in advance for how long the page won't change, you can use the Expires header. With this information, the browser won't make another request until expiration time.

[Anon]

This is basically the conclusion I have come to, but I still dont see why PHP/apache can't handle the conditional requests once I have set the Last-Modified and Etag headers.

I guess that's a feature request!

The only remaining questions are...

1. Is there a function to convert the HTTP date format back to epoch time.

2. Is there a way get to the If-Modified-Since header without using a loop to step through getallheaders()

3. Are any of the other conditional headers used regularly or is it really just If-Modified-Since? How good is the browser support for this because it could improve performance a lot.

and

4. If I set must-revalidate, what is the behaviour if the object has not expired? What is the function (if any) of the expires header in that situation?

Thanks in advance, I'm pleased to get a useful answer even if it isn't the one I'd hoped!

Olivier wrote:

If-modified-since is not handled the same when the page is static and when it's dynamic. For a static page, the web server is able by itself to say if the date has changed (it simply checks the file's date). For a dynamic page, it can't do the same (it would have no sense) so it executes the script. Therefore, you have to do the check yourself at the beginning of your script and return a 304 header if necessary, i.e, if you estimate that the result page hasn't changed since the previous date. Only you can perform this check; the date could be, for instance, the timestamp of the last record inserted into a table containing some news.

Other possibility : if you know in advance for how long the page won't change, you can use the Expires header. With this information, the browser won't make another request until expiration time.

[Anon]

"1. Is there a function to convert the HTTP date format back to epoch time."

I've tried strtotime(), it seems to work perfectly.

"2. Is there a way to get the If-Modified-Since header without using a loop to step through getallheaders()"

$HTTP_IF_MODIFIED_SINCE

"3. Are any of the other conditional headers used regularly or is it really just If-Modified-Since? How good is the browser support for this because it could improve performance a lot."

If-Modified-Since is the most widely used and is supported by all main browsers.

"4. If I set must-revalidate, what is the behaviour if the object has not expired? What is the function (if any) of the expires header in that situation?"

Must-revalidate is, I think, intended for proxies only, not for browsers. It's used to force proxies to contact the server again (with an If-Modified-Since) instead of lazily using their cache.
If the page has an expiration date, the browser makes no request, so the proxy isn't contacted neither. In fact, it makes little sense to use both Last-Modified and Expires dates; you choose one system.