dont use javascript to do a redirect, that's where your problem lies.
if you want to redirect, use the header function in php.
header("Location: ?page=home");
using the referer is unreliable because referers can be masked by proxies.
with the javascript you are using, the browser is loading index.php first, then going to ?page=home. so the browser's back button considers the last page to be index.php, not the one before that.
when you send a header in php, it is executed before any other output to the browser, no html no nothing, and tells the browser where to go, but the back button will still go back to the correct page before that.