there is another problem with HTTP_REFERER. If a valid user is grabbing the file from your site as intended, it still may fail becuase some proxies will not send the referring page in the headers. You could be keeping legitimate users from seeing what you want them to see.
Bottom line, if you make a file public on your site, it's basically public for everyone to find. you can only do so much to limit usage, but a determined person will always find a way, unless you protect everything via htaccess.