You should not use HTTP_REFERER for anything critical, including authentication or security checks of any kind.
An XMLHttpRequest request should be treated no differently to any other request with respect to authentication and authorisation:
- You must provide enough information for the server to identify WHO the requester is.
- You must check this identity on the server side.
- You must do all necessary checks to ensure that they are allowed to do the exact operation that they're trying to do.
This is necessary on every single request whether it originated via a normal web page or XHR.
Note that I believe that XHR is actually more resistant to CSRF attacks than conventional POSTs.
If you worry about CSRF, you should post additional tokens to try to prevent that too.
NB: CSRF is the least of your worries if there is even a tiny chance that your site has a XSS or SQL injection vulnerability.
Mark