As they say in the security world, it depends on what your attack vector is.
For example, if you are concerned about some random web site putting a form on their web page and posting to your web page, then HTTP_REFERER might be fine because most users at that other web site aren't going to be interested in taking the time to spoof the HTTP_REFERER (if they even know how).
On the other hand, if you are concerned about ONE hacker making a copy of your form, putting it on his local computer, altering the form (changing the hidden variables, for example) and posting it to your site, then no, HTTP_REFERER will not be sufficient because someone sophisticated enough to do that can easy spoof the REFERER and this will only be a tiny speedbump for her. (And this is a really good reason why you always need to validate the incoming data).
One technique you can use is this: When you build the form, pick a large random number and put it in a hidden field in the form. Then write that number to a database. When you process the form, check to see if that number is in the database. If yes, then process the form AND delete the number from the database so that it can't be used again. This will prevent the first problem but not the second one.
Another technique you can use is to md5 encrypt data that you put in hidden fields. This completely prevents the hacker from altering the values of the hidden fields.
If you are concerned about someone hitting your form too often (let's say you have a mortgage calculator or something and they are trying to put the burden on your server to do the calculations), then you might have to check for multiple visits from the same IP. You might put a limit of 10 per hour and no more than two in any given minute or something like that.
I had one client that asked customers to call a phone number where they gave out a password that could be used to submit the form twice. Once the password was used twice, it was deleted from the database and couldn't be used again.
The solution to your problem depends on how people are abusing your form.