That is correct. Every time a browser goes to a PHP page on the server, a session is either started or resumed. On a very small server, there shouldn't be much of a performance issue, but if most pages aren't using sessions, it's a waste of resources, even if performance isn't highly effected.
Of course, if every (or a vast majority) of page on the server use sessions, you should not notice a performance hit.
I'd say that you'd probably be better off coding in the session_start() at the beginning of the pages using sessions. I generally put all the session functions and code into a single include file and include that file in every page for the site that needs sessions. This way, you can change the session code easily for every page at once.
-Rich