Put the desired markup into a separate file, for example "header.html". At the start of each page, [man]include[/man] that file. In this example, I'm assuming the included file exists in a directory called "includes" just under your root web directory:
<?php
include $_SERVER['DOCUMENT_ROOT'].'/includes/header.html';
?>