Why is it a problem to display the error message together with the form? Seems like a good idea to me. Popups are blocked by most browsers today anyway.
However, if you really want to do it, validate your form before sending any HTML to the browser (well, the body tag anyway.) In the body tag call a JAVASCRIPT function that I've included to show the errors (I'll let you figure out the rest.)
function errorWindow(message,pageWidth,pageHeight) {
newWindow = window.open("","newWindow","width="+pageWidth+",height="+pageHeight+",left=50,top=50,status=yes");
newWindow.document.open();
newWindow.document.write('<HTML><TITLE>Errors</TITLE>');
newWindow.document.write('<BODY bgcolor="#FFFFFF" LEFTMARGIN="0" TOPMARGIN="0" MARGINHEIGHT="0" MARGINWIDTH="0" onBlur="self.close()">');
newWindow.document.write(message);
newWindow.document.write('</BODY></HTML>');
newWindow.document.close();
newWindow.focus();
}
Or...dynamically create a layer that sits over the form and write the error messages in it. Provide a link or button to hide the layer.