Well, there's really only a limited number of ways to make things faster:
1.) Increase the speed of the computer (memory, better CPU, etc.)
2.) Use a lower foot-print / faster server (LightTPD better than Apache, etc.)
3.) Rework your code to be faster
Other than that, the server itself is bogging down somewhere. Whether you choose to find where that bottleneck is (either by tracking times) is up to you. It's best to see what functions are causing your server to really go slowly. Perhaps you're doing some significant MySQL queries that take up lots of time. If that MySQL server is on your mac with Lighty, then there's a possible issue. If you can afford it, MySQL should be kept separate so that you can gain performance in MySQL and your server.
That's all I know. I've never run apache on a Mac before (though I want a mac) so I can't say definitively what the issue is.