Profiling Tips &amp; Tricks

Profiling Tips & Tricks

bubblenut

There has been a thread running recently in which a lot of useful information - about common places where bottleknecks are found; methods of profiling and ultimately optimizing applications - has come up. I thought it might be usefull to have a thread dedicated to the subject.

I'll start the ball rolling.

The obligitary quotes

Donald Knuth wrote:
We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.

Michael A. Jackson wrote:
The First Rule of Program Optimization: Don't do it. The Second Rule of Program Optimization (for experts only!): Don't do it yet.

A few general terms (yes I'm a proud wiki-whore)
Performance Analysis
Optimization

Helpfull extensions and packages
Xdebug (also) - The defactor profiler for PHP

Helpfull tools
KCacheGrind or WinCacheGrind - Callgrind profile data visualization tools
top - Unix tool for seeing resource usage by process
vmstat - Unix tool for seeing resource usage
mytop - Tool for analyzing MySQL performance

OK, now that I've got some helpfull links out of the way I suppose I should do at least one helpfull hint.

Profile and optimize in iterations
First you profile your slow script / application get your results and, hopefully, you'll see a few clear bottleknecks at the top of the list. Look at all of the ones which stand ahead of the pack and figure out roughly what you could do to improve performance, then pick the one which is easiest and do it. Now, here's the bit lots of people miss, then profile again! Any change to the source code changes the playing field so the previous profile is no longer valid, the bottlekecks you saw before may not be there anymore and new ones may have reared their heads.

I'm sure loads of you have come accross loads of helpfull tools, extensions, articles and little tips which have helped you profile and optimize your scripts in the past, so post away.

Roger_Ramjet

Some useful stuff on Virtual Memory and how it is used and how it can affect performance.

Understanding the Linux Virtual Memory Manager (download tar.gz)
Is Swap Space Obselete?
vmstat Linux Manual Page

The most important statistic to look at first in vmstat is the amount of page swapping that is going on (SI/SO). If figures are zero or near zero then no page swapping is going on so memory is not the problem. Conversely if the figures are high then your server is page-thrashing.

Page thrashing does not mean that you have to install more memory per se, most often in means that your application and configuration are at fault and adding more memory will not solve the problem (Unless of course you are trying to run a web server with 128Mb RAM)

Sxooter

I just want to make a point about premature optimization. There's a big difference between following good design practice and premature optimization, and a lot of times people don't see the difference.

A good design practice is to design your app so that the fewest queries can do the most work in the database. I.e. don't do a select id from one table, then run a query for each id returned against another table. Instead join the tables together in one query and get all the data at once. Add obvious indexes on things you'll be selecting on.

Once your server is up and running, the next mistake often made is to wait until things start to break to start optimizing it. Once it's up and it works is the time to start analyzing how it's working.

It's rather easy to add some code to log the time different sections of different pages take to execute. You can even do things like put them in html comments, so you have this in each page:

more page html goes here...

display stuff, build a form, etc...

The cost of adding these to a page is pretty small, and you can always check some debug variable somewhere if you wanna turn them off.

Logging them, with the page name and / or get url to some centralized logger might help for analysis. Or, you can just run the pages by hand while the site's under some kind of nominal user load.

Lastly, a lot of work has gone into newer versions of PHP, Linux, Windows, MySQL, PostgreSQL, and the other tools you might be using. You don't have to run the latest bleeding edge, but let's face it, a modern machine running PHP 5.2, PostgreSQL 8.1, MySQL 5.0, A linux distro on a late model kernel (RHEL4, FC6, the latest stable debian), a late model BSD, or windows XP, is gonna run a heckuva a lot faster than the same machine running RedHat 5.2, PHP 3.0, PostgreSQL 7.2, MySQL 3.23, Windows NT, or an early flavor of BSD etc...

Configuring your database server to log long running queries, grabbing those queries, and running explain analyze on them (analyze is a postgresqlism) to find out why they're slow and how to either make the db run them faster or how to rewrite them is a vital step to maintaining good application performance.

My first web/db server was a Dual PPro200MHz with 256M of ran and 6x9Gig USCSI hard drives. That machine was fast back in its day. Before long, when we'd replaced it with a dual P-IV-2800MHz machine with 2 Gigs of ram, it became our testing and integration server. We caught problems fast because if the old machine couldn't run your app fast all by itself, the new server wasn't going to see it until you'd fixed it. We had about 150 custom applications on the main server, btw, so one bad apple could ruin everybody's day. I highly recommend developing on an underpowered server so you spot problems faster.

Weedpacket

Sxooter wrote:
Lastly, a lot of work has gone into newer versions of PHP, Linux, Windows, MySQL, PostgreSQL, and the other tools you might be using.

A further corollary to this is that profilings made against one version of any of these may not necessarily map linearly to another version: PHP's memory management for example underwent an overhaul going from 5.1 to 5.2. So if you upgrade you'll probably want to start profiling afresh; the bottlenecks may even have shifted.

Sxooter

Weedpacket wrote:
A further corollary to this is that profilings made against one version of any of these may not necessarily map linearly to another version: PHP's memory management for example underwent an overhaul going from 5.1 to 5.2. So if you upgrade you'll probably want to start profiling afresh; the bottlenecks may even have shifted.

That is so true! I had forgotten what happened when I went from PHP 3 to PHP4. PHP 3 was REALLY fast at doing things like reading a file into memory, mangling it all at once, and writing it back out. PHP 4 was much much slower with large in-memory structures, but much faster at reading and acting on one line. I had to rewrite a whole bunch of php scripts because of that.

Good point.