Where has all the memory gone???

Anon

I recently ported a 2500 line script from Python to PHP. The script reads in 40,000+ records from a disk file and the comparable records from a database via a socket connnection, compares the two and updates the database with any changes.

Under Python there were no memory issues. With PHP, without memory limits imposed, the script chews up all 256MB of regular memory and starts in on the swap space. If I impose a memory limit, the script runs up to the limit and aborts.

The only way to get this script to run in under 64MB of memory is to "unset" every variable at the beginning of the main while loop. It still climbs from just under 3MB when it starts up to 55MB when it completes. If the input file gets any larger, I'll have to up the memory limit, I suppose.

Any comments on why PHP behaves this way? I would prefer to support a single language on the server and PHP is a comfortable choice. However, if I can't overcome this memory issue I guess I'll have to move on to Java or back to Python.

HELP!!

Scott

Anon

Do you know if your php interpreter was compiled with inline optimization turned on? I've found this increases memory usage, but php4 is definitely more memory hungry than php3 ever was.

So, either run php3, or rewrite your script to either process a single line at a time, or, if you can't do that, see about loading all that data into a database first, then doing the comparison there, which is probably a better place for data to exist anyway.

Anon

I did some testing on php4 versus php4 w/ inline optimization versus php3. Here's what I found:

file read: ~5megs

memory space used:

PHP4: 55 Megs
PHP4 w inlineopti: 55 Megs (same)
PHP3: 27 Megs.

However, I put a for next loop that just did a simple bit of math and store it in the same var $a, over and over. In php4, the script size jumped to 55Megs to hold the data, then sat there, but in php3, due to poor garbage collection, the memory usage ballooned up to over 150 Megs before the script timed out.

I think a 10 to 1 ratio of memory usage to file size is a bit extreme, and is something the zend folks ought to look at, but who knows if it's something they can change at this late date or not. Maybe it's time for PHP5 with a newer, more efficient inter storage system...