How important is memory usage?

jenni

I am trying to optimize some scripts and I was wondering how important memory usage was. If the memory usage is high, what affects can it have? I know it depends on the "umph" of your server, but are there general memory usage goals we should strive for? If it does matter, what can help reduce memory usage?

Piranha

Generally speaking developer time is more important than memory usage. What do I mean by that? Memory (and hardware in general) are cheap compared to the cost having a developer optimizing code. The first question to ask when it comes to optimization is if it is worth the time and effort, or if better hardware would solve the problems.

Another question to ask is why we need optimization. Scripts with few page hits normally doesn't need to be optimized, but big sites (google) really need optimization.

Even if it is decided to optimize code there are more important things to consider than memory. A few examples regarding databases:
1. Only select the columns you need, and not every column in the table(s).
2. Only select the rows you need. If you want to show the first 10 results only select the first 10 results and not the whole 10.000 rows.
3. Use count(*) instead of selecting every row and count them in PHP.
4. Use indexes in the database. It may do the difference between a 10 second query and a 0.001 second query. Also make sure that you know how to use them.

laserlight

3. Use select() instead of selecting every row and count them in PHP.

That is probably a typo error and should be: "Use count() instead..."

Piranha

laserlight wrote:
That is probably a typo error and should be: "Use count(*) instead..."

It is indeed. Changed in the post. Thanks for noticing m8 🙂

Jack_McSlay

memory should not be a concern unless you're developing something that really takes up memory (which is rare on PHP). if you work with so much data it hogs the server's memory, you should start by thinking over how your system works instead of optimizing your code straight away. Most programs take at very most 16MB of ram per execution, and for a period of time far shorter than a second, when all the memory used on the script is freed.

A good hint is to obfuscate your code. By eliminating useless characters, it will decrease the size of the files, resulting in a shorter parsing time.

leatherback

Jack McSlay wrote:
A good hint is to obfuscate your code. By eliminating useless characters, it will decrease the size of the files, resulting in a shorter parsing time.

Afaik know, this makes no noticeable difference. Anybody done any benchmarking on this? I like to comment my code...

NogDog

As far as I'm concerned, if you're in a situation where the time it takes the PHP parser to skip over a comment is worth worrying about, then you probably should be using compiled executable programs instead of runtime parsed and compiled scripts.

bpat1434

If you're at the point where memory is being overloaded, perhaps its time to think about separating services on servers. By this I mean purchase a fairly good server for MySQL, Postgres and any other RDBMS you want, and keep Apache on the other one.

Of course, to help optimize even more, you could put LightTPD behind Apache and use Apache to server static content (HTML, images, css, js) and use LightTPD to handle all dynamic content (SHTML, PHP, Python, Perl). I've heard it's faster (and seen benchmark documents to prove it). I've never done it, but willing to try it 🙂

Weedpacket

NogDog wrote:
leatherback wrote:
Jack McSlay wrote:
By eliminating useless characters, it will decrease the size
of the files, resulting in a shorter parsing time.

Afaik know, this makes no noticeable difference. Anybody done any benchmarking
on this? I like to comment my code...
if you're in a situation where the
time it takes the PHP parser to skip over a comment is worth worrying about,
then you probably should be using compiled executable programs instead of
runtime parsed and compiled scripts.

Bytecode compiler. That would be smarter: eliminates the parsing step almost
completely, and allows the source to be as programmer-legible as possible (source code is primarily for the benefit of humans, after all. Without it we'd all be toggling machine code). An optimising bytecode compiler also offers advantages.

Generally, if it's not a busy part of the application, and/or it doesn't offer at least a half-order-of-magnitude performance gain then I've got better things to do with my time. If it takes me five minutes to speed a page up by ten nanoseconds then even if the page is being hit a thousand times a second around the clock it'll still take more than twenty years for the job to be worthwhile. Now if I could get a page that takes three seconds to load to load in half a second instead, and it takes half an hour to do the job ... at one page hit per second I'll be ahead within twelve hours.

That's time, not memory. But time is more important than memory. Memory's a cheap commodity, but time is a limited resource. (On a side note, php.ini's default maximum memory allocation used to be 8MB; now it's 128MB.)

bpat1434 wrote:
and use LightTPD to handle all dynamic content

I've heard it the other way around: using Lighttpd as a static content server (I think because the computational grunt of Apache is better suited for the dynamic load). I'm willing to guess the choice of filesystem would also be significant at this level.

TeraTask

One cannot discredit memory concerns without taking into context the program. Some PHP scripts which edit graphics frequently run out of memory. If you have lots and lots of connections coming in, then each "little" bit of memory will sum to be quite a bit of memory and may cause problems. The point? Look at your script and the server(s) it will run on. Optimize as necessary. Then, as you learn how to optimize things incorporate those techniques into future projects.

leatherback

TeraTask wrote:
One cannot discredit memory concerns without taking into context the program. Some PHP scripts which edit graphics frequently run out of memory. If you have lots and lots of connections coming in, then each "little" bit of memory will sum to be quite a bit of memory and may cause problems. The point? Look at your script and the server(s) it will run on. Optimize as necessary. Then, as you learn how to optimize things incorporate those techniques into future projects.

Abolutely true. But one has to be aware where the optimization goes. As Weed pointed out: Gianing a 2 ms gain in script parsing is a bit of a useless effort, as is winning 500bytes by removing spaces, in my view.

However, when you statr working on images, the input image size explosively affects your scripts' memory and processor loading. (I do image processing as my day job. I am unfortunately too well aware of it 😐)

Jack_McSlay

not inly media processing, but search functions are also known to hog the system completely. If you make up a search function that's basically a query "SELECT * FROM (...) WHERE content LIKE %$search%" you can be dead sure it's gonna hog your site if you get a lot of users.

(btw, who said anything about cleaning spaces? there's propably at least half a dozen free, automatic PHP obfuscators around, and it's dead easy to make one up yourself using token_get_all() )

bradgrafelman

Jack McSlay wrote:
there's propably at least half a dozen free, automatic PHP obfuscators around, and it's dead easy to make one up yourself using token_get_all()

True, but all those do is complicate your code and make it essentially unreadable and thus uneditable, while saving little (or no) time, memory, etc.

TeraTask

bradgrafelman wrote:
True, but all those do is complicate your code and make it essentially unreadable and thus uneditable, while saving little (or no) time, memory, etc.

And, yet, the PHP people saw fit to extend PHP to include just such a function:

http://us3.php.net/manual/en/function.php-strip-whitespace.php

Must have been drunk when they did it? lol. Silly PHP programmers just wasting their time building useless functions.

leatherback

TeraTask wrote:
And, yet, the PHP people saw fit to extend PHP to include just such a function:

http://us3.php.net/manual/en/function.php-strip-whitespace.php

Must have been drunk when they did it? lol. Silly PHP programmers just wasting their time building useless functions.

hm.. feels like you were taking thing personally here...

in any case.. Did you read what the function is for:

Returns the PHP source code in filename with PHP comments and whitespace removed. This may be useful for determining the amount of actual code in your scripts compared with the amount of comments. This is similar to using php -w from the commandline.

Yeah, does sound like they were drunk 😃

laserlight

And, yet, the PHP people saw fit to extend PHP to include just such a function:

http://us3.php.net/manual/en/functio...whitespace.php

Must have been drunk when they did it? lol. Silly PHP programmers just wasting their time building useless functions.

Perhaps you have missed the part that says that "this may be useful for determining the amount of actual code in your scripts compared with the amount of comments". Clearly the function in question is not intended for code obfuscation as it would be rather easy to undo the obfuscation. It is probably not intended to speed up the processing of the code either, since a bytecode compiler would be far more efficient at that task.

TeraTask

Not, not taking it personally. Just providing a counter point. And, yes, I read what the function is for. I think the truth of the origination of the function is that a lot of people go out and get programs to do just that, so they added it into the core language. I've known quite a few people who have done just that. I haven't known anyone who was "trying to determine the amount of actual code", however.

Brad's argument was a de facto argument against compression.

Someone else argued against a suggestion I put forth because it would save only a fraction of a second.

Some others have argued against optimizations b/c they aren't significant.

Those individuals seem to be missing the point: Optimization is optimization. When you take a .002 optimization and multiply it by 1000, you get a 2 second savings. One must think in terms of scalability when looking at those mini optimizations b/c you never know when the site you're working on will get 1000 or more visitors in a short amount of time. That 2 seconds may be the difference between your server staying up and going down.

laserlight

Those individuals seem to be missing the point: Optimization is optimization. When you take a .002 optimization and multiply it by 1000, you get a 2 second savings. One must think in terms of scalability when looking at those mini optimizations b/c you never know when the site you're working on will get 1000 or more visitors in a short amount of time. That 2 seconds may be the difference between your server staying up and going down.

The thing is, this optimisation is inferior to the use of a bytecode compiler, and requires the same amount of maintenance (i.e., two copies of the code must exist). Consequently, there is no reason to prefer it if the optimisation is necessary.

TeraTask

laserlight wrote:
The thing is, this optimisation is inferior to the use of a bytecode compiler, and requires the same amount of maintenance (i.e., two copies of the code must exist). Consequently, there is no reason to prefer it if the optimisation is necessary.

OK. I'll byte. 😃 Which bytecode compiler do you use that's free, not experimental, and available on lots of hosts?

laserlight

Which bytecode compiler do you use that's free, not experimental, and available on lots of hosts?

I don't use a bytecode compiler at all (though I would use Zend's if I saw the need), but then I am a hobbyist programmer (at least for the next two years, due to compulsory military service that ate two precious years of my life), so I do not consider that optimisation necessary.

The way I see it is that we should concentrate on more important issues (i.e., optimisations with more impact as opposed to small efficiencies). This particular optimisation, whether one uses a bytecode compiler or a simple whitespace stripping function, is easy to apply as an after thought. Of course, you could argue that because it is easy to apply, it should always be applied, but then there is still a small cost in maintenance, as opposed to other minor optimisations that have negligible additional cost.