I've got a virtual server that has been encountering some slowness. Thanks to the hard-won wisdom from a prior experience, I decided to check the log files. It looks like the issue this time is that PHP-FPM's process pool often lacks enough processes and occasionally maxes out. Here's a snippet of a much larger php-fpm log file:

[22-Jan-2019 09:15:30] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 45 total children
[22-Jan-2019 09:15:31] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 47 total children
[22-Jan-2019 09:15:32] WARNING: [pool www] server reached pm.max_children setting (48), consider raising it
[22-Jan-2019 09:16:49] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 11 total children
[22-Jan-2019 09:17:04] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 13 total children
[22-Jan-2019 09:17:05] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 0 idle, and 15 total children
[22-Jan-2019 09:17:20] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 12 total children

Now I could surely edit /etc/php/7.0/fpm/pool.d/www.conf and change the current values:

pm = dynamic
pm.max_children = 48
pm.start_servers = 2
pm.min_spare_servers = 2
pm.max_spare_servers = 6

But I'm hoping to do something smarter than just Try It And See (TIAS). For starters, I'd like you guys to let me know if I'm thinking about this correctly.
- when a page request comes in, apache has a pool of processes to handle the request. if these are all too busy, we can get slow connections, connection timeouts, etc.
- once apache accepts an incoming request for a php script, it will contact the php-fpm pool of processes and try to hand off the request for processing. if the php-fpm processes are all busy, what happens? Will I see an error in the apache error log? Will there be 5xx responses in the apache access log?
- assuming that every page request is for a php script, apache and php-fpm should have roughly the same number of processes running. Or might apache need more because it also handles images and other static content?

In the interest of getting numbers that are suitable for my machine, it seems logical to set them high enough to consume all the CPU & RAM capacity on the server, but not much more. If we take on too many requests, we might start swapping memory and the server could grind to a halt. To get suitable values for PHP-FPM's pm.max_children and for apache's MaxRequestWorkers settings (the latter currently set to 300??) we would ideally have some idea of how much memory each process consumes. I've massaged these commands a bit, and hope you folks might tell me if I'm overlooking anything. In particular, I'm wondering:
- are these the right values for memory consumption?
- do I need to worry about double-counting of memory usage because i'm showing parent & child processes or something weird lack that?

So to get an idea of the PHP processes running and how much RAM & CPU they consume, I can call these commands. If anyone has suggestions -- in particular, I'd like to know how to make this data tab-separated -- that'd be nice.

ps -C php-fpm7.0 -o "uname,ppid,pid,%cpu,pmem,rss,command"

Right now I see 6-8 processes, each consuming about 30MB.

For Apache, this:

ps -C apache2 -o "uname,ppid,pid,%cpu,pmem,rss,command"

Currently I see about five processes, each consuming 20MB.

So if we use my caveman logic that every apache process needs a PHP process to serve it and we want to consume all of my 8GB of RAM (and totally ignoring the CPU consumption for the moment) then we can assume 50MB per apache/php pair and that comes out to about 8000MB/50MB = 160. I'm thinking I should set apache's MaxRequestWorkers and PHP-FPM's pm.max_children to 160.

Does that sound right? Any thoughts or input here would be much appreciated.

sneakyimp

Or might apache need more because it also handles images and other static content?

Yes, from here. But maybe not too many. Alas, I've not used PHP-FPM yet.

Is Apache the ONLY thing on this box? I wouldn't make it a goal to use all 8GB of RAM, although 75%/6GB wouldn't scare me. Is the DB server on a separate box?

    dalecosp

    dalecosp Alas, I've not used PHP-FPM yet.

    I was told by the apache guys on the #httpd IRC channel that event mode is much more efficient and scalable. The apache wiki says PHP-FPM is preferred above all other recipes, and is suitable for apache 2.4 and newer and offers some details about setting it up. I also documented the steps I used to configure apache using the debian/ubuntu package manager if you want to avoid compiling from source and stuff.

    dalecosp Is Apache the ONLY thing on this box? I wouldn't make it a goal to use all 8GB of RAM, although 75%/6GB wouldn't scare me. Is the DB server on a separate box?

    Yes, apache/php is the only thing this box does -- although there is some CLI PHP that happens (a variety of cron jobs which might have significant resource requirements). I do think I should reserve a GB or so for the non-webserver stuff. All DB work, mail, and some resource hosting is offloaded to db server, sendgrid, and a CDN, respectively.

    I'm mostly wondering if I have the basic calculus right:

    (TOTAL_MEMORY - ONE_GB_OR_SO_FOR_NON_WEB_SERVER_STUFF) / (MEMORY_PER_PHP_PROCESS + MEMORY_PER_APACHE_PROCESS) = MAX_WORKERS
    And then I assign MAX_WORKERS to apache's MaxRequestWorkers conf setting and to PHP-FPM's pm.max_children setting.

    That logic basically divides the memory available for web server duties (let's call it 6-7GB ) by the amount of RAM required by the combined memory consumed by one apache process and one php process.

    I'm also wondering if my ps commands will give me an accurate idea of how many processes we are talking about and how much RAM (and CPU) each one consumes. So far, there doesn't appear to be any heavey CPU work going on. I suppose I could further enhance this formula to take CPU into account.

    A couple of questions about the ps commands:

    • is RSS the right specifier? I read the man page which says it is resident set size, the non-swapped physical memory that a task has used (in kiloBytes).
    • does %cpu take into account the number of cores? does it show a fraction of total CPU power available on all cores or does it just refer to the percentage consumed on a single core?
    • does the output really show all the processes I need to consider or do I need to look for some kind of master parent process? Or child threads?

    Here's a typical outut of the php-fpm ps command:

    $ ps -C php-fpm7.0 -o "uname,ppid,pid,%cpu,pmem,rss,command"
USER      PPID   PID %CPU %MEM   RSS COMMAND
root         1  1424  0.0  0.3 31768 php-fpm: master process (/etc/php/7.0/fpm/php-fpm.conf)
www-data  1424  3750  1.6  0.5 44940 php-fpm: pool www
www-data  1424  4390  1.7  0.4 37940 php-fpm: pool www
www-data  1424  4482  1.5  0.4 39364 php-fpm: pool www
www-data  1424  4873  1.6  0.4 37684 php-fpm: pool www
www-data  1424  5032  1.7  0.3 31364 php-fpm: pool www
www-data  1424  5361  1.5  0.3 31628 php-fpm: pool www
www-data  1424 16113  0.0  0.3 27356 php-fpm: pool www

    I'd also love to know how I might get that output in a more structured format. It looks like fixed-with columns. It'd be great if we could get tab-separated or CSV format output.

            Does anyone have suggestions about getting a more highly structured data format from the ps command? The -o flag lets you specify an output string, but it doesn't really seem to offer much beyond selecting which columns there are.

              -o format
          described in the STANDARD FORMAT SPECIFIERS section below.  Headers may be renamed (ps -o pid,ruser=RealUser -o comm=Command) as desired.  If all column headers are empty (ps -o pid= -o comm=)
          then the header line will not be output.  Column width will increase as needed for wide headers; this may be used to widen up columns such as WCHAN (ps -o pid,wchan=WIDE-WCHAN-COLUMN -o comm).
          Explicit width control (ps opid,wchan:42,cmd) is offered too.  The behavior of ps -o pid=X,comm=Y varies with personality; output may be one column named "X,comm=Y" or two columns named "X" and
          "Y".  Use multiple -o options when in doubt.  Use the PS_FORMAT environment variable to specify a default as desired; DefSysV and DefBSD are macros that may be used to choose the default UNIX

              Not a power user of the ps command, but I've always used...

              ps auxw

              ...and often pipe it to grep if there's something specific I'm looking for.

                4 months later

                The platform I am using to host PHP server has Apache and PHP-FPM auto configured. The default configuration has proved to be the best so far. I don't have any issue with the performance. I am attaching the screenshot of my php-fpm config.

                [upl-image-preview url=https://board.phpbuilder.com/assets/files/2019-05-30/1559209962-497011-2-2-1024x501.png]

                  That's pretty subtle marketing there. 😉

                    Write a Reply...