Hello all,

I've wasted an entire work day on this problem, so if you can help me, I will send 9,000,000 karma points in your direction (and throw in wishes of snax and girls...if that's your thing).

I set up the following gallery upload page, which is essentially a copy of another working upload page that I'm currently using for a client. But, somehow, this second one creates an error. The only difference is that this new one should accept larger files (3.75M instead of 1M):
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 18432 bytes) in /path/to/Upload_Gal.php on line 110

Problem - I've defined MAX_FILE_SIZE to accept up to 3.75M (3932160 bytes). Even if there were three maxed out uploads, that would only add up to 10.5M total. The code seems to work fine for files that are 1M or smaller, but immediately fails with only a single "large" upload (one file at 3.5M).

PLEASE READ THIS
My php.ini file (which I've quadruple checked...paths and settings), allows 32M for uploads, files, posts, etc...so the problem isn't there, I don't think.
I've read, generally, that this could be due to bad coding - which I wouldn't doubt, since I'm no aficionado. But, I can't find where the problem is. The error says line 110, but that's general (marked below)

//STEP 2 UPLOAD
//max size 3.75 mb
define('MAX_FILE_SIZE',3932160);

if(array_key_exists('upload',$_POST) && isset($_SESSION['gal'])) {
	//define file sizes for orig and thumbs
	define('MAX_FILE_WIDTH',360);
	define('MAX_FILE_HEIGHT',360);
	define('MAX_THUMB_WIDTH',108);
	define('MAX_THUMB_HEIGHT',108);
	//define upload dir
	define('UPLOAD_DIR','..path/');
	//define thumbs_dir
	define('THUMBS_DIR',UPLOAD_DIR.'Th/');
	$gal = $_SESSION['gal'];
	//permitted MIME types
	$permitted = array('image/jpeg','image/pjpeg');
	//convert the max size (bytes) to MB
	$max = number_format(MAX_FILE_SIZE/1048576,1).'MB';


foreach($_FILES['image']['name'] as $number => $file) {
	//replace spaces & assign var to $file
	$file = str_replace(' ','_',$file);

	//assume file is unacceptable
	$name_lengthOK = false;
	$sizeOK = false;
	$typeOK =false;
	$unique = false;

	//check name length
	$file_length = strlen($file);
	if($file_length <= 25) {
		$name_lengthOK = true;
	}

	//check filesize
	if($_FILES['image']['size'][$number] > 0 && $_FILES['image']['size'][$number] <= MAX_FILE_SIZE) {
		$sizeOK = true;
	}

	//check MIME type
	foreach($permitted as $type) {
		if($type == $_FILES['image']['type'][$number]) {
			$typeOK = true;
			break;
		}
	}//endforeach

	//make sure image is not a duplicate
	if(!file_exists(UPLOAD_DIR.$file)) {
		$unique = true;
	}

	if($name_lengthOK && $sizeOK && $typeOK && $unique) {
		$original = $_FILES['image']['tmp_name'][$number];
		switch($_FILES['image']['error'][$number]) {
			case 0:		
		//move the file to the upload folder and rename it
		  //get new sizes from orig
		list($width, $height, $type) = getimagesize($original);
		//calculate ratios
		if ($width <= MAX_FILE_WIDTH && $height <= MAX_FILE_HEIGHT) {
			$file_ratio = 1;
			if($width > $height) {
				$thumb_ratio = MAX_THUMB_WIDTH/$width;
			}
			else {
				$thumb_ratio = MAX_THUMB_HEIGHT/$height;
			}
		}
		elseif ($width > $height) {
			$file_ratio = MAX_FILE_WIDTH/$width;
			$thumb_ratio = MAX_THUMB_WIDTH/$width;
		}
		else {
			$file_ratio = MAX_FILE_HEIGHT/$height;
			$thumb_ratio = MAX_THUMB_HEIGHT/$height;
		}
		//strip the extension off the image filename
		$imagetypes = array('/\.jpg$/','/\.jpeg$/');
		$name = preg_replace($imagetypes,'',basename($file));
		$source = imagecreatefromjpeg($original); //------THIS IS LINE 110 -------------------
		//calculate new dimensions
		$new_width = round($width * $file_ratio);
		$new_height = round($height * $file_ratio);
		$thumb_width = round($width * $thumb_ratio);
		$thumb_height = round($height * $thumb_ratio);
		//create image resources
		$new_file = imagecreatetruecolor($new_width, $new_height);
		$thumbnail = imagecreatetruecolor($thumb_width, $thumb_height);
		//create resized copies
		imagecopyresampled($new_file, $source, 0,0,0,0, $new_width, $new_height, $width, $height);
		imagecopyresampled($thumbnail, $source, 0,0,0,0, $thumb_width, $thumb_height, $width, $height);
		$sql_filename = mysql_real_escape_string($name.'.jpg');
		$sql = "INSERT INTO images (image_name,gallery) VALUES ('$sql_filename','$gal')";
		$sql_result = mysql_query($sql) or die(mysql_error());
		//save resized copy			
		$success = imagejpeg($new_file, UPLOAD_DIR.$name.'.jpg',100);
		$success .= imagejpeg($thumbnail, THUMBS_DIR.$name.'.jpg',100);
		imagedestroy($source);
		imagedestroy($new_file);
		imagedestroy($thumbnail);
		if($success) {
			$upload_result[] = '<span id="blue_text">"'."<em>$file</em>".'" '."UPLOADED SUCCESSFULLY</span>";
		}
		else {
			$upload_result[] = "Error uploading $file. Please try again.";
		}
		break;
		case 3:
		$upload_result[] = "An error occurred while uploading $file. PLEASE TRY AGAIN.";
		default:
		$upload_result[] = "SYSTEM ERROR uploading $file. Contact webmaster.";
		}//end switch
	}//endif($name_length...sizeOK)
	elseif(!$unique && $_FILES['image']['error'][$number] !== 4) {
		$upload_result[] = "ERROR: A file named $file already exists.";
	}
	elseif(!$name_lengthOK) {
		$upload_result[] = 'ERROR: "'."<em>$file</em>".'" '."must be renamed to be 25 characters or less.";
	}
	elseif ($_FILES['image']['error'][$number] == 4) {
		$upload_result[] = 'No file selected';
	}
	else {
		$upload_result[] = '"'."<em>$file</em>".'" '."COULD NOT BE UPLOADED:<br/>
		&nbsp;-&nbsp;<em>Max size: $max<br/>
		&nbsp;-&nbsp;jpeg only</em>";
	}
}//endforeach
}//end if $_POST
<p><strong>Step 2:</strong><br/>
    Once you've chosen a gallery to view (above), you may upload up to 3 photos at a time. Images will be resized for web-friendliness. However, if your files are large, upload may take some time.
    </p>
      <label for="image1">1. Choose Image:</label>
       <input type="hidden" name="MAX_FILE_SIZE" value = "<?php echo MAX_FILE_SIZE; ?>*" />
      <input type="file" name="image[]" id="image1" />
      <br/>
      <label for="image2">2. Choose Image:</label>
      <input type="file" name="image[]" id="image2" />
      <br/>
      <label for="image3">3. Choose Image:</label>
      <input type="file" name="image[]" id="image3" />
    </p>
    <p>
      <input type="submit" name="upload" id="upload" value="Upload Image(s)" />
    </p>


</div>
</form>

I DO NOT WANT A WORKAROUND, so please do not suggest that I allocate more memory, because I shouldn't have to. I need to know where I'm tricking php into thinking that either my files are larger than they are, or that it doesn't have enough memory to assume these tasks. Thanks in advance.

    How big is the image? I'm not talking about file size, but pixel dimensions: when you create the image on line 110 you'll need enough memory to store width&#215;height&#215;3 bytes' worth of pixel data.

      Of the selected files that are failing: 4608x3072
      Info says 3.7M on disc. But, when I open the file with Photoshop, it says 15"x10" at 300dpi. That's weird.

      What's strange is, shouldn't it just fail if it's too big, and echo the result[] of $max? Because, in the aforementioned working version of this, it simply says, "Unable to upload, because ...the message pertaining to max_file_size..."

        That message is based on the size on disc, however to create the image resource you do 4608 3072 3 = 42467328 bytes which is not the size on disc.

          understood, but then why is the code failing to recognize this as a scripted limitation (ie:MAX_FILE_SIZE)?

          If I try the same file(s) on the working version, the php error message works as scripted. Here's the working version (which accepts SMALLER images), with five of the 'large images':

          But, with this other one, the script simply fails with this "exhausted..." message - even with only one image:

            veeps wrote:

            understood, but then why is the code failing to recognize this as a scripted limitation (ie:MAX_FILE_SIZE)?

            Because MAX_FILE_SIZE refers to the maximum size of the file, not the size of the image.

            The script needs access to enough memory to store the actual pixel data - not the compressed contents of the file.

            As a test; here is a ~100kB JPEG file. Can your script handle it or does it fall over on the imagecreatefromjpeg() line?

              Thanks, again.
              no...couldn't do it. I understand on a surface level, but always thought the the pixel/inch ratio had a corresponding filesize (unless specifically compressed).😕
              ...
              so, can/should I leave MAX_FILE_SIZE as is, but actually do need to allocate more memory via php.ini to allow for reading pixel data?

              What are your recommendations, if you don't mind?

                veeps wrote:

                no...couldn't do it.

                That's because its 4096x4096 pixels, making for 48MB of raw data (I forgot to look at the printer resolution I used - I think it was 72dpi - but that just affects how big the pixels are on paper, not how many of them there are).

                veeps wrote:

                (unless specifically compressed)

                That's it right there: JPEG is a compressed format. Most image formats are, except when processing speed is the priority.

                PHP's default setting for memory_limit is "128M". While it's possible to set this to "0" to mean "no limit", you do run the risk a runaway script gobbling all the memory in sight and bringing the whole server to its knees.

                If you want to completely avoid PHP allocating the amount of memory required to manipulate an image (and thus risk having it exceed its limitation) then you could have PHP pass the manipulation job off to a separate program and pick up the results. This by definition will require a separate program; the most common would be Imagemagick (search these forums for a number of threads discussing it), and if you go with this approach you'd probably prefer to use a pre-made wrapper class to hide all the extra complication.

                  You could also check the raw data size before trying to process to throw an error:

                  function checkImageRes($w,$h) {
                     $max = return_bytes(ini_get('memory_limit'));
                     return ($w*$h*3) < $max : TRUE ? FALSE;
                  }
                  
                  list($width,$height) = getimagesize('uploades/imagejpg');
                  if( checkImageRes($width,$height) ) {
                     // Process image
                  } else {
                     echo 'Image too large for the GD Library';
                  }

                    Thanks for the help, guys. May you be rained upon with women and snax. Derekorian, thanks, I was hoping for something like that. Will try now.

                      Hey veeps: I didn't realize when I wrote that that return_bytes is defined in the script I was looking at, so if you need that function here it is.

                      function return_bytes($val) {
                          $val = trim($val);
                          $last = strtolower($val[strlen($val)-1]);
                          switch($last) {
                              // The 'G' modifier is available since PHP 5.1.0
                              case 'g':
                                  $val *= 1024;
                              case 'm':
                                  $val *= 1024;
                              case 'k':
                                  $val *= 1024;
                          }
                      
                      return $val;
                      }

                        Heh, I just noticed that the class I linked to makes a big point of not being affected by PHP's maximum memory limit - it's the first featured bullet point 🙂

                          Derekorian,
                          Your insight helped! Thanks.

                          For anybody else trying to solve this problem, here's the breakdown of what worked for me:

                          //THIS IS WRONG...I think
                          function checkImageRes($w,$h) {
                             $max = return_bytes(ini_get('memory_limit'));
                             return ($w*$h*3) < $max : TRUE ? FALSE;
                          } 
                          

                          should actually be

                          //This, i think, is right
                          function CHECK_IMAGE_RES($w,$h) {
                          		   $max = RETURN_BYTES(ini_get('memory_limit'));
                          		   return ($w*$h*3) < $max ? TRUE : FALSE;
                          		}

                          The two above functions work hand-in-hand, needing only my input value for 'uploades/imagejpg', like this (I put the following funcs in an include, to avoid 'disturbing the Matrix'):

                          /*THIS SCRIPT CONTAINS TWO FUNCTIONS - ONE TO NEST INSIDE THE NEXT */
                          
                          //Return the number of bytes specified by ini_get() in second function
                          		function RETURN_BYTES($val) {
                          			$val = trim($val);
                          			$last = strtolower($val[strlen($val)-1]);
                          			switch($last) {
                          				// The 'G' modifier is available since PHP 5.1.0
                          				case 'g':
                          					$val *= 1024;
                          				case 'm':
                          					$val *= 1024;
                          				case 'k':
                          					$val *= 1024;
                          			}
                          
                          		return $val;
                          	} 
                          
                          //Check image res, and return true or false		
                          		function CHECK_IMAGE_RES($w,$h) {
                          		   $max = RETURN_BYTES(ini_get('memory_limit'));
                          		   return ($w*$h*3) < $max ? TRUE : FALSE;
                          		}

                          Thereafter, I used the function in to determine if $sizeOK = true in the original file above:

                          //get original from php's temp directory with [$number] referring to image/upload[#]
                          $original = $_FILES['image']['tmp_name'][$number];
                          //now test using both CHECK_IMAGE_RES & MAX_FILE_SIZE
                          if($_FILES['image']['size'][$number] > 0 && $_FILES['image']['size'][$number] <= MAX_FILE_SIZE && CHECK_IMAGE_RES($width,$height)) {
                          					$sizeOK = true;
                          				}
                          

                          It may not be poetry, but it sure feels like it sometimes!

                            The code above worked for rejecting certain large files and accepting some that were within the limitations. I don't understand what happened, because it successfully rejected 3.7M jpeg files that were actually 40.5M (15"x10" @ 300dpi) for example, but exhausted the memory while trying to upload a 1.9M file that was actually 25.6M (19"x14" @ 180dpi). Small images uploaded fine.

                            I wish I could find the flaw in my code's size check, but in the meantime, I boosted my php_ini memory_limit from 32M to 128M. It works now, but I don't feel that's the real solution.

                            For anybody who has the time/willingness, here's the resulting code:

                            //-----------------INCLUDE-----------------
                            //Return the number of bytes
                            		function RETURN_BYTES($val) {
                            			$val = trim($val);
                            			$last = strtolower($val[strlen($val)-1]);
                            			switch($last) {
                            				// The 'G' modifier is available since PHP 5.1.0
                            				case 'g':
                            					$val *= 1024;
                            				case 'm':
                            					$val *= 1024;
                            				case 'k':
                            					$val *= 1024;
                            			}
                            
                            		return $val;
                            	} 
                            
                            //Compare bytes to php_ini	
                            		function CHECK_IMAGE_RES($w,$h) {
                            		   $max_php = RETURN_BYTES(ini_get('memory_limit'));
                            		   return ($w*$h*3) < $max_php ? TRUE : FALSE;
                            		}
                            //----------------------END INCLUDE FILE ---------
                            
                            /*---------         IN-PAGE ...        -----------------*/
                            //STEP 2 UPLOAD
                            //max size 3.75 mb
                            define('MAX_FILE_SIZE',3932160);
                            if(array_key_exists('upload',$_POST) && isset($_SESSION['gal'])) {
                            	//define file sizes for orig and thumbs
                            	define('MAX_FILE_WIDTH',360);
                            	define('MAX_FILE_HEIGHT',360);
                            	define('MAX_THUMB_WIDTH',108);
                            	define('MAX_THUMB_HEIGHT',108);
                            	//define upload dir
                            	define('UPLOAD_DIR','..path/images/');
                            	//define thumbs_dir
                            	define('THUMBS_DIR',UPLOAD_DIR.'Th/');
                            	$gal = $_SESSION['gal'];
                            	//permitted MIME types
                            	$permitted = array('image/jpeg','image/pjpeg');
                            	//convert the max size (bytes) to MB
                            	$max = number_format(MAX_FILE_SIZE/1048576,1).'MB';
                            
                            
                            foreach($_FILES['image']['name'] as $number => $file) {
                            	//get path/val of file in tmp_name
                            	$original = $_FILES['image']['tmp_name'][$number];
                            	//replace spaces & assign var to $file
                            	$file = str_replace(' ','_',$file);
                            
                            	//assume file is unacceptable
                            	$name_lengthOK = false;
                            	$sizeOK = false;
                            	$typeOK =false;
                            	$unique = false;
                            
                            	//check name length
                            	$file_length = strlen($file);
                            	if($file_length <= 25) {
                            		$name_lengthOK = true;
                            	}
                            
                            	//check filesize
                            	if(!empty($original)) {
                            		list($width,$height) = getimagesize($original);
                            		if($_FILES['image']['size'][$number] > 0 && $_FILES['image']['size'][$number] <= MAX_FILE_SIZE && CHECK_IMAGE_RES($width,$height)) {
                            		   $sizeOK = true;
                            		}
                            	}
                            	//check MIME type
                            	foreach($permitted as $type) {
                            		if($type == $_FILES['image']['type'][$number]) {
                            			$typeOK = true;
                            			break;
                            		}
                            	}//endforeach
                            
                            	//make sure image is not a duplicate
                            	if(!file_exists(UPLOAD_DIR.$file)) {
                            		$unique = true;
                            	}
                            
                            	if($name_lengthOK && $sizeOK && $typeOK && $unique) {
                            		switch($_FILES['image']['error'][$number]) {
                            			case 0:		
                            		//move the file to the upload folder and rename it
                            		  //get new sizes from orig
                            
                            		list($width, $height, $type) = getimagesize($original);
                            
                            		//calculate ratios
                            		if ($width <= MAX_FILE_WIDTH && $height <= MAX_FILE_HEIGHT) {
                            			$file_ratio = 1;
                            			if($width > $height) {
                            				$thumb_ratio = MAX_THUMB_WIDTH/$width;
                            			}
                            			else {
                            				$thumb_ratio = MAX_THUMB_HEIGHT/$height;
                            			}
                            		}
                            		elseif ($width > $height) {
                            			$file_ratio = MAX_FILE_WIDTH/$width;
                            			$thumb_ratio = MAX_THUMB_WIDTH/$width;
                            		}
                            		else {
                            			$file_ratio = MAX_FILE_HEIGHT/$height;
                            			$thumb_ratio = MAX_THUMB_HEIGHT/$height;
                            		}
                            		//strip the extension off the image filename
                            		$imagetypes = array('/\.jpg$/','/\.jpeg$/','/\.JPG/');
                            		$name = preg_replace($imagetypes,'',basename($file));
                            /*** The next line receives the 'Allowed Memory...' error message during upload attempt ***/
                            			$source = imagecreatefromjpeg($original);
                            			//calculate new dimensions
                            			$new_width = round($width * $file_ratio);
                            			$new_height = round($height * $file_ratio);
                            			$thumb_width = round($width * $thumb_ratio);
                            			$thumb_height = round($height * $thumb_ratio);
                            			//create image resources
                            			$new_file = imagecreatetruecolor($new_width, $new_height);
                            			$thumbnail = imagecreatetruecolor($thumb_width, $thumb_height);
                            			//create resized copies
                            			imagecopyresampled($new_file, $source, 0,0,0,0, $new_width, $new_height, $width, $height);
                            			imagecopyresampled($thumbnail, $source, 0,0,0,0, $thumb_width, $thumb_height, $width, $height);
                            			$sql_filename = mysql_real_escape_string($name.'.jpg');
                            			$sql = "INSERT INTO images (image_name,gallery) VALUES ('$sql_filename','$gal')";
                            			$sql_result = mysql_query($sql) or die(mysql_error());
                            			//save resized copy			
                            			$success = imagejpeg($new_file, UPLOAD_DIR.$name.'.jpg',100);
                            			$success .= imagejpeg($thumbnail, THUMBS_DIR.$name.'.jpg',100);
                            			imagedestroy($source);
                            			imagedestroy($new_file);
                            			imagedestroy($thumbnail);
                            			if($success) {
                            				$upload_result[] = '<span id="blue_text">"'."<em>$file</em>".'" '."UPLOADED SUCCESSFULLY</span>";
                            			}
                            			else {
                            				$upload_result[] = "Error uploading $file. Please try again.";
                            			}
                            			break;
                            			case 3:
                            			$upload_result[] = "An error occurred while uploading $file. PLEASE TRY AGAIN.";
                            			default:
                            			$upload_result[] = "SYSTEM ERROR uploading $file. Contact webmaster.";
                            			}//end switch
                            		}//endif($name_length...sizeOK)
                            		elseif(!$unique && $_FILES['image']['error'][$number] !== 4) {
                            			$upload_result[] = "ERROR: A file named $file already exists.";
                            		}
                            		elseif(!$name_lengthOK) {
                            			$upload_result[] = 'ERROR: "'."<em>$file</em>".'" '."must be renamed to be 25 characters or less.";
                            		}
                            		elseif ($_FILES['image']['error'][$number] == 4) {
                            			$upload_result[] = 'No file selected';
                            		}
                            		else {
                            			$upload_result[] = '"'."<em>$file</em>".'" '."COULD NOT BE UPLOADED:<br/>
                            			&nbsp;-&nbsp;<em>Max size: $max<br/>
                            			&nbsp;-&nbsp;jpeg only</em>";
                            		}
                            	}//endforeach
                            }//end if $_POST
                            <p><strong>Step 2:</strong><br/>
                                </p>
                                  <label for="image1">1. Choose Image:</label>
                                   <input type="hidden" name="MAX_FILE_SIZE" value = "<?php echo MAX_FILE_SIZE; ?>*" />
                                  <input type="file" name="image[]" id="image1" />
                                  <br/>
                                  <label for="image2">2. Choose Image:</label>
                                  <input type="file" name="image[]" id="image2" />
                                  <br/>
                                  <label for="image3">3. Choose Image:</label>
                                  <input type="file" name="image[]" id="image3" />
                                </p>
                                <p>
                                  <input type="submit" name="upload" id="upload" value="Upload Image(s)" />
                                </p>
                            
                            
                            </div>

                              A GD image resource is larger than pixels * color_depth. It's in fact close to 1.7 times larger than this. You can easily see this for yourself by creating larger and larger sizes of new images and compare their actual size to their expected size.

                              For very small images, this factor is off by quite a bit, which seems to indicate some initial GD resource overhead, which also fits well with the fact that the "addional memory factor" is decreasing as image sizes increase. However, this change in the factor is rather small once you get beyond a decent image size. Besides you don't really need to worry about images that either fit well in memory or are far off from doing so.

                              Either way, it is easy to see how big this factor is for images of varying sizes, since all you have to do is create such images and compare actual memory usage to expected memory usage.

                              ini_set('memory_limit', '650M');
                              function format_size($size) {
                                if ($size < 1024) {
                                  return $size . ' B';
                                }
                                else {
                                  $size = round($size / 1024, 2);
                                  $suffix = ' KB';
                                  if ($size >= 1024) {
                                    $size = round($size / 1024, 2);
                                    $suffix = ' MB';
                                  }
                                  return $size . $suffix;
                                }
                              }
                              
                              $clean_mem = memory_get_usage();
                              $imres_mem = memory_get_usage();
                              
                              for ($i = 2500; $i < 2700; $i += 50)
                              {
                              	$clean_mem = memory_get_usage();
                              	$im = imagecreatetruecolor($i, $i);
                              	$imres_mem = memory_get_usage();
                              
                              echo 'Image Size: ' . format_size($imres_mem - $clean_mem) . "\n";
                              $ff = ($imres_mem - $clean_mem) /
                              	 ($i * $i * 3);
                              echo '(Image Size - [assumed resource overhead]) / (Width * Height * Bytes per pixel)'. PHP_EOL .
                              format_size($imres_mem - $clean_mem) . ' / ( ' . $i . ' * ' . $i . ' * 3 ) = ' . $ff . "\n";
                              imagedestroy($im);
                              
                              echo "\n";
                              }
                              

                              Thus, to get expected memory usage

                              pixels * 3 * 1.67 = pixels * 5.01
                              

                              Also note that it's not enough to have the memory needed for imagecreatefromjpeg when you create the image resource from the uploaded file. You will obviously also need enough memory for your calls to imagecreatetruecolor a few lines below that.

                                Write a Reply...