I have a script that sends and receives XML. It works great for results returned < 200, but unexpectedly quits somewhere after 200 results. 550 results is about 2.5MB so I can's seem to figure out why it's failing. My web host does not have the ability to use the memory_get_usage() function so it is terribly difficult to debug. I get the following error in my error logs:

FATAL: emalloc(): Unable to allocate 20279 bytes

I understand that this is a memory problem, but I can't just increase the memory on a shared server (and I assume there must be a way to use less memory).

My script is quite long, but I will paste some of it below.

Three things which might help: 1.) A way to see how much memory the script is using as it is processing (that doesn't use memory_get_usage), 2.) A simple fix 🙂 3.) A miracle 😃

Send & receive the XML

$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $query_url);
curl_setopt ($ch, CURLOPT_HEADER, 0);
curl_setopt ($ch, CURLOPT_HTTPHEADER, Array("Content-Type: text/xml"));
curl_setopt ($ch, CURLOPT_POST, 1);
curl_setopt ($ch, CURLOPT_POSTFIELDS, $query);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);

$results = curl_exec ($ch);

curl_close ($ch);

Process the XML

$dom = domxml_open_mem($results);
unset($results);
$root = $dom->document_element(); 
$listing = $root->child_nodes();

// loop through returned rows
for ($i = 0; $i < count($listing); $i++)
{
	$elements = current($listing);

if ($elements->node_name() == "listing")
{
	$elements = $elements->child_nodes();
	for ($j = 0; $j < count($elements); $j++)
	{
		$node = current($elements);
		$node_content = $node->get_content();
		$node_name = $node->node_name();
		$data[$node_name] = $node_content;
		next($elements);
	}

	// Process the results and insert into a DB
	// There is much, much more code here, that I can post if needed
}
}

    Try adding this at the end of your for() loop:

    // Begin for Loop
ob_start();
// Do stuff
ob_end_clean();
// End For Loop

    So your code would look like:

    <?php
// loop through returned rows
for ($i = 0; $i < count($listing); $i++)
{
    ob_start();
    $elements = current($listing);

if ($elements->node_name() == "listing")
{
    $elements = $elements->child_nodes();
    for ($j = 0; $j < count($elements); $j++)
    {
        $node = current($elements);
        $node_content = $node->get_content();
        $node_name = $node->node_name();
        $data[$node_name] = $node_content;
        next($elements);
    }

    // Process the results and insert into a DB
    // There is much, much more code here, that I can post if needed
}
ob_end_clean();
}
?>

    I got this from:
    PHP Manual: ob_clean()

    Although it is mentioned in the manual, you have to be careful when using output buffering in big cycles (such as mass mail sending scripts), because ob_clean() actually does not free any memory, and with each iteration the amount of memory allocated from your script will increase significantly. Instead of calling ob_clean() at the end of the cycle, you have to either use ob_get_clean(), which is a combination of ob_get_contents() and ob_end_clean(), or just ob_end_clean() to free the memory.

    ~Brett

      Thanks Brett,

      I tried your suggestion, but this did not help. I read up on the Output Control functions. It seems like they are for handling output to the browser. My script doesn't output anything so I guess that's why this doesn't help. I ran the function ob_get_status() for each loop and the array returned nothing. I have no other ideas right now.

      -Sean

        So far from what I've seen, the only way to really flush the memory is to use ob_end_clean() and flush() respectively. Try using both at the end of the for() loop.

        <?php
for(. . .)
{
    ob_start();
    // Do something
    ob_end_clean();
    flush();
}
?>

        ~Brett

          OK, I see what is going on. I accidentally came across a post for almost the same problem. It looks like the problem was the function domxml_open_mem(). I was using unset to clear this resource, but I needed to use free().

          I had this code:
          $dom = domxml_open_mem($results);
          unset($dom );

          But I needed this code:
          $dom = domxml_open_mem($results);
          $dom->free();

          I guess domxml_open_mem() uses a lot of memory. Thanks for all of the help.

          -Sean

            Write a Reply...