Serious memory management problem

[Melville]

I have a PHP page that reads emails from an email account, parses them, processes the resulting information, and then sends out a response. I am in the process of testing it right now, and I am using a small sample (11 emails). I expect, when the site is fully functional, that I could be processing hundreds of emails at a time. I am running the site on a managed server that is dedicated, so I have a fair amount of memory.

Unfortunately, I get a fatal error:

Fatal error: Allowed memory size of 41943040 bytes exhausted (tried to allocate 6 bytes) in funcs_parse_emails.php on line 48

after only six emails. Clearly, if I am using up 42 megabytes, I have a serious problem.

Is there someone out there who can point me to a book or website that discusses PHP memory management issues or provides tools I can use? Alternatively, is there someone who has expertise in this area who would be willing to help me?

Thank you,

Mel

[veridicus]

I have some basic suggesions on saving memory.

First, if possible, don't read all of the email contents before looping. In your loop get each email, do what you need to do, and then clear the variables or re-use them on the next loop. Even better would be to only read in the parts of the email you require.

Second, consider how you're parsing. Breaking big text into arrays can be memory intensive. If you're just trying to pick out some text attempt to do it without regular expressions.

What's your method for retrieving the emails? That may best explain what's going on.

[Melville]

Thank you for the reply, veridicus. I will look at the ideas you mentioned. What I cannot understand, though, is how I could have gone through 42 megs of memory in six emails. The largest email is only 10K bytes. So I would have to make almost a thousand copies of it to eat up 42 megs with six emails.

I really do need to read (and store in a mySQL database) the entire message, so I cannot save there. You asked how I am retrieving the emails. Here is the beginning of my script:

$cfg['forward_headers'] = array(
'from',
'Date',
'Subject',
'message_id'
);
// array to hold message info
$mail = array();

// open mailbox
//$mail['stream'] = imap_open($cfg['mailbox'],$cfg['account'],$cfg['password']);

$mail['stream'] = imap_open('{' . $cfg['host'].':' . $cfg['port_protocol'].'}' .
$cfg['mailbox'],$cfg['account'],$cfg['password']);
//$mail['stream'] = imap_open ("{localhost:110/pop3}INBOX", "m37634967-25", "chem1395");
//$mail['stream'] = imap_open ("{localhost:110/pop3}INBOX", "user_id", "password");
if(!$mail['stream']) {
$subject = "BU: process_email.php died";
$mail_body = <<<EOMAILBODY
mail['stream'] does not exist
EOMAILBODY;
$extra_header_stuff = "From: [e-mail]\r\n";
$mailsend = mail("[e-mail]", $subject, $mail_body, $extra_header_stuff);
die("Could not open mailbox : ".imap_last_error());
}
// get number of messages in box
$mail['message_count'] = imap_num_msg($mail['stream']);
echo $mail['message_count'] . " messages as follows:<BR>";
// loop all messages

for($i=1; $i<=$mail['message_count']; $i++) { // process each email
echo "message " . $i . "<BR>";
// initialize for this record
$account_id = 0;
$prop_id = 0;
$contents_array = "";
$owner_email = "";
$start_date = "";
$start_year = "";
$end_date = "";
$num_nights_for_stay = 0;
$num_people = 0;
$source_site = "";
$source_site_id = 0;
$email_array_body = "";
$amt_due_at_booking = 0;
$balance_due_date = "";
$total_supplemental_person_charge = 0;
$total_supplemental_person_charge_formatted = "";
$grand_total = 0;

$multiple_dates_flag = FALSE;
$mail['cur_body'] = imap_body(@$mail['stream'],$i);
$mail['cur_headers'] = imap_headerinfo(@$mail['stream'],$i);
$mail['cur_headers']->from_formatted = @$mail['cur_headers']->from[0]->mailbox . '@' . @$mail['cur_headers']->from[0]->host;

// get the various parts of the message to work on...
$email_array = array();
$email_array['date'] = @$mail['cur_headers']->date;
$email_array['from'] = @$mail['cur_headers']->from_formatted;
$email_array['subject'] = @$mail['cur_headers']->subject;
$email_array['body'] = @$mail['cur_body'];
$lc_subject = strtolower(@$email_array['subject']);
$pet_deposit_formatted = "";
$supplemental_person_charge_text = "";

$prop_id = 0;
$contents_array = array();

The processing can get pretty involved, with several accesses of the database for different reasons. The .php file is more than 1500 lines long.

Any other thoughts?

thanks again,
mel

[Melville]

Thank you to those who were willing to help me here. It is terrific to have a support community!
I found the PHP function memory_get_usage, and by putting in a bunch of echo statements, I was able to determine where my problem was. (It turned out to be an "=" instead of an "==" in a conditional, which resulted in an endless loop. It's always something trivial, isn't it?)

For those of you facing memory usage problems, this function (memory_get_usage)is great.

Mel