Mass Emailer Questions

Sharif

In my database of members, I have .. let's just say tons of entries.

I am trying to create a script where I can email all of them at once, without crashing my server, because I know there is a limit on how many emails can actually be sent within a certain period of time.

I coded the script to get the email address from the database and mail each person uniquely... but how can I limit to how many users it sends to every minute?

bpat1434

Try using the execution time....

<?php
$mtime = microtime();
   $mtime = explode(" ",$mtime);
   $mtime = $mtime[1] + $mtime[0];
   $starttime = $mtime;

// Your emailing stuff here

$mtime = microtime();
   $mtime = explode(" ",$mtime);
   $mtime = $mtime[1] + $mtime[0];
   $endtime = $mtime;
   $totaltime = ($endtime - $starttime);
// Now loop to the top, and continually add
?>

Then, just pause if it hasn't been 60 seconds...

NOTE: The code above is not meant to work properly, just give you a guide as to how to do it.

~Brett

Sharif

Ok trying to understand this. Here is a basic structure of my code currently:

//sql to get emails
// $sql = "SELECT..."
// $query = mysql_query($sql);

//loop for each member
while($row = mysql_fetch_array($query)
{
  //email the member
  mail($row['email'], $subject, $message);
}

Where in there would I perform the time check and loop?

Weedpacket

You've already got the loop. The time check code goes inside that, and the mailing code goes where bpat has put the comment "your mailing stuff here".

bpat1434

There is a downside to actually doing it this way though.... the [man]mail[/man] function opens and closes an SMTP socket for each message:

PHP Manual: mail wrote:
Note: It is worth noting that the mail() function is not suitable for larger volumes of email in a loop. This function opens and closes an SMTP socket for each email, which is not very efficient.

You might want to look into PEAR as an alternative to producing a mass-mailer type of system.

~Brett

Sharif

Ok in an attempt to simplify this, I came up with this:

//limit of emails to be sent per 30 seconds
$limit = 50;

//sql to get emails
// $sql = "SELECT..."
// $query = mysql_query($sql);

//set count of emails sent so far
$count = 0;

//loop for each member
while($row = mysql_fetch_array($query)
{
  //if limit has been reached
  if($count == $limit)
  {
    //reset count
    $count = 0;
    //pause script for 30 seconds to allow SMTP sockets to refresh
    sleep(30);
  }
  //email the member
  mail($row['email'], $subject, $message);

  //add to count of emails sent
  $count++;
}

How does that look? good? bad?

Weedpacket

Well... between

//limit of emails to be sent per 30 seconds
$limit = 50;

and

    sleep(30);

you're not going to get many emails sent out. You should read bpat1434's last post.

jazz_snob

bpat1434 wrote:
There is a downside to actually doing it this way though.... the [man]mail[/man] function opens and closes an SMTP socket for each message:

You might want to look into PEAR as an alternative to producing a mass-mailer type of system.

~Brett

Correct me if I'm wrong, but mail() by default on *nix systems uses sendmail, which is exim4 on my system. I can't imagine a better way to send a bunch of emails then to ask exim to do it for me. So (scratches head and squints) I don't see where using mail() under those circumstances is bad. The PEAR classes ultimately end up using either sendmail or SMTP ( see Mail::factory ). I don't claim to know what I'm talking about here 😉, so please enlighten me if I've misunderstood something.

justsomeone

Also, if you're going to have a script that "sleeps" a lot, the server is going to kill it off. There are limits to how long any script can stay executing (sleeping counts as executing).

An elegant solution might be to cron a number of smaller jobs and divide the load between them.

For example, if you have a registration timestamp on your usertable, you could cron a job to run every hour and code this job to select all users whose "hour" portion of their timestamp matches the current hour, and email them.

So, at 14:00 it would find all users who registered at a time beginning with "14:" on any date, and email them.

This would divide the load over the whole day.

If each hour's mailshot is still too big, you could break it down further... maybe running every 15 minutes... So, at 14:30 it would find all users who registered at a time beginning with "14:30" on any date, and email them.

Weedpacket

jazz_snob wrote:
Correct me if I'm wrong, but mail() by default on *nix systems uses sendmail, which is exim4 on my system. I can't imagine a better way to send a bunch of emails then to ask exim to do it for me. So (scratches head and squints) I don't see where using mail() under those circumstances is bad. The PEAR classes ultimately end up using either sendmail or SMTP ( see Mail::factory ). I don't claim to know what I'm talking about here 😉, so please enlighten me if I've misunderstood something.

The problem is the way that mail() communicates with sendmail:

the mail function opens and closes an SMTP socket for each message:

So....

Open SMTP socket.
Send one email.
Close SMTP socket.
Open SMTP socket.
Send one email.
Close SMTP socket.
Open SMTP socket.
Send one email.
Close SMTP socket.
...

And the majority of the script's time is spent opening and closing the same socket over and over again.

An SMTP session (between socket open and socket close) can do a lot more than just "Send one email."

Sharif

What if you specified SMTP information and all that in mail()... would it help?

Sharif

Weedpacket wrote:
you're not going to get many emails sent out. You should read bpat1434's last post.

I am trying to understand the loop... I just don't get how it's supposed to work 🙁

bpat1434

function getStartTime()
{
   $mtime = microtime();
   $mtime = explode(" ",$mtime);
   $mtime = $mtime[1] + $mtime[0];
   $starttime = $mtime;
}

$maxmsgs = 25; // Send a max of 25 emails every minute

for($i=1; $i<$num_of_emails_to_send; $i++)
{
   global $starttime, $maxmsgs;

###################################
### PUT YOUR MAILING CODE IN BETWEEN HERE ###
###################################

mail($to, $from, $subject, $body, $headers);

#################################
### YOUR MAILING CODE SHOULD BE ABOVE!! ###
#################################

   $mtime = microtime();
   $mtime = explode(" ",$mtime);
   $mtime = $mtime[1] + $mtime[0];
   $endtime = $mtime;
   $totaltime = ($endtime - $starttime);

   if($i%$maxmsgs==0)
   {
      $sleeptime = 60-$totaltime;
      // wait for a few seconds
      sleep($sleeptime);
      getStartTime();
      continue;
   }
   elseif(($totaltime>=60) && ($i%$maxmsgs!=0))
   {
      // We've spent over 1 minute trying to send 25 emails, let's take a break
      sleep(15);
      getStartTime();
      continue;
   }
}

Something like that might work.... not going to say it's 100%.

~Brett

EDIT
1,500th post... woohoo!!!

hairulazami

in this line

$mtime = microtime();
   $mtime = explode(" ",$mtime);
   $mtime = $mtime[1] + $mtime[0];
   $endtime = $mtime;
   $totaltime = ($endtime - $starttime);

where is the $starttime value ? 🙂

is the value got from function getStartTime() above ?

laserlight

is the value got from function getStartTime() above ?

That is intended, though in the sample code it does not actually happen since getStartTime() was not called at that point. Admittedly this is rather poor code that could be improved (in readability, at least) by simply returning the result instead of relying on $starttime being global... but then this is sample code intended for Sharif to modify for actual use.

hairulazami

I am sorry if my question is too bad...

But I just mean that, want to know and learn .. so I ask about this value..
I think it's not poor code...it's cool code body

Thanks