Problem with sending mutliple BUT personalised emails...

bodzan

k... I'm anything but a newbie in PHP, but these last few weeks were hell for me and I'm starting to feel again like one and not in a good way...

My problem as the title states is sending multiple emails that have to be personalised. Now, being dumb when it somes to creating email and having 0 (zero) knowledge of the email servers I started using phpmailer class to achieve what I need in this mailing list I created out of customers of my clent's site...

This was all peachy and great but on very first use when some 350 emails should have been sent by my clients - disaster! Some of their clients received 100s of emails and finally sendmail server went dead! Of course - technical support pointed finger at me as a creator of the script that creates these emails and I was in a bit of a mess here...

Sorry, for the long intro and now comes the problem:
My clients demand that every user on their list receives personalised email stating "Dear Name Lastname, etc" on the top and having the unsubscribe link on the bottom of the HTML email they want to send.

So, after trying to figure out this thing for myself I have to say - enough! I'm simply not clever to figure this one out... 🙁

Is there a way to send this email all at once not to disturb my mail server nor exceeding darn 30 sec's script timeout? I figure I might be forced to use cronjob for the job, but again I'll have the same issue with the email server unless I figure out best way of doing this...

bpat1434

I feel your pain.....

SMF has an intriguing way of doing a mass-mailer (that's semi-customized)....

They take the input information (in your case, names) and send out batches of emails (like 20 - 40) until all the users have been emailed. And it automatically re-submits the form (or reads from the head) where you are in the list, what list to send to, and what email to send.

It's crude, but it works. It also destroys the 30-sec timeout because it auto-refreshes the browser (and gives you an update as to how far along it is (10%, 50%, etc.)). I'm guessing it's laid out something like:

<?php
session_start();
// If the session has nothing, give defaults (or posted) values:
$body = !isset($_SESSION['body'])?$_POST['body']:$_SESSION['body'];
$group = !isset($_SESSION['group'])?$_POST['group']:$_SESSION['group'];
$start = !isset($_SESSION['start'])?0:$_SESSION['start'];
$current = !isset($_SESSION['current'])?0:$_SESSION['current'];
$max = 15;

// Write new values to the session (overwrites old data)
$_SESSION['body'] = $body;
$_SESSION['group'] = $group;
$_SESSION['start'] = $start;
$_SESSION['max'] = $max;

// Quick query to get total emails to send ;)
$q = "SELECT COUNT(id) AS total FROM users WHERE groupID=(SELECT groupID FROM groups WHERE groupName LIKE '%" . $group . "%' LIMIT 1)";
$res = mysql_query($result);
$total = mysql_fetch_row($result, 0, 'total');
$_SESSION['total'] = $total;

// Get the list to send to:
$query = "SELECT id, email, firstname, lastname FROM users WHERE groupId=(SELECT groupID FROM groups WHERE groupName LIKE '%" . $group . "%' LIMIT 1) LIMIT $start, $max";
$result = mysql_query($query);
$error = array();
while($row = mysql_fetch_assoc($result))
{
    $txt = str_replace('{FIRST}', $row['firstname'], str_replace('{LAST}', $row['lastname'], $body));

$headers = "From: A List-mailer :) <no-reply@domain.com>\n";
$headers = "To: A Mail List :) <list@domain.com>\n\n";

if(mail($row['email'], 'A List Message!!', $txt, $headers) == FALSE)
{
    $error[] = $row['firstname'] . ' ' . $row['lastname'] . ' (' . $row['email'] .  ')';
}
}

if(isset($_SESSION['error']))
    array_push($_SESSION['error'], $error);
else
    $_SESSION['error'] = $error;

// Update session
# We want to start at the next place (not the last email sent)
$_SESSION['start'] = $row['id']++;
# Update how many we've sent
$_SESSION['current'] = $_SESSION['current']+$_SESSION['max'];

// Generate a percent
$percent = round(($_SESSION['current']/$_SESSION['total'])*100);

echo '
<html>
<head>
    <title>Sending Emails . . . [' . $percent . '%]</title>
    <meta http-equiv="REFRESH" content="5;http://domain.com/' . $_SERVER['PHP_SELF'] . '" />
</head>
<body>
  <h3>Sending your emails . . .</h3>
  <p>Currently sending your email(s) to ' . $_SESSION['total'] . ' recipients in batches of ' . $_SESSION['max'] . ' messages.<br>
    Currently, the process is ' . $percent . '% complete.  If you do not wish to wait, you can click the link below to proceed:</p>
  <p><a href="http://domain.com/' . $_SERVER['PHP_SELF'] . '">Click here if you do not wish to wait</a></p>';
if($_SESSION['total'] == $_SESSION['current'])
{
    if(!empty($_SESSION['error']))
    {
        echo '
  <p>There were errors during the sending of emails.  Unfortunately, not all recipients were reachable.  The following is a list of emails that were not delivered:</p>
  <p style="max-height: 25em;  overflow: auto;">';
        foreach($_SESSION['error'] as $error)
            echo '
    ' . $error . '<br>'; // The carriage return is for code formatting purposes only...
    echo '
  </p>';
    }
    else
        echo '
  <p>No errors were found during the process.</p>';
}
echo '
</body>
</html>';

Like I said, very crude (and not the best) but it should get you started in a direction....

bodzan

I'll try it first thing when I wake up since it is 4:25 Am here and I'm beat...

Will let you know how it went...

bodzan

Well, as far as I can see the script works like it should but there is one server admin who thinks it is inefficient and it will cause server errors because it doesn't wait for server response after every email that is being sent...

I figured that I don't need to wait for those responses and that mail server (qmail) will be able to send 20 emails with 5 sec delay but it sent to some emails more than one so I figured maybe this server admin is correct...

I tried to create a test script that reads txt file that has some 100 emails listed to see whether it would work like I want it and some of the user got more emails than intended (well, I was most of those and my client the rest)...

I cannot get straight answer fom server administrator (not the guy mentioned previously) of whether my code fails and why or is it mail server that fails...

Anyone has any thought on the subject?

ysu

Write a proper mail sender script instead.
It's no big deal, you can use whichever mail class you want, (htmlMimeMail is my favourite) customize it to be able to send personalized mails and you're good. This one can wait for the sendmail status if you're using it in sendmail mode too.

Unless you're on some god-forsaken windows server, you can refresh the 30 sec timeout at every mail send attempt, that's not an issue.

Don't forget to log what's happening into a file.

If you want to separate from the browser and connection difficulties, you can set the ignore_user_abort as well.

bodzan

What do you men by writting a proper mail sender script?

Please, be so kind and define "proper mail sender script"...

I am using PHPMailer to send emails, but I am to try to use SwiftMailer instead and see if it makes any difference...

Anyways what am I supposed to do when I cannot pinpoint on what went wrong in the first place?

ysu

Sorry. I was referring to the 'auto-resubmitting' method of sending, not your original - I haven't seen that so I can't comment on that obviously.

Write a script which creates the mail in a cycle.
For every user, customize certain variables, like replacing [FIRSTNAME] in the text with the customers' firstname and so on...

Then set the time limit in every cycle to 30 sec. That way if the program hangs for any reason it'll stop, otherwise it'll happily send out as many mails as you like.

You can use smtp sending method or mail method, both will wait for the mail to be injected to the SMTP server, the don't flood the server itself, altho they go into a queue so they can clog the system up a little bit. But I've sent 6000+ customized mails this way without problems.

The only drawback is that this won't work on certain IIS servers (eg the 30 sec time limit), they have a hard-limit for script running times - this f@cks up your script properly 🙂

Altho in an IIS server the sendmail method wont' work either 🙂

HtmlMimeMail has direct smtp method, but I guess the other mail objects have it too.

Also: you never know what happens, so log everything into a file.

bodzan

Well, here's what I got so far:

Now I use SwiftMailer - which uses the SMTP to send emails - so I guess SMTP sending is solved with my script...

Well, at the moment I have the total of 400 emails in a group so I guess it won't clog the system...

The IIS is not an issue sine I have *nix system here...

Now, I know that helping me with my problem without actually seeing the code is to say it mildly hard so here goes:

$mail = new PHPMailer();
$mail->From = "$_SESSION[reply]@somesite.com";
$mail->FromName = "Site's newsletter";

if(isset($_SESSION['tmID']) && is_array($_SESSION['tmID'])){
			while(list($k,$v)=each($_SESSION['tmID'])){
				while(list($key,$val)=each($v)){
					$sql = my_query("SELECT * FROM $MEMB WHERE id = '$key' AND signed='1'");	
					if(mysql_num_rows($sql)>0){
						$r = mysql_fetch_assoc($sql);
						$mail->To("Site newsletter <maillist@somesite.com>");
						$mail->AddBcc($r['email']);
						$member_name = $r['firstName']." ".$r['lastName'];
						$mail->WordWrap = 80; 
						$mail->Subject = $subj;
						$message = str_replace('{ID}',$r['id'],$message);
						$mail->Body = str_replace('{MEMBER_NAME}', $member_name, $message);
						$mail->isHTML(true);
						$message2 = stripslashes($message2);
						$message2 = str_replace('{ID}',$r['id'],$message2);
						$mail->AltBody = str_replace('{MEMBER_NAME}', $member_name, $message2); 
						$mail->Send();
						$mail->ClearAddresses();
					}
				}
			}
		}

Now one important detail:
Each $SESSION['tmid'] looks something like: [$lim][$r['id']] where $lim is a number dividable by 50 and $r['id'] is a member's id - so basically what I get when I cycle through it is something like:
$SESSION['tmid'][50][75] = 1; //this means that user with id 75 which is listed in the group of users from the database that are located between 50 and 100 is selected for sending - somewhat complicated, but I had to do it but that's not the issue here

Does this code make any sense to you? Now, I'm developing a new one since this type crashed the server and I'm calling SMTP for every email and I hope it will work O.K. although haven't tested it so far. When I finish it I'll post that one as well...

Thanx for taking an interest...

ysu

While it makes sense, there are prolems (obviously) 🙂

Sending in BCC is not the best idea. IIRC you can enter a maximum of 255 addresses in BCC. Plus the recipient will wonder why did he/she receive that mail, when the addresse is someone else.

You're adding to the BCC in every cycle, now if the clearAddresses() does not clear it properly the result is exactly what you described.

I recommmend putting the address into the 'to' field, that's the proper place for it.

Set the timeout as I said earlier.

The $message and the $message2 vars are not initialized. You're trying to change them every round like this:
$message = str_replace('{ID}',$r['id'],$message);
it wont' work, once the ID is replaced it'll stay replaced. either put the result directly into the body or use an interim variable.
(hint: the str_replace can work with arrays)

For testing try not to send the mails, instead dump out all fields to see if they are correct, save yourself the headache of sending 100's of wrong messages.

And try to initialize the mail object at every cycle, to be sure. if it does not have an init routine, then $obj=NULL at the end, and $obj=new Object at the start. For a few hundred emails it won't matter if you're recreating the object every round.