Emulating Multithreading to send mail to different servers at the same time

tanis

Hello,

I'm trying to find a solution in order to be able to use more than one SMTP server at once with my newsletter application (written in PHP of course).

The scenario is that we have 3 SMTP servers that we're using to send emails from.
All of the emails are personalized (with name, surname, etc..).

There's a command line script that cycles through the newsletters that have to be sent, looks up the individual email addresses and other data and then starts sending them out.

At the moment it's just cycling through the three servers and using the phpmailer class to send the mail. What happens is that with every send, the script has to wait for the communication with the SMTP server to be done. This means that the other 2 servers in the meanwhile are just sitting there and waiting.

I would like to use all three at the same time and sort of check which one is free and use it. That way, if one of the servers is already overwhelmed and is answering slowly, I can be sure that I am still using the other 2 efficiently.

I originally thought about using multithreading which is the most natural approach to this kind of situation.. but since PHP does not support it natively I'm in a bit of stall as to what to try out.

Any suggestions?

etully

I suppose you could simulate multithreading by running three scripts. The scripts are basically copies of eachother except that each one points to a different SMTP server. The logic of the script looks like this:

$smtp_servername = "smtp1.yourdomain.com";
Open Database
Start Loop
lock table
find next email to send out
mark that record as "processing"
unlock table
send email
mark that record as "sent"
continue loop

This way, all three scripts are reading from the same database, you don't have to worry about two scripts sending the same newsletter, and it's ok if one smtp server is running 100 times faster than the other two servers. In fact, you could add new smtp servers (scripts) easily without even having to alert or alter any of the other scripts.

The table in the database might contain:
email address
name
surname
which newsletter they are supposed to receive
status (ready / processing / sent)
You might even have a field to record which smtp server processed this record.

Then you run all three (or more) scripts on the command line.

tanis

Thanks for the suggestion. Seems like a viable solution but I forgot to tell you about one more thing that might make this solution unpractical in my environment.

We're using clustered mysql servers that where there's only one master where you can write and all the others are slaves where you can read but that you can't be sure are up to date with data when you read from them. This is due to the way clustering has been done (not by me, but I have to live with that).

That way, if I let a process mark a record as "processing", I can't be sure that another script will actually see that. Chances are high that it will still be in "ready" and I will send it out three times. I had similar problems before with different scripts.

That's why I am looking for a solution that involves a "controller" to manage the single emails to send out and then giving a different task the job of actually sending it out without blocking.

MarkR

What you should do is install a local MTA on your web server, and configure that to round-robin the messages via your three relays.

Have PHP deliver to the local relay (e.g. with mail() )

This also guards against the highly likely situation that PHP is trying to send via a down or unreachable mail relay, which will result in severe delay and/or mail loss with the setup described above.

Mark