MIME decoding

[mwood_2k2]

Ok, here is my problem ...

I have got a script working on my host whereby I can have email piped through to a script, the only problem I'm having is that the script is really basic when it comes to splitting headers and unfortunately doesn't handle MIME emails yet, could you assist me? here is the full php file contents .....

#!/usr/bin/php -q
<?php
$fd = fopen("php://stdin", "r");
$email = "";
while (!feof($fd)) {
    $email .= fread($fd, 1024);
}
fclose($fd);

$lines = explode("\n", $email);
$from = "";
$subject = "";
$headers = "";
$message = "";
$splittingheaders = true;

for ($i=0; $i<count($lines); $i++) {
    if ($splittingheaders) {
        $headers .= $lines[$i]."\n";
        if (preg_match("/^Subject: (.*)/", $lines[$i], $matches)) {
            $subject = $matches[1];
        }
        if (preg_match("/^From: (.*)/", $lines[$i], $matches)) {
            $from = $matches[1];
        }
    } else {
        $message .= $lines[$i]."\n";
    }
    if (trim($lines[$i])=="") {
        $splittingheaders = false;
    }
}

$link = mysql_connect("host", "user", "pass")
        or die("Could not connect: " . mysql_error());

mysql_select_db("database") or die("Could not select database");

$query = mysql_query("SELECT count(email) from table where email='$from'"); 
$num = mysql_num_rows($query); 

if ($num) 
{ 
$query = mysql_query("SELECT email from table");
$num = mysql_num_rows($query); 
while ($email = mysql_fetch_array($query) )
{ 
$to = $email["email"]; 

$header = "From: [e-mail]\n"; 
$header .= "Reply-To: [e-mail]\n"; 

mail("$to", "$subject", "$message", "$header"); 
}
} 
else
{ 
mail("$from", "Unauthorised", "You are not authorised to post to this list...", "$header"); 
}

?>

I am basically attempting to add in MIME decoding so that messages from such services as yahoo! and msn do not add the various MIME headers etc into messages, I just need text/html to be recieved. Any help would be vastly appreciated!

Thanks in advance!

[stolzyboy]

headers i use to send html messages

$headers  = "MIME-Version: 1.0\r\n";
$headers .= "Content-type: text/html; charset=iso-8859-1\r\n";

$headers .= "From: You <[e-mail]>\r\n";
$headers .= "Cc: [email][e-mail][/email]\r\n";

[mwood_2k2]

ok, so sending in mime will eliminate my problem? I need to decode mime messages dont i? all the random numbers etc....

[mwood_2k2]

Ok guys,

I have been doing further messing and testing, here is my code....

$lines = explode("\n", $email);
$from = "";
$subject = "";
$headers = "";
$message = "";
$splittingheaders = true;

for ($i=0; $i<count($lines); $i++) {
    if ($splittingheaders) {
        $headers .= $lines[$i]."\n";
        if (preg_match("/^Subject: (.*)/", $lines[$i], $matches)) {
            $subject = $matches[1];
        }
        if (preg_match("/^From: (.*)/", $lines[$i], $matches)) {
            $from = $matches[1];
        }
		if (preg_match("/^MIME-Version: (.*)/", $lines[$i], $matches)) {
            $headers = $matches[1];
        }
		if (preg_match("/^Content-type: (.*)/", $lines[$i], $matches)) {
            $headers = $matches[1];
        }
    } else {
        $message .= $lines[$i]."\n";
    }
    if (trim($lines[$i])=="") {
        $splittingheaders = false;
    }
}

And I am still getting what I think are MIME headers in my message, even though it does a match for the mime and content type headers, when I recieve a mail through my pipe from yahoo! I get

"--0-754539781-1051893179=:38354 Content-Type: text/plain; charset=us-ascii asdasd --------------------------------- Do you Yahoo!? The New Yahoo! Search - Faster. Easier. Bingo. --0-754539781-1051893179=:38354 Content-Type: text/html; charset=us-ascii"

and then my message, which then displays the proper html yahoo sig ...

I don't know what to do, I'm stuck! Please help!

Thanks in advance

[mwood_2k2]

Please someone!

This is doing my head in, I've been messing with it for like 2 days lol ... Is there no-one with the knowledge to help me out? I don't know a lot about MIME stuff ....

Cheers in advance!

[wrestler145]

http://www.phpbuilder.com/board/showthread.php?s=&threadid=10238080

[mwood_2k2]

Ok I'm sorry but I don't get how to adapt that into my current script, but it does seem to contain the correct things, I'm sorry but could you have a go (I have been searching the forum but never found THAT post) I tired and annoyed and my brain has stopped working lol ....

Heres what I get into my inbox (raw)

From [e-mail] Fri May 02 12:58:11 2003
Return-path: <[e-mail]>
Envelope-to: [e-mail]
Delivery-date: Fri, 02 May 2003 12:58:11 -0600
Received: from web20203.mail.yahoo.com ([216.136.226.58])
by megs11.100mwh.com with smtp (Exim 3.36 #1)
id 19BfjX-0004Wh-00
for [e-mail]; Fri, 02 May 2003 12:58:11 -0600
Message-ID: <[e-mail]>
Received: from [62.30.112.1] by web20203.mail.yahoo.com via HTTP; Fri, 02 May 2003 12:00:17 PDT
Date: Fri, 2 May 2003 12:00:17 -0700 (PDT)
From: Mark Wood <[e-mail]>
Subject: LOOOOOO
To: [e-mail]
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="0-2018987153-1051902017=:66347"
X-Spam-Status: No, hits=0.9 required=5.0
tests=HTML_30_40,HTML_MESSAGE
version=2.53
X-Spam-Level:
X-Spam-Checker-Version: SpamAssassin 2.53 (1.174.2.15-2003-03-30-exp)

--0-2018987153-1051902017=:66347
Content-Type: text/plain; charset=us-ascii

BOOOOOO

---

Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.
--0-2018987153-1051902017=:66347
Content-Type: text/html; charset=us-ascii

<DIV>BOOOOOO</DIV><p><hr SIZE=1>
Do you Yahoo!?<br>
<a href="http://us.rd.yahoo.com/search/mailsig/*[url]http://search.yahoo.com[/url]">The New Yahoo! Search</a> - Faster. Easier. Bingo.
--0-2018987153-1051902017=:66347--

and heres my script;

#!/usr/bin/php -q
<?php
$fd = fopen("php://stdin", "r");
$email = "";
while (!feof($fd)) {
    $email .= fread($fd, 1024);
}
fclose($fd);

$lines = explode("\n", $email);
$from = "";
$subject = "";
$headers = "";
$message = "";
$splittingheaders = true;

for ($i=0; $i<count($lines); $i++) {
    if ($splittingheaders) {
        $headers .= $lines[$i]."\n";
        if (preg_match("/^Subject: (.*)/", $lines[$i], $matches)) {
            $subject = $matches[1];
        }
        if (preg_match("/^From: (.*)/", $lines[$i], $matches)) {
            $from = $matches[1];
        }
		if (preg_match("/^MIME-Version: (.*)/", $lines[$i], $matches)) {
            $headers = $matches[1];
        }
		if (preg_match("/^Content-type: (.*)/", $lines[$i], $matches)) {
            $headers = $matches[1];
        }
		if (preg_match("/^X-Spam-Checker-Version: (.*)/", $lines[$i], $matches)) {
            $headers = $matches[1];
        }
		if (preg_match("/^X-Spam-Level: (.*)/", $lines[$i], $matches)) {
            $headers = $matches[1];
        }
		if (preg_match("/^X-Spam-Status: (.*)/", $lines[$i], $matches)) {
            $headers = $matches[1];
        }

} else {
    $message .= $lines[$i]."\n";
}
if (trim($lines[$i])=="") {
    $splittingheaders = false;
}
}

$link = mysql_connect("localhost", "user", "pass")
        or die("Could not connect: " . mysql_error());

mysql_select_db("database") or die("Could not select database");

$query = mysql_query("SELECT count(email) from maillist where email='$from'"); 
$num = mysql_num_rows($query); 

if ($num) 
{ 
$query = mysql_query("SELECT email from maillist");
$num = mysql_num_rows($query); 
while ($email = mysql_fetch_array($query) )
{ 
$to = $email["email"]; 

$header = "From: [email][e-mail][/email]\n"; 
$header .= "Reply-To: [email][e-mail][/email]\n"; 
$header .= "MIME-Version: 1.0\n";
$header .= "Content-type: text/html\n;"; 
$header .= "X-Priority: 1\n"; 
$header .= "X-MSmail-Priority: High\n"; 
$header .= "X-mailer: DAUMED\n"; 

mail("$to", "$subject", "$message", "$header"); 
}
} 
else
{ 
mail("$from", "Unauthorised", "You are not authorised to post to this list...", "$header"); 
}

?>

Cheers in advance (sorry to be a burden!)

[wrestler145]

$header = "From: [e-mail]\n";

$header .= "Reply-To: [e-mail]\n";

$header .= "MIME-Version: 1.0\n";

$header .= "Content-Type: multipart/related;\n boundary=$message_Boundary";

$header .= "X-Priority: 1\n";

$header .= "X-MSmail-Priority: High\n";

$header .= "X-mailer: DAUMED\n";

The problem I had with sending Content-Type: text/html was my browser saw it as html, but the person I was sending it to saw the html coding.

After you make this change you need to start the boundary with
(contained in the message body)
--$message_boundary

next you need to tell the browser what to expect next

Content-Type: text/html;charset=US-ASCII
Content-Transfer-Encoding: 7bit\n

Then your message

Boundary ending
\n\n--$message_boundary--\n

See if that doesn't help you.

[mwood_2k2]

that appears to only help with sending ... what I'm after is decoding MIME email, because thats what comes in, and as the script waits until a blank line before assuming the rest is the body, it ends up including the mime headers in the message ...

Would sending it how you've suggested correct the headers?

Cheers

[CaptianChaos]

See:
http://www.phpbuilder.com/board/showthread.php?s=&threadid=10238097 for a solution using PEAR's mailparse class.

MIME-Decoding is not going to be a fun subject. You will spend a few days getting the Yahoo mail format to work, then you'll find another message format that you need to decode. You'll spend a few more days getting that format to work. Then another, then another... you will struggle a lot...

If you really want to parse these messages yourself, you need to read up on rfc822 and rfc2045, which define the standard for mail message format.

http://www.faqs.org/rfcs/rfc822.html
and
http://www.faqs.org/rfcs/rfc2045.html

Then, get really good at regular expressions, you'll need to use them extensively. Here is a start:

http://etext.lib.virginia.edu/helpsheets/regex.html

[mwood_2k2]

There has to be some simple way of doing it, like some way of making sure the script picks up those 2 headers after the first 2 blank lines and then the rest is the message, like I said the problem I think is just that the MIME headers arent being included in headers because of those 2 blank lines, it's being included in the message itself....

I don't quite understand how to use regular expressions from that link ... lol

Cheers

[mwood_2k2]

After doing some strenous searching, I have found this maildecode file that doesn't seem to rely on PEAR being installed, I have it attatched ....

Could you please help me on how I could fit this into the mail splitting code?

Cheers in advance, and thanks for your help 🙂

[CaptianChaos]

Here is how to use this class. Save the class in a file called DecodeMessage.inc then try the code below. The problem I found is that it doesn't actually decode your message the way you want it to. I think the Result() funtion in the class is supposed to give back the found text/html portion of the email. I haven't been able to get it to work, though. Try this, then maybe we can fix the class to actually work.

<?php
include("DecodeMessage.inc");
$msg = implode("", file('email3'));
$mObject = new DecodeMessage();
$mObject->InitMessage($msg);
$headers = $mObject->Headers();
$body = $mObject->Body();
$result = $mObject->Result();

print $headers;
print "\n";
print $body;
print "\n*****************\n";
print $result;
?>

[mwood_2k2]

Ok, I tried what you suggested, I set email3 as a file containing a raw email .... here are the results ...

From [e-mail] Fri May 02 16:34:46 2003 Return-path: Envelope-to: [e-mail] Delivery-date: Fri, 02 May 2003 16:34:46 -0600 Received: from web20205.mail.yahoo.com ([216.136.226.60]) by megs11.100mwh.com with smtp (Exim 3.36 #1) id 19Bj78-0001It-00 for [e-mail]; Fri, 02 May 2003 16:34:46 -0600 Message-ID: <[e-mail]> Received: from [62.31.224.1] by web20205.mail.yahoo.com via HTTP; Fri, 02 May 2003 15:36:54 PDT Date: Fri, 2 May 2003 15:36:54 -0700 (PDT) From: Mark Wood Subject: sdfsdfdfbvfhgfhfghjdfg To: [e-mail] MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-1001802025-1051915014=:81410" X-Spam-Status: No, hits=0.9 required=5.0 tests=HTML_30_40,HTML_MESSAGE version=2.53 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.53 (1.174.2.15-2003-03-30-exp) --0-1001802025-1051915014=:81410 Content-Type: text/plain; charset=us-ascii asdashdvfjsdvhcs --------------------------------- Do you Yahoo!? The New Yahoo! Search - Faster. Easier. Bingo. --0-1001802025-1051915014=:81410 Content-Type: text/html; charset=us-ascii
asdashdvfjsdvhcs

---

Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo. --0-1001802025-1051915014=:81410-- ***************** Array

Ok, any ideas? lol

Cheers again

[CaptianChaos]

YIKES! Let's make the output more pretty:

<?php
include("DecodeMessage.inc");
$msg = implode("", file('email3'));
$mObject = new DecodeMessage();
$mObject->InitMessage($msg);
$headers = $mObject->Headers();
$body = $mObject->Body();
$result = $mObject->Result();

print nl2br(htmlspecialchars($headers));
print "<br>\n";
print nl2br(htmlspecialchars($body));
print "<br>\n*****************<br>\n";
print_r($result);
?>

My guess is that this will work fine, however, the last part, after the ********** will produce nothing. I am not terribly excited about fixing this class to work properly. Why reinvent the Wheel? Have you tried using the mailparse.so file I gave you the URL for earlier. If that method works, you will be in much better shape.

[CaptianChaos]

Here is complete output from my last test. Notice the part after the ****************. that is the $result variable. Interesting, eh, the body section inside the $result object is exactly what you are looking for. Now you just have to find a way to manipulate the $result array to get the text part out.

Message-ID: <[e-mail]>
Date: Sun, 4 May 2003 09:16:36 -0700 (PDT)
From: David Koopman <[e-mail]>
Subject: From Yahoo
To: [e-mail]
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="0-233061671-1052064996=:54937"

--0-233061671-1052064996=:54937
Content-Type: text/plain; charset=us-ascii

This message is a test from Yahoo

---

Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.
--0-233061671-1052064996=:54937
Content-Type: text/html; charset=us-ascii

<DIV><STRONG>This message is a test from Yahoo</STRONG></DIV><p><hr SIZE=1>
Do you Yahoo!?<br>
<a href="http://us.rd.yahoo.com/search/mailsig/*[url]http://search.yahoo.com[/url]">The New Yahoo! Search</a> - Faster. Easier. Bingo.
--0-233061671-1052064996=:54937--

---

Array
(
[0] => Array
(
[0] => Array
(
[body] => Array
(
[type] => text/html
[body] =>
<DIV><STRONG>This message is a test from Yahoo</STRONG></DIV><p><hr SIZE=1>
Do you Yahoo!?<br>
<a href="http://us.rd.yahoo.com/search/mailsig/*[url]http://search.yahoo.com[/url]">The New Yahoo! Search</a> - Faster. Easier. Bingo.

                    )

            )

    )

)

Ok, that's it. Hmm, this is tricky. The piece we want is actually here:
$result[0][0]['body']['body'];

So, instead of printing the entire object:
print_r($result);
we could:
print $result[0][0]['body']['body'];

But how do we traverse down the array so that you find the displayable text everytime? I need help with this, I don't know.

[CaptianChaos]

I have tried this example with the class you gave me on a message that has an attachment and it says:

Cannot open file "attachments/example.doc"

and dies. This means that once we get this class working for the example from Yahoo, you still have a slew of kinks to work out. This task is getting harder by the minute.

I think you should try the mailparse.so example. Somehow, someway, you need to use mailparse. It is made to do exactly what you are trying to do and it works. Why reinvent the wheel?

If you cannot get it to work with the mailparse.so file I sent you, you can download mailparse for PHP-4.2 here:
http://thebrainroom.com/opensource/php/mailparse.php

[mwood_2k2]

That doesn't work, I have tried it, it complains about the include ... (pear.php) and so my host must not have it installed, although my host does have php 4.2.3

Edit: Having just tested the second solution, I get the following error output in my browser ...

"Fatal error: Call to undefined function: mailparse_msg_parse_file() in /home/mwood_2k/public_html/decode/try.php on line 15"

Line 15 points to the line below;

$mime = mailparse_msg_parse_file($filename);

Why does this have to be so hard ... ya think someone woulda done something more compatible overall, or something that didn't have dependencies ... I mean it would help so many noobs (including me lol). Maybe you could hand the floor over to someone else? There has been plenty of views on this here thread, I bet I'm annoying you royally arent I lol. If you can find someone that would give this a shot (for the sheer challenge of doing so lol) then thats fine, I appreciate ur help so far you've been great!

Any more tricks up them there sleeves of yours? 🙂

Cheers

[mwood_2k2]

Ok, I have managed to get this working ...

<?php
include("DecodeMessage.inc");
$msg = implode("", file('email3'));
$mObject = new DecodeMessage();
$mObject->InitMessage($msg);
$headers = $mObject->Headers();
$body = $mObject->Body();
$result = $mObject->Result();

?>

And I have been able to get it to render properly on a webpage with both yahoo and msn emails! The code to get the body out ($result[0][0]['body']['body']😉 works on both hotmail and yahoo emails even when I have tried varying messages, and so it seems to work doesnt it?

I set the variable body2 with $body2 = $result[0][0]['body']['body']; and then outputted the body2 variable with an echo to accomplish it.

Does this script also output the headers in an array so I could do the same with the body but for things like content-type and reply-to etc??

Cheers