[RESOLVED] mCrypt and Linebreaks

[bpat1434]

Hi all...

Working on a system to encrypt and decrypt files and strings and such. I have the encryption down fine. Where my issue is, is the decryption.

Now, if I download the file directly, I see there are line-breaks there ("\n"). If I use the following code, it doesn't catch them:

$lines = explode("\n", $contents);

Now, $contents holds the encrypted file contents returned from file_get_contents(). If I just dump the $contents variable, it's as expected (the encrypted information). But it's the decryption that frustrates me. I am passing the IV along with the file, and it's either on the first or last line of the file. So I need to explode the string, but it just won't do it.....

Any ideas? Why would mCrypt be causing issues with PHP finding line endings? I've tried them all (\r \r\n \n) and none work.

Oh, and after the explosion, if I check to see if it's an array (and not a string), it passes, and the first element is the entire string 🙁

[bogu]

Grab the file content using php file function ...

[bpat1434]

It's closer... but not quite....

Now that I get everything like that, I finally got the IV value. So decrypting the first like 5 lines works. But the rest of the file is still encrypted..... Any ideas?

function decryptIt($lines)
{
	global $algorithm, $mode, $cryptKey, $crlf;

$dcrypt = array();

$iv = trim($lines[0]);
array_shift($lines);

$td = mcrypt_module_open($algorithm, '', $mode, '');
$ks = mcrypt_enc_get_key_size($td);

$ck = substr($cryptKey, 0, $ks);
mcrypt_generic_init($td, $ck, $iv);

foreach($lines as $line)
{
	$dcrypt[] = mdecrypt_generic($td, trim($line));
}

mcrypt_generic_deinit($td);
mcrypt_module_close($td);

$data = implode($crlf, $dcrypt);
return $data;
}

$lines is the returned value of [man]file/man
All the globals are defined in a pre-included config file.
The file was encrypted using the same algorithm, mode, and crlf settings....
If I don't use trim($line) during decrypt, nothing gets decrypted....

[Drakla]

Hmmm, at a guess I'd say that the crypt is producing characters that are getting interpreted as line breaks so you're getting a false split. I add the length of the data as an int at the front by packing it, then the IV, which is a fixed size for each type of encryption which I use [man]mcrypt_enc_get_iv_size[/man] to find.

Actually I'll just tag on the class I made - it's a bit patchy as it has some internal flags I didn't think too well about, but the rest is fine.

[bpat1434]

Yeah, I took your class and pretty-much rewrote it for my needs..... doesn't seem to work... although it decrypts more of the file, it still doesn't get all of it... nor does it retain line-breaks 🙁

Here's my class.... maybe you can see something:

<?php
include_once('config.php');

class mCrypt
{
	/** Some default settings **/
	var $_dfltAlgorithm		=	'serpent';
	var $_dfltMode			=	'ofb';
	var $_dfltKey			=	'MyPersonalCryptKey';

/** And now the basics for encryption **/
var $ckey				=	null;
var $keySize			=	null;
var $iv					=	null;
var $ivSize				=	null;

/** The encryption method vars **/
var $algorithm			=	null;
var $mode				=	null;
var $td					=	null;
var $encrypted			=	null;
var $decrypted			=	null;

var $isEncrypted		=	null;

function __construct()
{
	/* PHP 5 Operability */
	if($algorithm === null)
		$this->set_algorithm($this->_dfltAlgorithm);
	if($mode === null)
		$this->set_mode($this->_dfltMode);
	if($key === null)
		$this->set_key($this->_dfltKey);
}

function mCrypt()
{
	/* PHP 4 Compatiblity */
	$this->__construct();
}

/**
 *		PUBLIC
 *
 *	Functions to be called at any point
 *	during the encryption process.
 */

function encrypt($decrypted = null, $key = null)
{
	if($decrypted !== null)
		$this->set_decrypted($decrypted);
	if($key !== null)
		$this->set_key($key);

	if($this->open() === false)
		return false;

	$key = substr($this->ckey, 0, $this->keySize);

	if(mcrypt_generic_init($this->td, $key, $this->iv) < 0)
	{
		$this->close();
		return false;
	}

	$data = pack('N', strlen($this->decrypted)) . $this->decrypted;

	$this->encrypted = $this->iv . mcrypt_generic($this->td, $data);

	$this->isEncrypted = true;
	$this->close();

	return $this->encrypted;
}

function decrypt($encrypted = null, $key = null)
{
	if($encrypted !== null)
		$this->set_encrypted($encrypted);
	if($key !== null)
		$this->set_key($key);

	if($this->open() === false)
		return false;

	$key = substr($this->ckey, 0, $this->keySize);
	$iv = substr($this->encrypted, 0, $this->ivSize);
	$decrypted = substr($this->encrypted, $this->ivSize);

	if( (strlen($iv) < $this->ivSize) || (strlen($this->ckey) == 0) || (mcrypt_generic_init($this->td, $this->ckey, $iv) < 0) )
	{
		mcrypt_module_close($this->td);
		return false;
	}

	$this->decrypted = mdecrypt_generic($this->td, $decrypted);

	$this->close();

	$length = unpack('N', substr($this->decrypted, 0, 4));
	$this->decrypted = substr($this->decrypted, 4, $length[1]);

	$this->isEncrypted = false;

	return $this->decrypted;
}

/**
 *
 *		PRIVATE
 *
 *	Functions used privately by other
 *	functions of this class.
 *
 */
function set_encrypted($enc)
{
	$this->encrypted = $enc;

	$this->isEncrypted = true;
}

function set_decrypted($dec)
{
	$this->decrypted = $dec;

	$this->isEncrypted = false;
}

function get_encrypted()
{
	if($this->isEncrypted)
		return $this->encrypted;
	else
		return $this->encrypt();
}

function get_decrypted()
{
	if(!$this->isEncrypted)
		return $this->decrypted;
	else
		return $this->decrypt();
}

function set_algorithm($alg)
{
	if($alg !== null)
		$this->algorithm = $alg;
	else
		return false;
	return true;
}

function set_mode($mode)
{
	if($mode !== null)
		$this->mode = $mode;
	else
		return false;
	return true;
}

function set_key($key)
{
	if($key !== null)
		$this->ckey = $key;
	else
		return false;
	return true;
}

function open()
{
	$this->close();

	$this->td = mcrypt_module_open($this->algorithm, '', $this->mode, '');

	if(false === $this->td)
		return false;

	$this->keySize = mcrypt_enc_get_key_size($this->td);
	$this->ivSize = mcrypt_enc_get_iv_size($this->td);
	$this->iv = $this->create_iv();
}

function close()
{
	if(is_resource($this->td))
	{
		mcrypt_generic_deinit($this->td);
		mcrypt_module_close($this->td);
	}
}

function create_iv($size = null)
{
	$iv = '';

	$size = ($size !== null) ? $size : ( ($this->ivSize !== null) ? $this->ivSize : mcrypt_enc_get_iv_size($this->td) );

	for($i=0; $i < $size; $i++)
		$iv .= chr(mt_rand(0, 255));

	$this->iv = $iv;

	return $iv;
}
}

?>

[Drakla]

Nope, can't see anything wrong. I've also run my version with a piece of data with stacks of line breaks and it decrypts it back fine, even when changing the methods to serpent and ofb.

I've just read that on windows you've got to get the new copy of libcrypt.dll otherwise it's bad batch, maybe it's that. And I notice you're calling the decrypt generic in a loop - I thought that you had to initialise before every call of decrypt, so it might be that, too. In my method I do the whole lot as one massive chunk.

[bpat1434]

I've just read that on windows you've got to get the new copy of libcrypt.dll otherwise it's bad batch, maybe it's that.

No, it's running on a linux box on the server that's actually doing the encryption/decryption when I finish this....

And I notice you're calling the decrypt generic in a loop - I thought that you had to initialise before every call of decrypt, so it might be that, too. In my method I do the whole lot as one massive chunk.

Where in my class (based off yours) do I do a loop? I do it as a chunk too, and it doesn't even finish the first line....

So let me pose a few questions:
1.) You pack and unpack the data..... how does that help you? I don't really get it.... 🙁 I've read the manual a few times, but what's the point of padding to 32-bits?
2.) Why cut the decrypted information the way you do? It's in the decrypt function... you'll see it....

[Drakla]

I pack the length of the unencrypted string as a 32 bit int and put it at the start of the string before it's encrypted, that way I know the first 4 bytes of my result will give me back the original length. The reason for that is that just like it says in the manual for mdecrypt_generic:

"Note that the length of the returned string can in fact be longer then the unencrypted string, due to the padding of the data."

As for looping, I don't know the reason you're doing it, but it wouldn't be any part of the class. If you were doing lots of encrypting then

foreach ($strings as $string)
{
	$mc->set_decrypted($string);
	$encr[] = $mc->encrypt();
}

But check my class as it is with your data doing something like the test in the zip

$string = "I'm a little teapot, now some jibba +_)(*&^%\$£!{}[]:@;'<>?,./áéíóú <b>And after the jibba jabba</b> back to normal.
a
b
c
d
e
f
g
h
i
j
k
l
m
n
o
p
q
r
s
t
u
vw
x
yz";
$key = 'mah jajjaaa';

$mc = &new Mcrypt();
$mc->set_key($key);

$mc->set_decrypted($string);

$enc = $mc->encrypt();
$dec = $mc->decrypt();


echo "Encrypted: $enc<br />";

echo "Strlen encrypted: ".strlen($enc).'<br />';
echo "Key Size: ".$mc->keySize.'<br />';
echo "IV Size: ".$mc->ivSize.'<br /><br />';

echo 'Decrypted: '.nl2br($dec).'<br />';
echo "Strlen data: ".strlen($dec).'<br /><br />';

[bpat1434]

Okay, that's understandable. But your linebreaks aren't coded like mine. I mean, mine is like:

$data = 'Some data for whatever reason' . $crlf . 'Some more data on a different line';

Where $crlf is either "\r\n", "\n", "\r" or whatever other delimiter I choose.....

As for the looping, where do I loop? You said that I looped to encrypt the data? Where? I dont' see it.

[Drakla]

I think there might be some confusion, I thought you were looping to encrypt a lot of different pieces of data seperately. If you have one big piece of data with line breaks in it then just encrypt the whole lot in one call. It doesn't matter what you're using for line breaks as the method is binary safe.

[bpat1434]

Okay, that's what I'm doing now... and it's losing the line-breaks.... I'll work on this more later today (like in an hour) as i'm at work now.

I have a feeling it's something easy.

Extra Info
I was originally splitting the information up line-by-line to encrypt it, and decided not to do that. So now it's all one big string 😉

[Drakla]

Okey doke - but in that case, are you initialising the encrypt call every time in the loop - because you have to according to Mr. Php Manual - so your error might be in the encoding in the first place, and that's why you only get the 1st line decrypted.

[bpat1434]

Well, I've tried a few things... and it's getting more than the first line done. It's just that after a while, it stops decoding.....

Here's the class to encode/decode:

function encrypt($decrypted = null, $key = null)
	{
		if($decrypted !== null)
			$this->set_decrypted($decrypted);
		if($key !== null)
			$this->set_key($key);

	if($this->open() === false)
		return false;

	$key = substr($this->ckey, 0, $this->keySize);

	if($this->minit(null, $key, $this->iv) === false)
		return false;

	$data = pack($this->_packFormat, strlen($this->decrypted)) . $this->decrypted;

	$this->encrypted = $this->iv . mcrypt_generic($this->td, $data);

	$this->isEncrypted = true;
	$this->close();

	return $this->encrypted;
}

function decrypt($encrypted = null, $key = null)
{
	if($encrypted !== null)
		$this->set_encrypted($encrypted);
	if($key !== null)
		$this->set_key($key);

	if($this->open() === false)
		return false;

	$key = substr($this->ckey, 0, $this->keySize);
	$iv = substr($this->encrypted, 0, $this->ivSize);
	$encrypted = substr($this->encrypted, $this->ivSize);

	if( (strlen($iv) < $this->ivSize) || (strlen($this->ckey) == 0) || (mcrypt_generic_init($this->td, $this->ckey, $iv) < 0) )
	{
		mcrypt_module_close($this->td);
		return false;
	}
	if($this->minit(null, $key, $iv) === false)
		return false;

	$this->decrypted = mdecrypt_generic($this->td, $encrypted);

	$this->close();

	$length = unpack($this->_packFormat, substr($this->decrypted, 0, 4));

	$this->decrypted = substr($this->decrypted, 4, $length[1]);

	$this->isEncrypted = false;

	return $this->decrypted;
}

/*** A ways down in the class ***/
function minit($td = null, $key = null, $iv = null)
	{
		if($key === null || $iv === null || ($td === null && empty($this->td)))
			return false;
		if($td === null && !empty($this->td))
			$td = $this->td;

	if(($err = mcrypt_generic_init($td, $key, $iv)) < 0)
	{
		$this->close();
		return false;
	}

	return true;
}

No loops are used to encode it. It's done on one line. It's given one large string, and it encodes it. During decode, it decodes either just less than 1 line, or it decodes the first 20 lines, and then stops.....

[EDIT]
Woops... missed the initialization of the buffer... but that still doesn't help. I'm still not getting the true decrypted data.... only a little bit of it....

[Drakla]

Can you zip up the data or script that generates it if it's not too big/confidential to see if I'm getting the same thing here? I'll put it on my server (Linux) as well as testing it locally.

[bpat1434]

It kind of is sensitive. It's a forum dump, so it's got passwords and email addresses.

But if you just do a quick dump of a database into a string, you can try. I'll zip the files (edited for my security) and post them here.

[Drakla]

I did multiple read/encode/write/readback/decode of dumps with my class and it all worked fine, so I had a look at your conversion of the class and there were a few things missing. You've got the var $ckey instead of key and forget to use $this in __construct for the tests - I've changed that and it seems to be all aok. Mind you, I'm just using your version of the class not using all of the other scripts.

	/** And now the basics for encryption **/
	var $key 				=	null;	// <-- added
	var $ckey				=	null;
	var $keySize			=	null;
	var $iv					=	null;
	var $ivSize				=	null;

/** The encryption method vars **/
var $algorithm			=	null;
var $mode				=	null;
var $td					=	null;
var $encrypted			=	null;
var $decrypted			=	null;

var $isEncrypted		=	null;

function __construct()		// <-- added the $this->...
{
	/* PHP 5 Operability */
	if($this->algorithm === null)
		$this->set_algorithm($this->_dfltAlgorithm);
	if($this->mode === null)
		$this->set_mode($this->_dfltMode);
	if($this->key === null)
		$this->set_key($this->_dfltKey);
}

see if that works, it can't be that far off now.

[bpat1434]

1.) key is a reserved word in PHP, so I changed $this->key to be $this->ckey. So $this->key doesn't seem necessary.

2.) Why did you add $this-> to __construct? I see no purpose for it.

I've done multiple read/encode/decodes with the class too, and for some odd reason, it works. But when I get the information from mySQL databases, it doesn't seem to work.

Right now, I'm going to modify the code to write the SQL to a file, then read that file, then encode it and try and decode it. See if that does anything.....

[Drakla]

Well I've used $key throughout the class, so unless all the other instances are changed then it's busted from the start.

I use $this-> because it's needed, otherwise you get:

Notice: Undefined variable: algorithm in E:\MyPhp\php5\bpat\mcrypt.class.php on line 31

Edit: actually the constructor needs parameters for the inside code to make sense, and of course then you would take off the $this-> on the tested vars

[bpat1434]

Hmm... I'm not using PHP5... using PHP4..... but okay.....

AaaaaaHAAAA!!!!
Okay... so I've really narrowed it down!!!!

After testing (with the intermediate file) I decided to see something. I didn't delete the intermediate file, then wrote a script to create two instances of the class. One encrypted it. The other decrypted. The original was unencrypted.

This worked as expected. All the data was there, and was unencrypted properly 🙂

So, I figure it has something to do with the FTP transfer. So this now begs the question: which method to use?
FTP_BINARY yields mostly good decoding info
FTP_ASCII doesn't yield too much decoded info.

So what could be going on? I really need the FTP availability. I plan on using it to have multiple backups of the same file(s) as redundancy. So now this turns into an encryption and FTP issue I guess.....

[Drakla]

I'm using both php4 and 5, and tested under 5 as I thought that's what you were using. You definitely need binary mode in and out. Stupidly it was one of the first things I thought of when looking at your code but then forgot about it straight away.

I'll run some ftp tests too and see what's cookin'