alternative for preg_replace('/([\\x00-\\x2f\\x3a-\\x40\\x5b-\\x60\\x7b-\\xc2])/e')

lordkadmus

Hi fellows,

I was given a task to strip unnecessary characters out of a article compilation. The compilation being few gigs means a lot of work.
I am using the following code to rip the characters

$thetext=preg_replace('/([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xc2])/e', "_" , $thetext);

I am not using the 'only allow' mechanism because the books contain iso-8859-1 chars which implies that it can have like so many other characters that I will have to personally include to allow!

Anyways
this code

$thetext=preg_replace('/([\\x00-\\x2f\\x3a-\\x40\\x5b-\\x60\\x7b-\\xc2])/e', "_" , $thetext);

kinda kills the cpu, it is very resource intensive. So, my question being, IS THERE A better way of doing this pleaseeee?

NogDog

I'm not so sure that the preg_replace() is the problem so much as the fact that you are trying to process a "few gigs" at one time.

lordkadmus

ah well, I know I am going after a lot of data but i record the time interval for every 100 kilobyte which has increased a lot after preg_replace('/([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xc2])/e') .

Before using this hex pattern I was using individual symbols for replacement purposes which was around 5-10 times faster but it was not a very friendly function. At times php assumed that few characters were joined with the other to make a single character which had consequences! The items couldn't be removed then.

So any more suggestions?

NogDog

I just noticed the "/e" modifier, but I see no reason for it from the code you've supplied. I would start by getting rid of that, as it instructs PHP to eval() the following argument, which is certainly going to add to the processing time.