Delete patterns from text file

[scrupul0us]

So in follow up to my pattern/regex thread:
http://www.phpbuilder.com/board/showthread.php?t=10371836

I know have 80% of the data I need inside a 2d array... What I need to do is loop throw that array, and based on the contents of sub-array element 0 remove that data from the text file

example:

array (
[0] => array (
     [0] the entire string found
     [1] the entire
     [2] string found
)
[1] => array (
     [0] jimmy cracks corn dude
     [1] jimmy cracks
     [2] corn dude
)
)

so based on this sample array I'd need to remove 'the entire string found' and 'jimmy cracks corn dude' from my massive text file (mind you my actual data ranges from 1-6 lines long)

[halojoy]

<?php

//$fullarray ... is the old array to fix

$newarray = array(); //to put fixed subarrays in

foreach($fullarray AS $subarr){
    // test and fix each sub array
    echo $subarr[0] . '<br>';
    if( $subarr[0] == xxxx ){ //if you need to test before delete
          unset($subarr[0]); // so, remove this value
    }
    $newarray[] = $subarr; //Load new without $subarr[0] value
}

// See the result of operation
echo '<pre>';
print_r($fullarray);   // before
print_r($newarray); // after

?>

[scrupul0us]

I don't need to remove the array value, I need to take the contents of the sub-array[0], find it in the text file and then delete it

[scrupul0us]

Well I thought I could try this:

$newData='';
foreach($machines as $machine)
{
	$newData .= str_replace($machine[0],'',$testData);
}
$file_put_contents('source-modified.txt',$newData);

but that threw an exhausted memory error on the str_replace line...

Fatal error: Allowed memory size of 16777216 bytes exhausted (tried to allocate 218945 bytes)

i have 2gb of ram in the machine so perhaps I should raise the mem limit

[scrupul0us]

Well I raised the limit from 16M to 128M and it works... but... it's not removing ANY of the instances where there are \r or \n (any blocks) just single line entries...

Trying to use preg_replace doesn't work either because the strings may or may not contain characters that need to be escaped, etc

There has to be a way!

[bradgrafelman]

scrupul0us;10945278 wrote:

Trying to use preg_replace doesn't work either because the strings may or may not contain characters that need to be escaped, etc

[man]preg_quote/man ?

[scrupul0us]

OK so I tried:

preg_replace('#'.preg_quote($machine[0]).'#','REPLACED',$testData);

...now nothing gets replaced... 🙁

[bradgrafelman]

scrupul0us;10945280 wrote:

OK so I tried:

preg_replace('#'.preg_quote($machine[0]).'#','REPLACED',$testData);

...now nothing gets replaced... 🙁

Does '#' appear in $machine[0]? You'll want to specify that '#' is your delimiter when using [man]preg_quote/man. Does the case match?

Could you show us some sample $testData as well as what was in $machine[0]?

[scrupul0us]

What I ended up doing was this for each regex:

preg_match_all($regexUsed1,$testData,$matches1,PREG_SET_ORDER);
$newData = preg_replace($regexUsed1,'###REGEX1###',$testData);

It dawned on me... if the regex is good enough to match the data, it's good enough to replace it immediately following =)

For the time being I'm replacing with ###REGEX### so I can view which is eating what data and determine what if any regex needs to be made less/more greedy so it doesnt eat/spew partial blocks

[Weedpacket]

Going back to a slightly earlier version...

$newData='';
foreach($machines as $machine)
{
    $newData .= str_replace($machine[0],'',$testData);
}
$file_put_contents('source-modified.txt',$newData); 

While not addressing the problem you mention, I do see two other issues.

First is the most serious: you're appending a copy of the entirety of $testData (less the results of one match) to $newData each time through the loop.

Second, [man]str_replace[/man] (and [man]preg_replace[/man] can take arrays of search strings, so the loop itself only needs to collect all the strings to be replaced into a single array. The replacement can be done afterwards.

[scrupul0us]

Weedpacket;10945418 wrote:

First is the most serious: you're appending a copy of the entirety of $testData (less the results of one match) to $newData each time through the loop.

Odd... It never grows in lines or filesize... It's actually been decreasing in lines and filesize as I've been pulling more data out... Very odd b/c the default mode is appeand not overwrite

For the moment however this is all happening "offline" with local copies and the output to TXT file is nothing more than a system of checks and balances with the regexes

I will have to look into why that isn't appending though😕