Not working!? Capture shell output with exec()

kristo5747

Greetings!

I have the task to parse log files for several web servers and output a report in a web page (the later part's easy).

My problem: the log files are huge (> 1G😎 so using any of the PCRE functions is not doable. Out of memory error will happen.

So I need to rely on the OS to do the bulk of the work. From the command line, I can run this grep statement to look for a specific URL:

grep \"basket.htm?skuId=10010\" weblogfile | grep --only-matching '\"GET[^\"\-]*\"' | sort -u | wc -l

if a match is found for that Sku, I simply count the number of occurrences in the log file (that's how I build the report).

If no match is found, the grep command returns 0 (as expected) and I am supposed to output "N/A" in my report. I started reading up about exec() this morning but I can't get it to capture the grep command's output

...
/$CMD5 = "grep \"static_tvapp.htm?widgetId=".$widgetID[$i]."\" tmpfile | grep -oa '\"GET[^\"\-]*\"' | sort -u | wc -l";
@exec($CMD5, $CMD5_output, $retval);
if ($CMD5_output != "0") {
exec('echo N/A' >> Baskets.csv');
} else {
...do something else...
}

It simply does not work. What am I missing? Can someone please help?

Thank you.

Al.

bradgrafelman

kristo5747 wrote:
My problem: the log files are huge (> 1G😎 so using any of the PCRE functions is not doable.

Why not? Who says you have to load the entire file into memory all at once?

kristo5747 wrote:
What am I missing?

You're treating $CMD5_output as if it were a scalar value, when in fact it is not - it's an array (as per the manual for exec()).

sneakyimp

If you wanted to parse the file one line at a time you could do so using [man]fgets[/man] and then preg_match each line.

I think using grep is probably going to be faster. You should read the manual on [man]exec[/man]. I could also advise checking $retval to make sure the command executed properly.