Custom Log Charting

webdes03

We currently run AWStats on our webservers, but I have a project where we want to make some custom charts from the access log. Initial thoughts were to use pChart and some PHP scripting to read the logs and generate cached stats (possibly setup on a cron job to update every night).

I wanted to ask if anyone has had any experience with this type of project before, or any suggestions on how to proceed.

While reading and processing the log is a big piece of the puzzle, I have yet to figure out the best way of actually accessing the data as it's not in a public folder. If we run the updates via a cron job that won't matter, but if we wanted to do live charts then we'd have to make a copy to the public folder.

I'm pretty good with PHP, but I'm totally new to setting up my own cron jobs.

Any input is greatly appreciated, thanks!

HalfaBee

The data doesn't have to be in a public folder, php should be able to read it even if it is outside the public dirs.

For more help on the data extraction, more information is needed.

webdes03

So assuming I can simply open the file using fopen() or a similar function, what would be the best way of actually reading the file? I found this information online about extracting the data from each line, but I'm not sure how to go about reading each line of the log. Once I've read by line, I can use the code snippit below to add each value to an array, so I have an array of all the IP's, an array of all the request codes, etc., and build the charts from those arrays.

preg_match (
  '/^(\S+) (\S+) (\S+) \[(.+)\] \"(.+)\" (\S+) (\S+) \"(.*)\" \"(.*)\"/'
, $logLine
, $match
);

$l_clientAddress     = $match[1];
$l_rfc1413           = $match[2];
$l_username          = $match[3];
$l_localTime         = $match[4];
$l_httpRequest       = $match[5];
$l_statusCode        = $match[6];
$l_bytesSentToClient = $match[7];
$l_referer           = $match[8];
$l_clientSoftware    = $match[9];