I've been using this Log class to conveniently write to a log such that the log file will not grow without bound. The class takes care of rotating the log file and keeping a few backups so that the file never gets too big. I was considering using it in a multiprocessing application -- i.e., there are many processes potentially trying to write the same exact log file. If that log file happens to be in need of rotation, there is the potential for exceptions to be thrown if two processes are trying to write and/or rotate the log at the same time. The odds are pretty long but it could happen.

Can anyone suggest how I might improve this log class such that many processes can simultaneously write to the same log file without the possibility of processes trying to write a file that is being rotated or rotate a file that is being written?

<?php
/**
 * A class to simplify log management. Easy to instantiate a logging object
 * which will take care to rotate logs when the current log becomes large
 * so we don't have to worry about logs growing without bound and using
 * up all the hard drive space.
 */

class Log {
	/**
	 * The maximum filesize (in bytes) permitted for a log.
	 * I have set it here to 100MB
	 * @var int
	 */
	const MAX_LOG_SIZE = 100000000;
	/**
	 * The number of old copies of the file we keep
	 * @var int
	 */
	const BACKUP_FILE_COUNT = 5;

/**
 * The path and filename of the log file
 * E.g., /path/to/file.log
 * @var string
 */
private $full_log_file_path = NULL;
/**
 * Path to the directory where these log files are getting saved
 * @var string
 */
private $log_dir = NULL;
/**
 * The basename of the log file path.
 * E.g., file.log
 * @var string
 */
private $basename = NULL;

/**
 * constructor, duh. creates an instance of this class
 * you can happily use for logging. This routine parses the provided log path
 * into parts in order to manage log rotation, etc.
 * @param string $full_log_file_path The path and filename of the main log file to be written
 */
public function __construct($full_log_file_path) {
	$this->full_log_file_path = $full_log_file_path;
	$path_parts = pathinfo($full_log_file_path);

	$this->log_dir = $path_parts["dirname"];
	$this->basename = $path_parts["basename"];

	// if no path was specified, assume CWD:
	if (strlen($this->log_dir) == 0){
		$this->log_dir = ".";
	}

	// validation / error checking
	if (!$this->basename || strlen($this->basename) == 0){
		throw new Exception("Unable to create Log object.  Basename is not valid");
	}

	clearstatcache();
	if (!is_writable($this->log_dir)){
		throw new Exception("Log constructor failed. Log directory (" . $this->log_dir . ") is not writable");
	}
}



/**
 * This function rotates the logs, deleting
 * one if we have reached our limit.  It's called
 * when our current log file has met or exceeded
 * self::MAX_LOG_SIZE
 */
function rotate_logs() {
	//delete the last log if it exists
	$last_log = $this->get_log_filename(self::BACKUP_FILE_COUNT);
	if (file_exists($last_log)) {
		if (!unlink($last_log)) {
			throw new Exception("Log::rotate_logs failed.  Unable to delete last log (" . $last_log . ")");
		}
	}

	// rotate the other backup logs
	for($i=self::BACKUP_FILE_COUNT; $i>1; $i--) {
		$old_log = $this->get_log_filename($i - 1);
		$new_log = $this->get_log_filename($i);
		if (file_exists($old_log)) {
			rename($old_log, $new_log);
		}
	}

	// rename the main log file to .1
	$current_log = $this->get_log_filename();
	$new_log = $this->get_log_filename(1);
	if (file_exists($current_log)) {
		rename($current_log, $new_log);
	}
}

/**
 * Returns a complete path to either the main log file
 * or, if optional argument $num is supplied, returns 
 * the full path to one of the numbered backup files
 * (e.g., /path/to/file.log.1)
 * @param int $num
 */
function get_log_filename($num=NULL) {
	if (is_null($num)) {
		return $this->full_log_file_path;
	} else {
		return $this->full_log_file_path . "." . $num;
	}
} // get_log_filename()

/**
 * This routine writes a message to the map log, taking care to
 * rotate it first if the resulting file would exceed the threshold size
 */
function write($msg) {
	clearstatcache();
	$msg_length = strlen($msg);
	if ($msg_length > self::MAX_LOG_SIZE) {
		throw new Exception("Unable to write log message to " . $this->full_log_file_path . " because the message length (" . $msg_length . ") exceeds the maximum log size (" . self::MAX_LOG_SIZE . ")");
	}
	if (file_exists($this->full_log_file_path)) {
		if (!is_writable($this->full_log_file_path)) {
			throw new Exception("Unable to write log file. " . $this->full_log_file_path . " is not writable");
		}
		$pending_file_size = filesize($this->full_log_file_path) + $msg_length;
		if ($pending_file_size > self::MAX_LOG_SIZE){
			$this->rotate_logs();
		}

	} elseif (!is_writable($this->log_dir)) {
			throw new Exception("Unable to create log file because directory " . $this->log . " is not writable");
	}
	file_put_contents($this->full_log_file_path, "[" . date("Y-m-d H:i:s") . "] - " . $msg . "\n", FILE_APPEND);
} // write()

} // class Log

    Maybe have a file in the log directory that you could use as a flag?

    // . . .
    
    private $flag;
    
    function __construct(){
    // . . .
        // get file handle for the log_access_flag
        // 'c' creates file if not exists (should only need happen once in all eternity)
        $this->flag = fopen( $this->log_dir.'/log_access_flag','c' );
        if( ! $this->flag ){
            thrown new Exception( "Log constructor failed. Unable to open/create file: {$this->log_dir}/log_access_flag" );
        }
    
    // . . .
    }
    
    function __destruct(){
        // explicitly release any locks (required since PHP 5.3.2)
        $this->log_lock( LOCK_UN );
    }
    
    function rotate_logs(){
        try{
            // put a write lock on the flag file so other processes won't try to write at the same time
            $this->log_lock( LOCK_EX );
    // . . .
    
    // . . .
            // revert to read lock when you're done
            $this->log_lock( LOCK_SH );
        }
        catch( Exception $e ){
            throw new Exception( "Log::rotate_logs() failed: ".$e->getMessage() );
        }
    }
    
    function write( $msg ){
        try{
            // put a write lock on the flag file so other processes won't try to write at the same time
            $this->log_lock( LOCK_EX );
    // . . .
    
    // . . .
            // revert to read lock when you're done
            $this->log_lock( LOCK_SH );
        }
        catch( Exception $e ){
            throw new Exception( "Log::write() failed: ".$e->getMessage() );
        }
    }
    
    /**
     * checks|gets log on flag file (creates file if it doesn't exist).
     *
     * @param int $mode    LOCK_SH (read lock; "registers" this process as using the logs), or
     *                     LOCK_EX (write lock; for writing or rotating logs), or
     *                     LOCK_UN (unlock; when you're done so other processes may do as they will)
     * @return bool        true if lock obtained; false otherwise
     */
    function log_lock( $mode=LOCK_SH ){
        // checks file handle
        if( ! is_resource( $this->flag ) ){
            // request lock on flag
            if( flock( $this->flag,$mode ) ){
                return true;
            }
        }
        // if we're here, everything went wrong
        throw new Exception( "Unable to lock log access file using mode:$mode." );
    }
    

    This is just a suggestion, off the top of my head. Completely untested.

    Disadvantage is that, if any given process hangs, it might prevent all others from accessing the logs too.

      Thanks, traq, for your thoughtful post. I'm pretty sure that structure that you've proposed is going to be more reliable than my code, but I wonder if it is completely reliable? My fear is that any process that is about to write to the log (or rotate the log files) might be interrupted and de-scheduled in between the lock check and the point at which the process actually attempts to write/rotate the log file. The problem with testing it is that the likelihood of failure is really remote -- even remoter if we can reduce the number of instructions between the lock check and the file-writing. I.e., it would be really hard to get the error condition to arise -- which is a good thing in that problems are infrequent but a bad thing in that it's hard to produce the error condition.

      I'm starting to think I might use MySQL to log these messages to a table. It's my understanding that MySQL is suitable for use in these multi-threaded situations.

        I honestly don't know. It's an untested idea. However, IF all of your processes are using this same class to write, they should all respect the file locks. By having both write() and log_rotate() get an exclusive lock, do their stuff, and then release it, it should create a queue of sorts among all the processes that are using it. I'd have to read up more on how "advisory file locking" works in PHP, but I would imagine it would work across processes (wouldn't be too useful if it didn't).

        On the other hand, yes, DB would be a good solution too. I'm going with SQLite as the default for the logging class I'm writing.

        edit

        BTW, just noticed a typo in my code above: in the log_lock() method, the check should read if( is_resource(...) ) - NOT if( ! is_resource(...) ).

          5 days later

          Thanks again for your thoughtful input. I ended up creating a log that writes to a DB table and I'll just rely on MySQL to deal with the concurrency. It seems to be working.

            If nothing else, it might help log searches. 🙂

              NogDog;11029901 wrote:

              If nothing else, it might help log searches. 🙂

              Agreed. I spent a significant amount of time contemplating what kind of structure to give my log table and ultimately just made it really simple.

              On the other hand, databases don't have a tail -f command where you can just watch the log filling up. I expect I'll concoct some kind of AJAX log-monitoring script.

                Write a Reply...