Flat File system

[gabriel]

I am trying to make a object/class that will send data from forms and other sources into a flatfile system. Basicly I'm asking for opinions on whether my system will work or will it eventually become too slow and hard to manage.

Example of the file that data is stored in:

1|some data2|some data3
2|some data2|some data3

Here is the code, its not complete but you might get an idea what I want to accomplish:

class ReadWrite
{
  var $_F = array();
  var $error = array();

  function ReadWrite($file)
  {
    $this->_F = $file;
    $this->_FileArray = file($file);
  }

// Does the actual writing to file
//

  function _write($quary)
  {
    if(is_array($quary))
      $quary = implode("",$quary);
    $output = $quary;
    $output = stripslashes($output);

$fp = fopen($this->_F, "w+");
if (flock($fp, LOCK_EX)) 
{ // do an exclusive lock
  if(!fwrite($fp, "$output"))
    $this->errors[] = "Unable to write file";
  flock($fp, LOCK_UN); // release the lock
  chmod($fthis->_F, 0777);
} else {
  $this->errors[] = "Couldn't lock the file !";
}
fclose($fp);

  return $output;
  }

  // Cleans the bad crap from users input
  //
  Function _Cleaner($quary)
  {
    if( is_array($quary) )
    {
	    while( list($k, $v) = each($quary) )
     	{
		    if( is_array($quary[$k]) )
		    {
			   while( list($k2, $v2) = each($quary[$k]) )
			   {
				  $quary[$k][$k2] = htmlspecialchars(trim($v2),ENT_QUOTES);
				  $quary[$k][$k2] = str_replace("|", "|",$v2);
			   }
			   @reset($quary[$k]);
		    }
		    else
		    {
			   $quary[$k] = htmlspecialchars(trim($v),ENT_QUOTES);
			   $quary[$k] = str_replace("|", "|",$v);
		    }
	   }
	     @reset($quary);
    }
    return $quary;
  }

  // Breaks the info store in a array into useable pieces
  //

  function _parts($value)
  {
    $value = addslashes(trim($value));
    $parts = explode("|",$value);
    $clean = $this->_Cleaner($parts);
    return $clean;
  }


  // Procosses Delete Data Command
  //

  function delete($delete)
  {
    foreach($this->_FileArray as $value)
    {
        $y = $this->_parts($value);

    if (!in_array($y[0],$delete))
    {
      $tmp[] = $value;
    }
}
$vars = $this->_write($tmp);
return $vars;
  }

}

########################################


$test = new ReadWrite("tEMP/links.txt");
$delete = array();
$delete[] = "one";
//die(print_r($delete));
print($test->delete($delete));
?>

[zabmilenko]

I recommend writing php scripts directly with your code:

$output = '';
$output .= "<?php \n";
$output .= 'class FlatFile33415 {' . "\n";
$output .= 'var $key1 = "'. addslashes($value1) .'";' . "\n";
$output .= 'var $key2 = "'. addslashes($value2) .'";' . "\n";
$output .= "}\n";
$output .= "?>";

You may have to do some extra exception catching (to prevent duplicate class errors) but you could simply include the class file and use it statically (fast!). If the class is included enough, any server-side caching you have will pick up on it and put it in shared memory.

Writes will be a little slower. There are better ways, but I like this one.

PHP alternatives include berkeley db and dbm extension.

[gabriel]

Thanks but its not quite what had in mind
The example:

1|some data2|some data3
2|some data2|some data3

Is showing how data looks in the TXT file after it was written from a form handler. I was wondering if the way data was stored apposed to using a DB would work and be efficient in the long run. The '|' are seperating the different fields of data and the records are seperated with a new line

[zabmilenko]

DB servers typically have better deadlock protection, smarter caching, and provide a more portable storage medium for data. Plus they are far more scalable than flat files.

That's "software" server, not a hardware one :-)

[MarkR]

Your locking code isn't sufficient. When reading the file, you make no attempt to lock it against concurrent modification, as you're using file() which applies no lock.

The correct way to lock in this situation is:

1. Open the file for reading AND writing
2. Obtain an exclusive lock
3. Read the file
4. rewind & truncate
5. write the file
6. close the file (which implicitly releases the lock)

Otherwise you could end up reading a file which is partially written by another process.

You could equally read a fully formed file, but have another process modify it in the interim, and overwrite those changes.

Mark

[gabriel]

Thank you Mark