php get_file_contents help

speedamp

hi everybody.

build a log parser over the past few months, and wanted to know some recommendations on a how to read larger files?

i am currently using get_file_contents, and it works up to a certain size, but then i get a 500 error.

any help would be appreciated.

-michael

halojoy

what statement (php code, thanks)
do you use when you get this error?

also, as file_get_contents works with remote URL, too:
Are you using it internally or to load an external file?

dougal85

I'm assuming it is a local file you mean file get contents.

The problem with that function is it loads the full file into memory - so its quite inefficient since on large files you are unlikely to want to open the full thing.

You will probably want to use a combination of fopen, fseek... file functions can be seen here; http://uk2.php.net/manual/en/ref.filesystem.php

This example might help you too http://www.phpguru.org/article.php/265

It all depends exactly what you want to do. do you want to get the last few lines? edit the file?

You could even do use the unix tail command...

speedamp

thanks for the information.

file size are averaging "21940562" , and i've basically been using file_get_contents, exploding the contents line by line ("\n") and then writing each line to a db.

what would be a good substitute for this line (which would allow me to read larger files):

$logs = explode( "\n", file_get_contents($logfile));

-michael

dougal85

If you need to open the full file I don't imagine that there is anything more efficient. Why do you need to read the full thing in?

speedamp

how would you recommend me best looping through line by line ?

-michael

laserlight

I would use [man]file/man to read a file line by line, but I am not sure of how well it handles large files.