file_get_contents timing out

DKY

I'm trying to import data from a flat file to my database using

$inv = file_get_contents($filename);

and it gives me a "page cannot be displayed error". I'm almost positive that this is because the flat file is so big. Is there a way around this? Should I adjust something in my php.ini?

blckspder

you can try to readjust the max execution time

ini_set(max_execution_time, 240);

Change the 240 to whatever you think will work. It goes by seconds so 240 should be 4 minutes.

DKY

already set to 240...

What about using fread()? Would it be better to code it so that it reads one line at a time and executes the insert query? If so, how would I code that?

dagon

how big is it? it could be an allocated memory issue, you may need to process it in chunks

DKY

320 MB, how do I process it in chunks? What would code look like for that?

blckspder

I just noticed you said in your previous post that you get a page cannot be displayed error. Typically in my experience I either get PHP errors or a blank page if error reporting is not turned on. Not a page cannot be displayed error.. do you have error reporting on in your php configuration?

DKY

How do I make sure that I have error reporting on? In my php.ini my "display_errors" is "On".

blckspder

Yes it should be on in the php.ini file, and you should probably set it to report all to make sure you are getting notices and fatal errors until you are done developing. and solving this problem...

I still however do not know why you are getting page cannot be displayed error.. This sounds more like a permissions issue of some sort rather than a coding problem.

DKY

So change "display_errors" from "On" to "All"?

blckspder

What version of PHP are you running? and do you have other php pages that work? Also check the file permissions for the page that is getting the display error and make sure your web server has access.

DKY

I am running PHP Version 5.2.9
I have another document that does exactly the same thing but with a smaller file. It runs successfully.
This web page does everything else that I'm asking of it until I put in the line
```
$inv = file_get_contents($filename); 
```
it's at that point that I get the "Page cannot be displayed" error after about 240 seconds (give or take).

It's the fact that the other document (referenced in point number 2 above) works with no problems and that it's much smaller that leads me to believe that it's timing out because the file is too big. I don't know for sure but am explaining my reasoning. Maybe I need to look into processing it in chunks like Dagon suggested but don't know how to do that...

blckspder

Ah ok.

http://www.ibm.com/developerworks/library/os-php-readfiles/

check out the above link it will help you to figure out how to read the file in chunks and that will hopefully solve your problem.

DKY

Okay cool. I'm able to get the first file to work which reads a flatfile and puts in the partnumber and a timestamp (amongst other things) and now I'm trying to use the same method of fgets to update the database with other information where the partnumber in the database is equal to the partnumber in the second flat file and I'm getting the following error:

Fatal error: Maximum execution time of 480 seconds exceeded in C:\xampp\htdocs\ScheduledTasks\DTLRPTB_SAS-MySQL.php on line 22

line 22 is this:

if (!($upload_result = mysql_query($upload_query)))

Here's my code

<?[
require("connect.php"); 
$filename = "C:/xampp/htdocs/ScheduledTasks/Downloads/DTLRPTB.txt";

$connectmysql = mysql_connect($host,$user,$password) or die ($cantconnectmysqlmessage); 
$db = mysql_select_db($database,$connectmysql) or die ($cantconnectdatabasemessage);

// Central Standard Time	 
date_default_timezone_set('America/Chicago');
$timenow = date("Y-m-d H:i:s");
// Central Standard Time

/*--- READING IN THE FILE ---*/  

$file_handle = fopen($filename, "r");
while (!feof($file_handle)) {
   $line = fgets($file_handle);

IF(TRIM(SUBSTR($line,0,25))<>""){$PARTNUMB = TRIM(SUBSTR($line,0,25));}ELSE{$PARTNUMB = "";}

   $upload_query = "UPDATE $DetailReportATable SET TIMESTAMPB = '". $timenow ."' WHERE PARTNUMB = '$PARTNUMB'";

if (!($upload_result = mysql_query($upload_query)))
	{
		$mcrmessage = mysql_error(); 
		echo "$mcrmessage<br>"; 
		die();			 
	}
}
fclose($file_handle);
?>

It seems to add some data to my database but doesn't finish and gives me this error....