CRON jobs processing large files

mfacer

I run a few CRON jobs to back up mySQL databases - nothing too intense... but I now need to run one which has to update over 19,000 records in a table... this will be run once every week - If I run the script in a browser window it causes a time out as it takes so long to process..... would this happen in a CRON situation??

the only way I could get this to work in the browser was to do 500 records at a time and get it to page through the process.....

thanks for any info 🙂

devinemke

if your script exceeds the max_execution_time specified in your php.ini then it will time out regardless of whether it is run via cron or via browser.

mfacer

aah bugger! I had one method which worked in FireFox where I processed 500 rows, then output to the browser a META REFRESH which then went to the next 500 records.... would that work in a CRON job?

devinemke

mfacer wrote:
aah bugger! I had one method which worked in FireFox where I processed 500 rows, then output to the browser a META REFRESH which then went to the next 500 records.... would that work in a CRON job?

no, as there is nothing to refresh. if you have access to your php.ini simply change the max_execution_time. if you don't, you can achieve the same thing via [man]ini_set[/man]. or you could split your cronjob up into smaller ones that run successively.

jara06

yeah, you either have to increase max execution time or split the job into smaller jobs..
remember to make some kind of check whether the script is running atm.

Eg:

if (fileexists($lockfile)) {
writetolog("jobsrunninginadvance");
die();
}
else {
writefile($lockfile);
// do job
deletefile($lockfilfe);
writetolog("jobdone");
}