How to handle a script that runs a really long time

schwim

Hi there everyone!

I'm trying to adapt a sitemap generator to one of my sites and I'm having an issue with the script exceeding PHP's execution cap. I understand that I could change this setting in php.ini but I'd rather handle it in another manner and want to run some ideas by the guys and gals in the know to see if I'm overlooking anything.

My first thought was to modify the script to run in batches. Looking at the script, it's a little confounding.

Secondly, trying to run it as a cron job. Does doing it this way allow me to circumvent the execution cap?

Would any of these methods appeal to you or would you do something completely different?

To better explain the situation, the site in question will have a ton of links that change and are added to often so it's something I need to do weekly. The script I'm trying to adapt is this one, but I'm not married to it and would be happy to switch to something else if there's something better.

Thanks for your time!

Weedpacket

[man]set_time_limit[/man]? Note that there is no timeout restriction for CLI-run scripts.

Me, I generally keep the work to be done in a database (SQLite would do), so that I can do things like add more work, interrupt and resume it later, or have several scripts each doing one section of work.

schwim

Weedpacket;11056379 wrote:
Me, I generally keep the work to be done in a database (SQLite would do), so that I can do things like add more work, interrupt and resume it later, or have several scripts each doing one section of work.

Hi there Weedpacket, and thanks very much for your help. Do you mean that you use the database to store the work that you have a cron job go through to determine what gets done? I'm planning on modifying the sitemap script so that I can keep track of last run and start/stop times so I can make sure that it's completing.

NogDog

Running it as a CLI script via cron seems like a reasonable solution if it does what you need. Maybe just set it up to run at 3:00am every morning (or whenever usage is low), and that way your sitemap is never more than 24 hours out of sync.