[Resolved] PHP "dying" in middle of process

kangaroo232002

I'm making a search engine for my final year project at uni with PHP, and at the moment im working on the trawler. Its fine for say 20 minutes, but when i go and leave it trawling (this is a mixture of using PHP to identify new links and text, which are then put into the MySQL database) it stops without any error - so i guess no timeout - it just stops (hence dying in quotes). This has never happened when I'm actually at the computer ( the script runs on localhost, Apache v2.0.35 on WinXP), just when I leave it plodding. It can be quite intensive on the CPU, when uploading data into the mysql db: would it be recommended to put a sleep() in there somewhere? Alternatively, any suggestions as to what is happening, or are there any "Keep Alive" functions that can be used... thanks in advance - i'm running PHP v4.3.3, and not sure if its a php thing, apache, or mysql (etc etc)...

ian

weekender

read up on [man]set_time_limit[/man] and [man]ignore_user_abort[/man]

i did my uni project in php too. would be interested to see yours when it's done

adam

kangaroo232002

yea, i know about set_time_limit(), but if the script times out, you get a php error - max execution time and all that malarky,... but literally, the script justs stops without any error, nothing !

all very weird. Will read up on the user_abort() anyway.

quite an old version of the trawler is on http://essence.nxm-uk.com/fsockopen.php.
(without all naive bayes for classifying pages... and a bunch of other stuff that makes it work properly!)
The name is a bit of a misnomer as it actually uses fopen instead (I was feeling pain with all headers stuff!)

thanks for the reply.

[moderator's note: used the edit button and fixed the URL]

jayant

recent versions of php have the default setting which doesnt display errors for security reasons.
you can change the behaviour from within the script/.htaccess/php.ini

Weedpacket

Does your trawler do any logging so's you can have a record of what it was doing when it stopped? I don't know how your trawler decides which links to follow, so coming at this cold I'd consider the possibility that it reckons that it simply ran out of work!

kangaroo232002

i suspect that it has something to do with the fopen timeout ( as i was using the default timeout of 60 seconds). I've changed it to 10 in php.ini so i will see if that improves the situation! Thanks for your help anyway (incidentally, the error messages are enabled) but i will keep on plugging.