Cron - php - AJAX problem

[agentmanatee]

I see there's lots of activity on these boards, so I hope someone has an answer for this one.

I have a large file system that I'm trying to backup. for simplicity let's say 20k files in 2k folders.

I'd like to run a cron job to execute my php script that runs through this folder.

The problem is that this script takes forever and a day to execute because it has to wait for connections to the other server via FTP.

I'd like to be able to break this up and have the 1 cron job break up the folders into multiple ftp connections as the server can handle multiple connections at once.

I tried to embed an http request in the php script the cron is calling, but it ignores it. I assume this is because javascript is a client side language. This is the only way I can think of right now to essentially have the one script start a bunch of simultaneous connections.

Does anyone have any suggestions?

Thanks in advance

[etully]

You're using the wrong tool for the job. Sure, you can write a backup utility in PHP... but there's already about four gazillion tools out there that:

1. are free
2. are faster than your PHP script will be
3. already have all the bugs worked out
4. are more efficient than your PHP script will be

I suggest that you set up a cron job to run rsync. Rsync backs up the directory of directories. It only transfers files that have changed since the last backup. It compresses the files as it sends them to the backup server. It only transfers the portion of the file that changed. (For example, if you have a 10GB log file, and the only difference from last time is the last 3 lines of the file, it only transfers the last 3 lines of the file. You can configure it to not backup certain files or directories. And if you do it over ssh, the backup is encrypted so that it's secure.

Another solution would be to write a PHP script that will tar/gzip the 20K files in the 2K directories (using the system command), and then FTP the tarball to the other server. Then you could have a cron job at the other end that opens the tarball. Not even remotely as fast as the rsync solution... and not as secure... and it's pretty inefficient... but it's still going to be more robust than the solution you're attempting to develop.

[agentmanatee]

I do what I know - and is already configured on the servers 😛

I'll look into rsync though.

I found someone's "Threading" class - but I haven't had time to try it out yet. I'm still looking for a way to do this - even if it's not for backups.

Thank you though 🙂