file transfer optimization

nemonoman

I built a little php script that runs as a cron job.

Script dumps the data from a MySQL db into a file. I then use FTP to push the file to a server at a different location.

Occasionally my users report a server slowdown, and I wonder if this script is part of the problem.

Anybody with some knowledge:

Is there a lot of overhead with a server-to-server FTP file push?

Would I get better performance by running a cron script on the OTHER server to read and copy the SQL dump file?

If so, is there any perfomance difference in using a basic file() type read or an FTP type?

Thanks.

Nemo

Jeb_

Originally posted by nemonoman
Would I get better performance by running a cron script on the OTHER server to read and copy the SQL dump file?

If so, is there any perfomance difference in using a basic file() type read or an FTP type?

For the last question, it depends on what you're reading. If you're reading a URL with file(), there's still the overhead of creating the connection, reading the data, etc. With FTP, there's always the overhead of creating the connection, logging in, reading, etc. etc.

Normally, a quickie FTP file push shouldn't cause enough overhead to worry about - how often is the cron job running?

nemonoman

Roughly 2.4 megs, every 2 hours.

Jeb_

I can see how that would cause a bit of slowdown.

The only way to optimise that transfer would be to compress the dump file. A good bz2 compress would cut the filesize down by quite a fair chunk.

There's always other options like rsync, but that's probably a bit overkill for transferring a dump file.

drawmack

I would suggest compressing the dump file, or even optimizing it so you only dump the entries that have been added/changed in the last two hours.