Automatically transferring a file to another server...

moosemoose · Apr 10, 2007

Hi all,

I've been looking all around the internets for a solution to a little problem I'm having with my latest php based website, which just happens to be two other things too - my university dissertation, and a major pain in the head...

The problem is, once one script has uploaded, processed, and saved information to a database, the file needs to be moved to another server (one is fast at processing, the other has 2Tb free space) for the indefinite hosting.

I can't seem to do this with PHP's FTP functions - and I'm assuming this is because of the limitations of PHP's file handling ability.

Is there a possiblity to do this using exec() and Fedora's built in FTP programme, or something similar?

The filesizes can vary from around 10mb to maybe 300mb. Both servers are on fibre links so speed isn't a problem. I'd thought FTP through exec might work, but can't figure out if FC4 can accept a complete FTP function in one go.

If it helps, I've got root access to the main server where the scripts are hosted.

Any help is much appreciated :o

sneakyimp · Apr 11, 2007

Your post first mentions a database and then mentions files. I'm assuming that you're talking about uploading individual files.

If the files are being properly uploaded to the machine where the scripts live, then permissions on that machine are not the problem. The problem would be getting PHP to copy the file to some other machine. Permissions on the DESTINATION machine then are potentially the problem.

Any chance you have tried some code already? Any errors get returned?

Can you run scripts on the destination computer? What sort of access do you have to this computer? Just FTP or also command line/linux login?

moosemoose · Apr 11, 2007

This is what I was trying to do with the script:

User uploads file to [FONT="Courier New"]servera.com/uploads/filename.ext[/FONT]
Server A processes file, creates new file at [FONT="Courier New"]servera.com/files/finalfile.ext[/FONT]
Server A sends file from [FONT="Courier New"]servera.com/files/finalfile.ext[/FONT] to [FONT="Courier New"]serverb.com/files/finalfile.ext[/FONT]
User adds information which is added to database

This is the script I wrote in PHP:

$ftp_server = "www.server.net";
$ftp_user_name = "user@server.net";
$ftp_user_pass = "password";
$source_file = "/home/user/public_html/directory/$exefile.ext";
$destination_file = "$exefile.ext";

echo $conn_id = ftp_connect($ftp_server); 

echo $login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass); 

if ((!$conn_id) || (!$login_result)) { 
        echo "FTP connection has failed!";
        echo "Attempted to connect to $ftp_server for user $ftp_user_name"; 
        exit; 
    } else {
        echo "Connected to $ftp_server, for user $ftp_user_name";
    }

echo $upload = ftp_put($conn_id, $destination_file, $source_file, FTP_BINARY); 

if (!$upload) { 
        echo "FTP upload has failed!";
    } else {
        echo "Uploaded $source_file to $ftp_server as $destination_file";
    }

// close the FTP stream 
ftp_close($conn_id);

The other thing I'd tried was to just give the shell parameters using [FONT="Courier New"]exec();[/FONT], as follows:

$command ="ftp\ropen\rwww.serverb.net\ruser@serverb.net\rpassword\rsend\r/home/user/public_html/directory/$exefile.flv\r$exefile.flv\rexit\r";

// send command to linux console
$output = shell_exec($command." 2>&1);

echo "<pre>$output</pre>\n";

the [FONT="Courier New"]$output[/FONT] brings this:

sh: ftp
open
www.serverb.net
user@serverb.net
password
send
/home/user/public_html/directory/file.ext
file.ext
exit
: No such file or directory

This confuses me, because the exact same commands work fine in the FC4 shell.

sneakyimp · Apr 11, 2007

possible reasons:

1) It may be that your user/pass does not have permission on serverA to access the file it's trying to send. Have you tried manually connecting with FTP using that user/pass to serverA and downloading the file? If you login via FTP and cannot download files from the directory where you are putting all these files then you will need to choose a different directory or you will need to grant permission to that user.

2) In your script, you do this:

$source_file = "/home/user/public_html/directory/$exefile.ext";

you should add this line immediately after to make sure that is in fact a real file that exists.

if (!file_exists($source_file)) {
  die ('the source file does not exist!');
}

3) Lastly, there might be some issue with the destination file on the destination machine but try steps 1 and 2 first.

moosemoose · Apr 11, 2007

sneakyimp wrote:
possible reasons:

1) It may be that your user/pass does not have permission on serverA to access the file it's trying to send. Have you tried manually connecting with FTP using that user/pass to serverA and downloading the file? If you login via FTP and cannot download files from the directory where you are putting all these files then you will need to choose a different directory or you will need to grant permission to that user.

2) In your script, you do this:
$source_file = "/home/user/public_html/directory/$exefile.ext";
you should add this line immediately after to make sure that is in fact a real file that exists.
if (!file_exists($source_file)) {
  die ('the source file does not exist!');
}
3) Lastly, there might be some issue with the destination file on the destination machine but try steps 1 and 2 first.

I've ensured it does work using the FC4 shell commands - and it works fine - if I'm logged in and typing it into the shell. It doesn't work if I'm trying to do it using [FONT="Courier New"]exec()[/FONT]. Copying the test file works fine, and works fast. I'm needing to handle potentially large (200mb) files, but the shell command and the fibre links handle this fantastically.
I've added this in, and the script seems to be doing something. After almost five minutes "loading" the page using the test file (330kb), it gave the following message:

Resource id #21Connected to www.serverb.net, for user user@serverb.net
Warning: ftp_put() [function.ftp-put]: PORT command successful in /home/user/public_html/directory/ftp.php on line 36
FTP upload has failed!

line 36 is:

echo $upload = ftp_put($conn_id, $destination_file, $source_file, FTP_BINARY);

It also placed a 0kb file on serverb.

What sort of issue do you mean? If it transfers fine using the console, how can PHP not manage it? I'm not understanding :o

I'm thinking of caving in and creating a [FONT="Courier New"]cron[/FONT] to run every few minutes to transfer the contents of the directory. It would free up the user script, but it'd also increase waiting time - and if one file takes more than, let's say, 5min, could the cron script not try to transfer the same file twice, at the same time, and bottom out both servers?

Thanks for your input so far! It seems like there is a light at the end of the code.. :o

greenie2600 · Apr 11, 2007

fopen() has supported FTP since PHP 3. This would definitely be the easiest way to copy files to a remote server. I did this today; it took about three lines of code, and it worked fine.

Have you tried this? If so, are you getting a specific error when doing so?

moosemoose · Apr 11, 2007

I'm missing something here, or the lack of sleep is affecting my eyes, but I can't see how I'd use [FONT="Courier New"]fopen()[/FONT] to transfer a file from one server to a second...? : bemused:

greenie2600 · Apr 11, 2007

$file_handle = fopen( "ftp://username:password@some.ftpserver.com/myfile.txt", "w" );
fwrite( $file_handle, $file_contents );
fclose( $file_handle );

sneakyimp · Apr 12, 2007

It would also be helpful for you to post your script - i have no idea what is on line 36. be sure to use the

 tags around it so it gets formatted.

moosemoose · Apr 12, 2007

the full script now:

<?php
						$ftp_server = "www.serverb.net";
						$ftp_user_name = "user@serverb.net";
						$ftp_user_pass = "password";
						$source_file = "/home/user/public_html/directory/file.ext";
						if (!file_exists($source_file)) { 
						  die ('the source file does not exist!'); 
						}
						$destination_file = "file.ext";

					echo $conn_id = ftp_connect($ftp_server); 

					echo $login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass); 

					if ((!$conn_id) || (!$login_result)) { 
					        echo "FTP connection has failed!";
					        echo "Attempted to connect to $ftp_server for user $ftp_user_name"; 
					        exit; 
					    } else {
					        echo "Connected to $ftp_server, for user $ftp_user_name";
					    }

					echo $upload = ftp_put($conn_id, $destination_file, $source_file, FTP_BINARY); 

					if (!$upload) { 
					        echo "FTP upload has failed!";
					    } else {
					        echo "Uploaded $source_file to $ftp_server as $destination_file";
					    }

					// close the FTP stream 
					ftp_close($conn_id);

the error message:

Resource id #21Connected to www.serverb.net, for user user@serverb.net
Warning: ftp_put() [function.ftp-put]: PORT command successful in /home/serverb/public_html/dir/ftp.php on line 23
FTP upload has failed!

it also creates a file on the server, that's rather empty:

Automatically transferring a file to another server...

Mmoosemoose

Ssneakyimp

Mmoosemoose

Ssneakyimp

Mmoosemoose

Ggreenie2600

Mmoosemoose

Ggreenie2600

Ssneakyimp

Mmoosemoose