Get the progress of a long "server to server" transfer?

VMX

Hi there. This is my first post here, so hello everybody 😃

Now my question:

I've coded a really simple PHP script that will basically pick a remote file:

$file="http://www.somedomain.com/somefile.jpg";

and then copy it over to a folder in the server where the script is:

$data = file_get_contents($file);
$fileHandler = fopen("folder/filename.jpg", "w+");
fwrite($fileHandler, $data);
fclose($fileHandler);

This works ok to transfer small files (images, etc.), and will work with bigger files depending on the PHP parameters in my server (max_memory_size, etc.).

However, even if it does work for bigger files, all you get in the browser is a white page, with a "Loading..." indication in the browser status bar. I guess I could make it display something like "Transferring the file..." in the browsers page, but still, it's not enough info for really loooong transfers (I'm speaking of hundreds of M😎.

So my question is: is there ANY way to periodically get the transferred bytes or something like that, so that I can give the user an estimation of the operation progress?

Maybe there's some way to trace the amount of transferred data by the "file_get_contents()" function?

Thanks in advance.

scrupul0us

you could place an iframe on the page that refreshes some PHP code that gets the file size of the file you are coping over every 'x' seconds

VMX

Thanks for your reply.

But there's a problem: the destination file only appears AFTER it has been totally downloaded.

If you look at the code, the download occurs when I run:

$data = file_get_contents($file);

But no file is created yet.

At that moment, the data of the original file begins to be stored in the variable $data, this is, in MEMORY, but not in the hard drive of the server. If the file is, say, 100mb big, I can start transferring it and nothing will happen. I can check the destination folder and there will be nothing there, it'll be empty. And then, after like 10 minutes, when the file has been fully downloaded, it is created in the destination folder and the data is copied over there in a second. I could also create it at the beginning of the script, but still, it would be 0 bytes big until the download has finished.

So what I would need to do is to somehow periodically check the "size" of the $data variable, which is something that I don't know how to do, nor if it's even possible to do.

Any ideas?

NogDog

within the confines of a PHP-only solution, all I can think of is doing something clever with [man]declare/man and "ticks", but I've not worked with it before so can only point you in that direction.

VMX

I wouldn't mind using some not-only-PHP solutions. I have a good experience with Flash + PHP for instance. The closer I could get to solve this was:

Make a Flash interface that first calls de PHP script which starts the transfer from server to server.
Once the transfer has been started, make the Flash movie periodically call another PHP file, which checks if the destination file exists.

Obviously, while the transfer is going on the file won't exist yet, and it will return 'false'. So I can tell the user to keep waiting. Then, if the transfer is completed at some point, the PHP script will return 'true' when it checks for the existence of the destination file, and then the Flash movie would tell the user that it's done.

This doesn't give the user any idea of the progress, but at least it does indicate when it's finished, which is more than what the PHP script does.

However, there's a problem with this too:

If the transfer fails at the very beginning, the PHP file will return an error message but the file will be created (being 0 bytes in size), which would mistake the user by telling him/her that it's done. I could code the PHP in such a way that it only creates the file if the download succeeds, but then, as it's never created, the user would be told to wait forever.

So basically I need to 'catch' those errors that the PHP file throws so that I could inform the user of the failure. My new question is: is there ANY way to do this?

As far as I know, Flash can only catch variables from the outside if they're sent in the form:

&variable1=value1&variable2=value2&

But PHP returns errors in the form:

Warning: file_get_contents(http://www ... in /mounted-storage/home94b/sub008/sc47438-UGYH/ ... on line 11

Can you think of any way to catch the error messages that the PHP file returns, and add something like this at the beginning of the string? :

&errors=

then send them to the flash movie?

Then I would search for different error strings in the "errors" variable and inform the user accordingly.

But how could I do that? Maybe running the PHP from another PHP? Like:

$errors=file_get_contents("original_php_script.php");
echo ("&errors=".$errors);

Is this possible? Or would this return the actual contents of the PHP file itself, rather than run it?

Any help would me much appreciated...
Thanks!

cbj4074

I'm guilty of skimming, but if you're open to solutions that leverage more than just PHP, perhaps see:

File upload progress bars with PHP — Martin Jansen
http://martinjansen.com/text/24092296/

Let us know how that works (if you try it)!

bradgrafelman

VMX;10877694 wrote:
Thanks for your reply.

But there's a problem: the destination file only appears AFTER it has been totally downloaded.

If you look at the code, the download occurs when I run:
$data = file_get_contents($file);
But no file is created yet.

So then don't use file_get_contents(). Download it in chunks and write each chunk as soon as you download it. Assuming you don't flock() the destination file, you should be able to use a PHP script to periodically examine the destination file size and compare it with the known file size.

Here's an example of what I'm talking about:

$url = 'http://somesite.com/download_me.zip';
$dest = 'path/to/destination.zip';

$size = get_headers($url, 1);
$size = $size['Content-Length'];

exec("php download_script.php $url $dest &");

while(($dest_size = filesize($dest)) < $size) {

echo "Please wait... ($size of $dest_size downloaded)<br>";
sleep(5); // wait 5 seconds and check again

clearstatcache(); // make sure PHP actually checks dest. file size
}

Basically, you make some script (download_script.php) that accepts two command-line arguments (source and destination, accessible via $argv for example) and does what I mentioned - reads the source in chunks and writes the chunks to disk. The ampersand in the exec() line should fork this process into the background (on a Unix server, anyway) so that it runs independently of the script that is actually outputting to the user's browser.

There's some failsafes you should code for as well, such as errors in retrieving the source file (in which the while() loop I have above would become an indefinite loop); this situation could be solved by making sure the destination file has a few bytes written to it after the first iteration or two of the while() loop.