Problem with large file uploads

optik

I have a pretty complex form which is using AJAX and APC for a progress bar, it all works fine for files up to a few hundred MB but in tests I'm doing now for files around 600MB it doesn't work.
I have all the PHP settings like max upload/script timeouts in php.ini and htaccess for file uploads very high so that shouldn't be a problem and I've also set limitbodyrequest in apache to be very high.

I know the file has uploaded because once the APC driven progress bar reaches 100% the page is forwarded to another page.

The form is sent via javascript when the upload commences so it seems that although the file is being uploaded something is preventing the form being processed directly after that as it should, maybe there is some other timeout going on I don't know about?

optik

The plot thickens, I have checked the logs and there is actually a harmless PHP error regarding the use of the date function for the page the form is being sent to. I know this error wouldn't prevent the script from working as the page has been tested on smaller files.

So the page for processing the form is being called and the file is being uploaded, yet there is no trace of there being any data actually making to the form processing page.

I have no idea what's up now 🙁

optik

It seems a problem has crept in somewhere else as smaller files aren't working either.

obhobh78

How do u manage to upload up to few hundred MB? APC keeps returning blank value even when i try to upload files around 70-80mb. Can u help me on this?

U can have a look at my server php.ini at kuanpress.com.my/test.php

Thanks.

optik

Yes I have it working now.

Your info file show max_file_uploads set to "20"

Apc seems to be enabled, you should also check the APC cache to see if an upload has been logged.

From my records I also had to update APC to get it to work and change some of the timeout settings.

yum install pcre-devel
pecl upgrade apc-3.1.2

chnage all 36000 to 720000 in apc_rfc1867.c

I warn you this was not easy to set up and took a quite a bit of configuration with APC, PHP and Apache.
But it does work well so don't let that put you off.

obhobh78

Sorry Optik...I don't get you really well there as I'm quite new to PHP and server settings myself.

Anyway, if you go to www.kuanpress.com.my/test.php ,
I]you would notice that I have set the following to 720000:

apc.gc_ttl to 720000
apc.ttl to 720000
apc.user_ttl to 72000

Is it correct and do I leave any settings behind? You mentioned about my max_file_uploads being set to 20. Is it correct or do I need to change it?

Would appreciate if your help in assisting me to solve my large file upload because i have been working on it for 2 long weeks...🙂

obhobh78

By the way Optik, did u encountered the IE timeout issue when uploading large files or you have found a workaround over it? I've google about it and seems like there is no solution except uploading through other browsers like Firefox.

optik

I've not had any problems with IE specifically.

obhobh78

That's weird...anyway, is my apc settings correct? :evilgrin:

obhobh78

perhaps my apache might be causing the problem too...do u mind to share with me what should i do with my apache setting as well?? Thanks.

optik

I did this 6 months ago so can't remember everything.

Basically what prevents APC working is either MAX upload limits or Timeouts.

This is what I have in my php.ini

[apc]
apc.enabled=1
apc.shm_segments=1
apc.optimization=0
apc.shm_size=128
apc.ttl=7200
apc.user_ttl=7200
apc.num_files_hint=1024
apc.mmap_file_mask=/tmp/apc.XXXXXX
apc.max_file_size = 3000M
apc.rfc1867 = on
apc.rfc1867_ttl = 72000000
apc.gc_ttl=72000000

You then also have to change all 36000 to 720000 in apc_rfc1867.c which is the source file for APC (at your own risk).

You can alter max upload settings in the .htaccess file of your site to be double sure like this

php_value upload_max_filesize 250M
php_value max_execution_time 8000000
php_value post_max_size 250M

max_file_uploads should be your limit setting so 20 wouldn't be enough.

You should be testing apc with small files first though if you haven't already.

optik

obhobh78;10950210 wrote:
Sorry Optik...I don't get you really well there as I'm quite new to PHP and server settings myself.

Anyway, if you go to www.kuanpress.com.my/test.php ,
I]you would notice that I have set the following to 720000:

apc.gc_ttl to 720000
apc.ttl to 720000
apc.user_ttl to 72000

Is it correct and do I leave any settings behind? You mentioned about my max_file_uploads being set to 20. Is it correct or do I need to change it?

Would appreciate if your help in assisting me to solve my large file upload because i have been working on it for 2 long weeks...🙂

I believe the problem lies in the source file as mentioned, for some reason the timeout is hard coded into this file despite you being able to change the settings in the php.ini file.

Weedpacket

max_upload_files shouldn't be confused with upload_max_filesize: the first one limits the number of files that can be attached to a single request; the second one limits the size of a single uploaded file.

The total maximum size of a single POST (unless limited by post_max_size) can therefore be as much as max_uploaded_files*upload_max_filesize + whatever other form data is being submitted along with the files.

obhobh78

Hi Optik,

You mentioned about changing all 36000 to 720000 in apc_rfc1867.c which is the source file for APC .

Actually how do i change it??

I have no problem following the SSH command to update APC but not this one. Help would be appreciated..

Many thanks. 🙂