[RESOLVED] 7zip, called via proc_open(), can create ZIP files but cannot update them

[cbj4074]

Hello,

I am using proc_open() to execute a 7zip command that creates a ZIP archive (or adds to it if the destination ZIP file already exists).

If the ZIP file does not already exist, it is created without issue. However, if the file does exist, when 7zip attempts to add the new file to the existing archive, the command fails with exit code 2 (fatal error).

Here is the input:

7z a -r -mmt -mx0 -y -tzip "/var/www/example.com/tmp/19I1m4.zip" "/var/www/example.com/tmp/qgy546/Directory to be Zipped/"

And the output:

7-Zip 9.04 beta  Copyright (c) 1999-2009 Igor Pavlov  2009-05-30
p7zip Version 9.04 (locale=C,Utf16=off,HugeFiles=on,16 CPUs)

Scanning

Updating archive /var/www/example.com/tmp/19I1m4.zip


Error:
Can not open file
19I1m4.zip.tmp
Permission denied                


System error:
E_FAIL

This seems to be a classic permissions problem. But I'm a bit confused as to why PHP is able to create the ZIP file but not modify it.

It's clear from the 7zip output that 7zip is attempting to create a temporary file with the same name and a ".tmp" extension, and this seems to be where the failure occurs. Given that the parent directory has 0777 permissions, shouldn't the www-data user be permitted to create files there? The same user just created the original ZIP file in the same location without issue.

Just to debug the issue, I set the permissions on the directory to which the ZIP file is written to 0777. Here is the ls -lah output:

drwxrwxrwx  5 web5     client1  4.0K Nov  7 22:07 .
drwxr-x--- 12 web5     client1  4.0K Nov  7 21:43 ..
-rwxrwxrwx  1 www-data www-data 7.3M Nov  7 22:07 19I1m4.zip
drwxr-xr-x  3 www-data www-data 4.0K Nov  7 22:07 qgy546

I also added a call to set the permissions on the ZIP file to 777 after it is created: chmod($zip->dest, octdec(777)).

To say a bit more about the users and groups in the output above, PHP runs via Apache's mod_php, and hence, shell commands are executed as the www-data user, who is a member of the client1 group.

I know that the application logic is sound because this works as expected on Windows if Apache is running with Administrator permissions.

Does anything jump-out at anybody? Any help is very much appreciated.

[Ashley_Sheridan]

Maybe 7zip is attempting to create the temporary file in a different location? On an aside, is there any reason why you are using 7zip to do this instead of one of the zip libraries in PHP, or the standard zip tool on the command line in Linux (I'm assuming Linux because of the types of permissions you're listing)

[cbj4074]

Hi, Ashley, and thank you for the reply.

That's a good point; it didn't occur to me that 7z may be trying to create the temporary file in a different location; it would be nice if 7z displayed the full file path in the error output.

I'm using 7z on the command-line over one of the many pure-PHP implementations primarily due to memory consumption. My experience with pure-PHP ZIP classes has been that they require an amount of memory that is at least 2x the file-to-be-zipped's size. This is because in order to validate the ZIP file, the script must call get_file_contents() and then crc32(); each function requires an amount of RAM equal to the file's size, and the memory held by get_file_contents() cannot be freed until after crc32() is called. I need to ZIP files up to nearly 2GB in size, and a script that requires 4GB+ of memory to do so is out of the question. If you know of a pure-PHP implementation that does not suffer from this limitation (and does not fall-back to a "file streaming mode" that uses very little memory but requires swap space on disk and is painfully slow), then please do share.

I'm not using Linux's stock zip utility because I need for the same code to function identically across Windows, Mac OS, and Linux, and 7z is available for all three platforms.

Beyond that, 7z is a more robust tool, as it allows me to compress to a wide variety of formats, choose from different compression algorithms, etc; it also takes advantage of multiple CPUs or cores.

So, I guess the next step is to determine with certainty to what directory 7z is attempting to write when it creates that .tmp file.

I tried mimicking PHP's behavior as closely as possible by su-ing to the www-data user and issuing the 7z command. Not surprisingly, this works as expected, and the files are added to the existing archive.

[Derokorian]

cbj4074;11016997 wrote:

I'm using 7z on the command-line over one of the many pure-PHP implementations primarily due to memory consumption. My experience with pure-PHP ZIP classes has been that they require an amount of memory that is at least 2x the file-to-be-zipped's size. This is because in order to validate the ZIP file, the script must call get_file_contents() and then crc32(); each function requires an amount of RAM equal to the file's size, and the memory held by get_file_contents() cannot be freed until after crc32() is called.

I'm not sure what classes you've used, but I have never had memory problems using the ZIPArchive class built into PHP. In fact I use it on all my sites to backup the DB, uploads and code base once a week to an external FTP.

As far as your issue, have you tried using the -w option to force 7z to use a defined directory (as seen http://docs.bugaco.com/7zip/MANUAL/switches/working_dir.htm)

[cbj4074]

Thanks for the -w switch suggestion; that's a good idea. I did see that documentation, but I didn't try that because the manual states:

By default, 7-Zip builds a new base archive file in the same directory as the old base archive file.

Even so, perhaps 7z is behaving differently across platforms. I'll give that a try next.

[bradgrafelman]

I was going to ask about the CWD as well. For example, where is the PHP script located that executes the [man]proc_open/man call, and what are you passing for the fourth parameter (if anything)?

[cbj4074]

Thank you, Ashley, and Derokorian! You're both heroes! Using the -w switch solved the problem.

It seems that you were exactly right, Ashley; 7z was attempting to use some other temporary/working directory.

The help is deeply appreciated, as is the experience/advice regarding pure-PHP zipping implementations. Some day, I may revisit that option.

Thanks again! 😃

P.S. bradgrafelman, I just saw your reply; the PHP script is located in the vhost's document root. I was not passing a CWD to proc_open(). Given that the -w switch solved the problem, I hate to consume any more of your time. Thanks for the help!