Repository navigation
macOS and files over 4GB in Finder-created zips #431
Description
Hey Nathan,
Looking at Issue #5 (#5), it seems that the patch didn't make it into the more recent versions (I'm testing with 2.9.0), as the example minizip program is not able to fully extract a zip with large files in it?
The test zip has just 3 files inside - 1-Small, 2-LargeOver4GB, 3-Small - after extracting the second file, it tries to extract the third one, but fails at
else if ((local) && (magic != MZ_ZIP_MAGIC_LOCALHEADER))
err = MZ_FORMAT_ERROR;
which is at mz_zip.c:218
Any suggestions on how to make the library accept macOS-originating Zip files?
Thanks,
Vasyl
Activity
nmoinvaz commentedon Oct 15, 2019
Do you have an example ZIP file?
supersv commentedon Oct 15, 2019
Sure, here's a zip file with a 5 GB random file (and two small ones) inside, created with Finder:
https://www.dropbox.com/s/lz2t58ai6gaadlt/MacZipDemo-dev-urandom.zip?dl=0
Interestingly enough, if the 5 GB file is all zeroes (and gets compressed nicely, to just 5 MB), the extraction works well; sample file:
https://www.dropbox.com/s/m1sa5avgihm7ssh/MacZipDemo-dev-zero.zip?dl=0
Fixed extract 4gb zip files without zip64 via recovery method. #431
Fixed extract 4gb zip files without zip64 via recovery method. #431
nmoinvaz commentedon Oct 20, 2019
I have made some changes that should fix it in
devbranch. It will attempt to recover the zip entries in a 4gb zip file that contains no zip64 information.supersv commentedon Oct 21, 2019
Thanks for the patch. I've tried recovering the first sample file linked above, with minizip built from
dev, but with the same error -103 as the result.nmoinvaz commentedon Oct 21, 2019
I tested against this file.
nmoinvaz commentedon Oct 21, 2019
It is working fine for me even with your test file. Not getting -103. I suggest you try and debug the source code and see what the issue is on your end.
More robust recovery method, seek to next local header and seek backw…
supersv commentedon Oct 22, 2019
Must have been an error on my side - now extraction works on both Mac and Windows. Thanks again.
supersv commentedon Oct 23, 2019
Found another problem. I'm testing on a zip archive with lots of files inside, among them a single large (> 4 GB) file, created with Finder.
Situation:
The failure happens in
mz_zip_entry_read_close(), on thezip->entry_crc32 != zip->file_info.crccheck.I can't share the test zip file in question, but if I manage to create a sample one, I'll post it. This is on a Mac,
cmake -G Xcode .and using the resultingminizip -x -d <...> <...zip>.supersv commentedon Oct 23, 2019
OK, here's a sample zip file - https://www.dropbox.com/s/pjb2v6s4ltblzu3/protobuf.zip?dl=0
Basically a clone of https://github.com/protocolbuffers/protobuf with the 5 GB
...-dev-urandom.zipfile placed insideprotobuf/cmakeand compressed with Finder.Fails at
...Extracting protobuf/benchmarks/util/__init__.pyError -105 saving entries to disk /.../.../protobuf.zipTested with the current
HEADofdev(75741f4).Coeur commentedon Oct 25, 2019
@supersv do you think that you could come up with a fix?
supersv commentedon Oct 25, 2019
@Coeur Let's hear Nathan's verdict first.
nmoinvaz commentedon Oct 30, 2019
macOS archive utility doesn't write zip files that are 4gb properly with ZIP64 extension. This is a long time known mac bug.
Coeur commentedon Nov 4, 2019
@nmoinvaz in that case, I guess I don't need to wait for a fix for that. Do you know how close we are from a minizip 2.9.1 release? I plan to update minizip once you'll be making a release.
nmoinvaz commentedon Nov 4, 2019
I was hoping to get to look at these two issues that are open before I do. Maybe I can get to it by next week. I'm currently working on incorporating code coverage into the build system.
nmoinvaz commentedon Nov 11, 2019
@supersv the zip file is no longer available..
supersv commentedon Nov 11, 2019
@nmoinvaz Try this link: https://www.dropbox.com/sh/me9lzpe1lbnpb8u/AAClsrIrjw4QWJxNf8zx1UsFa?dl=0
Fixed issues finding data descriptor when previous compressed size is…
nmoinvaz commentedon Nov 12, 2019
I think I have made a change that could possibly fix it so please test. But the problem is that your zip file contains zip files which make it hard to determine where some entries start and end.
supersv commentedon Nov 14, 2019
@nmoinvaz Seems to work like a charm. Thanks and congrats!