AES + lzma.zip = -105 (MZ_CRC_ERROR) during unzip

I did compile minizip lib and app under Ubuntu Linux,
creating encrypted archive with cmd:
./minizip -9 -p test -s -m test.zip test_files/

Minizip 2.3.4 - https://github.com/nmoinvaz/minizip
---------------------------------------------------
Creating test.zip
Adding: files
Adding: files/aa2f9b0c6d77807168de5b6354db825ecd644dae
Adding: files/569c51484a2b0d735d4650a2ad5f30ed8671628c
Adding: files/0186f31ace705d241c3d9c15dcb0bbe2c3ed23d2
Adding: files/792c5132370c133764fa3b5f1df1400edaed665a
Adding: files/2e57ba74a620da726713d19a50bfa6a66be053fa
Adding: files/cc9105dbbbbb87a4bf77d22a240c926f42adc3c9
Adding: files/bceadacd8252ad296ec0d34f41b5807fba078209
Adding: files/093a45321038528a04c47a5042094e91d46e7610
Adding: files/5785ea06b012c35b6ffb3b4c7f480d435ac0d7a6
Adding: reports
Adding: reports/match.txt
Adding: reports/sysinfo_agile-pc_5b3e5ca2_0.sz
Adding: reports/bulk_extractor_out_test.txt
Adding: reports/bulk_extractor_out_test
Adding: reports/bulk_extractor_out_test/lightgrep_histogram.txt
Adding: reports/bulk_extractor_out_test/alerts.txt
Adding: reports/bulk_extractor_out_test/lightgrep.txt
Adding: reports/bulk_extractor_out_test/report.xml

unzipping it always failing with error -105.
./minizip -x -d . -p test -s -m test.zip

Minizip 2.3.4 - https://github.com/nmoinvaz/minizip
---------------------------------------------------
Creating directory: ./files/
 Extracting: ./files/aa2f9b0c6d77807168de5b6354db825ecd644dae
 Extracting: ./files/569c51484a2b0d735d4650a2ad5f30ed8671628c
 Extracting: ./files/0186f31ace705d241c3d9c15dcb0bbe2c3ed23d2
 Extracting: ./files/792c5132370c133764fa3b5f1df1400edaed665a
 Extracting: ./files/2e57ba74a620da726713d19a50bfa6a66be053fa
 Extracting: ./files/cc9105dbbbbb87a4bf77d22a240c926f42adc3c9
 Extracting: ./files/bceadacd8252ad296ec0d34f41b5807fba078209
 Extracting: ./files/093a45321038528a04c47a5042094e91d46e7610
 Extracting: ./files/5785ea06b012c35b6ffb3b4c7f480d435ac0d7a6
Creating directory: ./reports/
 Extracting: ./reports/match.txt
 Extracting: ./reports/sysinfo_agile-pc_5b3e5ca2_0.sz
 Extracting: ./reports/bulk_extractor_out_test.txt
Creating directory: ./reports/bulk_extractor_out_test/
 Extracting: ./reports/bulk_extractor_out_test/lightgrep_histogram.txt
 Extracting: ./reports/bulk_extractor_out_test/alerts.txt
Error -105 closing entry in zip file

test.log

Can you attach the files that caused this or do u have a minimally reproducible file set? Thanks.

https://ufile.io/s5kao

I think this is fixed with #275 dev branch.

still reproducible with dev-branch.

Don't finalize compression stream if nothing written. #277

Ok should be fixed now.

fixed.

Finialize compression stream even if nothing written due to compatibi…

I'm still experiencing this issue with encrypted, zero-length (uncompressed size) entries copied over in raw mode, and also with freshly created (non-raw) encrypted entries when using DEFLATE compression.

7za is reporting this:

ERROR: CRC Failed in encrypted file. Wrong password?

Any hints where to look?

Self contained example for the DEFLATE case (I'm still investigating the LZMA raw mode one):

// clang test0.c -I../3rd/minizip2 -L../../../lib/darwin/clang -lminizip -lbz2 -lz

#include <string.h>
#include <stdio.h>
#include <stdlib.h>

#include "mz.h"
#include "mz_zip.h"
#include "mz_compat.h"

int main(void)
{
    static const char* names[] = { "test_deflate_32.zip", "test_deflate_64.zip", "test_lzma_32.zip", "test_lzma_64.zip" };
    int tmp;

    printf("ver: %s\n", MZ_VERSION);

    // create two .zip files, one regular and one with ZIP64 enabled, with an entry having zero length
    for(tmp = 0; tmp < 4; tmp++)
    {
        static const unsigned char content0[] = "";

        int zip64;
        int compr;

        zipFile hZip;
        zip_fileinfo zfi;

        switch(tmp)
        {
        case 0:
           zip64 = 0;
           compr = MZ_COMPRESS_METHOD_DEFLATE;  // FAIL
           break;
        case 1:
           zip64 = 1;
           compr = MZ_COMPRESS_METHOD_DEFLATE;  // FAIL
           break;
        case 2:
           zip64 = 0;
           compr = MZ_COMPRESS_METHOD_LZMA;  // OK
           break;
        case 3:
           zip64 = 1;
           compr = MZ_COMPRESS_METHOD_LZMA;  // OK
           break;
        }

        printf("zcreat: %p\n", (void *)(hZip = zipOpen2_64(names[tmp], APPEND_STATUS_CREATE, NULL, NULL)));

        memset(&zfi, 0, sizeof(zfi));
        printf("zaddfi: %d\n", zipOpenNewFileInZip5(hZip, "zero_length.txt", &zfi, NULL, 0, NULL, 0, NULL,
                                                    compr, 6, 0,
                                                    -MAX_WBITS, DEF_MEM_LEVEL, 0,
                                                    "password", 0x00000000 /* crc of 'content0' */,
                                                    0, 0, zip64));
        printf("zwrite: %d\n", zipWriteInFileInZip(hZip, content0, 0));
        printf("zfclos: %d\n", zipCloseFileInZip(hZip));
        printf("zclose: %d\n", zipClose(hZip, NULL));
    }

    return 0;
}

Make sure you are still reading the zip entry contents (calling unzipRead..) even if the content length is zero.

On zero-byte entries a deflate header is added some zip programs throw error when compressed size is 0 and compression method is DEFLATE. If you know ahead of time the file is going to be zero byte you can use compression method RAW.

The raw problem is caused by calling zipCloseFileInZipRaw64() with zero length and zero CRC, but these values are handled as a special case downstream in mz_zip_entry_close_raw(), triggering a mz_stream_crc32_get_value() call to recalculate the CRC32, resulting in an incorrect value in this case. Then the code also recalculates uncompressed_size by querying zip->crc32_stream, again resulting in a non-zero value.

Yes, that's a check, I do read the source zip entry in case of zero-length. The problem happens when closing the entry (as described above in #277 (comment)).

This patch fixes my test cases:

diff --git a/mz_compat.c b/mz_compat.c
index 5dc22082c2..91ea67761e 100644
--- a/mz_compat.c
+++ b/mz_compat.c
@@ -166,6 +166,8 @@ extern int ZEXPORT zipOpenNewFileInZip5(zipFile file, const char *filename, cons
         file_info.aes_version = MZ_AES_VERSION;
 #endif
 
+    file_info.raw = raw;
+
     if (raw)
         level = 0;
 
diff --git a/mz_zip.c b/mz_zip.c
index ab083c3232..a6ba1ede51 100644
--- a/mz_zip.c
+++ b/mz_zip.c
@@ -1430,7 +1430,7 @@ extern int32_t mz_zip_entry_close_raw(void *handle, uint64_t uncompressed_size,
         return MZ_PARAM_ERROR;
 
     mz_stream_close(zip->compress_stream);
-    if (crc32 == 0)
+    if ((crc32 == 0) && (zip->file_info.raw == 0))
         crc32 = mz_stream_crc32_get_value(zip->crc32_stream);
 
     if ((zip->open_mode & MZ_OPEN_MODE_WRITE) == 0)
@@ -1451,7 +1451,7 @@ extern int32_t mz_zip_entry_close_raw(void *handle, uint64_t uncompressed_size,
     }
 
     mz_stream_get_prop_int64(zip->compress_stream, MZ_STREAM_PROP_TOTAL_OUT, (int64_t *)&compressed_size);
-    if ((zip->compression_method != MZ_COMPRESS_METHOD_RAW) || (uncompressed_size == 0))
+    if (((zip->compression_method != MZ_COMPRESS_METHOD_RAW) || (uncompressed_size == 0)) && (zip->file_info.raw == 0))
         mz_stream_get_prop_int64(zip->crc32_stream, MZ_STREAM_PROP_TOTAL_OUT, (int64_t *)&uncompressed_size);
 
     if (zip->file_info.flag & MZ_ZIP_FLAG_ENCRYPTED)
diff --git a/mz_zip.h b/mz_zip.h
index 3e9a72f35c..ace69b9d23 100644
--- a/mz_zip.h
+++ b/mz_zip.h
@@ -48,6 +48,7 @@ typedef struct mz_zip_file_s
     uint16_t internal_fa;               // internal file attributes
     uint32_t external_fa;               // external file attributes
     uint16_t zip64;                     // zip64 extension mode
+    uint16_t raw;                       // raw mode
 
     const char     *filename;           // filename string
     const uint8_t  *extrafield;         // extrafield data

Fixed issues with closing entry when raw. #277

I think I have fixed it another way. Please take a look. Thanks.

Improve logic around raw entry close. #277

Thank you! Looking good! What remains to be solved is the compatibility interface: zipCloseFileInZipRaw*() functions now always call mz_zip_entry_close_raw(), where the latter assumes raw mode and never touches CRC/size, making non-raw entries wrong with zero CRC/size.

The name of the compatibility function is to blame — it's misleading: "Raw" does not mean to imply that it serves to close raw entries only. In fact it's supposed to close any entry (non-raw or raw). So to make it minizip1 compliant, we would still need to preserve the raw flag in zipOpenNewFileInZip*() and call mz_zip_entry_close_int() directly, with the raw argument set accordingly.

If this sounds reasonable to you, I can prepare a patch.

Fixed compatibility for zip close. #277

Thanks. I have made the changes to fix it a different way to just call mz_zip_entry_close from the appropriate functions. Let me know if I missed anything.

Fixed correct entry close in case wrong function called. #277

Thank you Nathan for your fixes and patience, tests (including production code) do work perfectly now!

Navigation Menu

Repository navigation

AES + lzma.zip = -105 (MZ_CRC_ERROR) during unzip #277

5 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

AES + lzma.zip = -105 (MZ_CRC_ERROR) during unzip #277

Description

Activity

nmoinvaz commented on Jul 7, 2018

Agile86 commented on Jul 7, 2018

nmoinvaz commented on Jul 7, 2018

Agile86 commented on Jul 7, 2018

nmoinvaz commented on Jul 9, 2018

Agile86 commented on Jul 9, 2018

vszakats commented on Jul 20, 2018

vszakats commented on Jul 20, 2018

nmoinvaz commented on Jul 20, 2018

nmoinvaz commented on Jul 20, 2018

vszakats commented on Jul 20, 2018

vszakats commented on Jul 20, 2018

5 remaining items

vszakats commented on Jul 21, 2018

nmoinvaz commented on Jul 21, 2018

vszakats commented on Jul 21, 2018

nmoinvaz commented on Jul 22, 2018

vszakats commented on Jul 22, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions