๐ŸŒ
Python
docs.python.org โ€บ 3 โ€บ library โ€บ zlib.html
zlib โ€” Compression compatible with gzip
The zlib manual explains the semantics and usage of the libraryโ€™s many functions. In case gzip (de)compression is a bottleneck, the python-isal package speeds up (de)compression with a mostly compatible API. compression.zstd โ€” Compression compatible with the Zstandard format ... ยฉ Copyright 2001 Python Software Foundation. This page is licensed under the Python Software Foundation License Version 2. Examples...
๐ŸŒ
Stack Abuse
stackabuse.com โ€บ python-zlib-library-tutorial
Python zlib Library Tutorial
July 17, 2023 - This example is very similar to the previous one in that we're decompressing data that originates from a file, except that in this case we're going back to using the one-off decompress method, which decompresses the data in a single method call. This is useful for when your data is small enough to easily fit in memory. ... import zlib compressed_data = open('compressed.dat', 'rb').read() decompressed_data = zlib.decompress(compressed_data) print(decompressed_data)
๐ŸŒ
Python Module of the Week
pymotw.com โ€บ 2 โ€บ zlib
zlib โ€“ Low-level access to GNU zlib compression library - Python Module of the Week
import zlib import binascii ... decompressed = zlib.decompress(compressed) print 'Decompressed :', len(decompressed), decompressed ยท $ python zlib_memory.py Original : 26 This is the original text....
๐ŸŒ
W3Schools
w3schools.com โ€บ python โ€บ ref_module_zlib.asp
Python zlib Module
Python Examples Python Compiler Python Exercises Python Quiz Python Challenges Python Practice Problems Python Server Python Syllabus Python Study Plan Python Interview Q&A Python Bootcamp Python Training ... import zlib text = b'Hello Tobias!
๐ŸŒ
Javatpoint
javatpoint.com โ€บ python-zlib-library
Python zlib Library
Python zlib Library with python, tutorial, tkinter, button, overview, entry, checkbutton, canvas, frame, environment set-up, first python program, operators, etc.
๐ŸŒ
O'Reilly
oreilly.com โ€บ library โ€บ view โ€บ python-standard-library โ€บ 0596000960 โ€บ ch02s23.html
The zlib Module - Python Standard Library [Book]
May 10, 2001 - (Optional) The zlib module provides support for โ€œzlibโ€ compression. (This compression method is also known as โ€œdeflate.โ€) Example 2-43 shows how the compress and decompress functions take string arguments.
Author ย  Fredrik Lundh
Published ย  2001
Pages ย  304
๐ŸŒ
Python Module of the Week
pymotw.com โ€บ 3 โ€บ zlib
zlib โ€” GNU zlib Compression
December 18, 2016 - $ python3 zlib_memory.py Original : 26 b'This is the original text.' Compressed : 32 b'789c0bc9c82c5600a2928c5485fca2ccf4ccbcc41c85 92d48a123d007f2f097e' Decompressed : 26 b'This is the original text.' The previous example demonstrates that the compressed version of small amounts of data can be larger than the uncompressed version.
๐ŸŒ
Python
docs.python.org โ€บ 3.4 โ€บ library โ€บ zlib.html
13.1. zlib โ€” Compression compatible with gzip โ€” Python 3.4.10 documentation
June 16, 2019 - A bytes object that contains any data that was not consumed by the last decompress() call because it exceeded the limit for the uncompressed data buffer. This data has not yet been seen by the zlib machinery, so you must feed it (possibly with further data concatenated to it) back to a subsequent decompress() method call in order to get correct output.
Find elsewhere
๐ŸŒ
GeeksforGeeks
geeksforgeeks.org โ€บ zlib-compresss-in-python
zlib.compress(s) in python - GeeksforGeeks
March 6, 2020 - Python3 1=1 ยท # import zlib and compress import zlib s = b'This is GFG author, and final year student.' print(len(s)) # using zlib.compress(s) method t = zlib.compress(s) print(len(t)) Output : 43 49 Example #2 : Python3 1=1 ยท # import zlib and compress import zlib s = b'GeeksForGeeks@12345678' print(len(s)) # using zlib.compress(s) method t = zlib.compress(s) print(len(t)) Output : 22 27 ยท
๐ŸŒ
Tutorialspoint
tutorialspoint.com โ€บ python โ€บ zlib_module.htm
The zlib Module - Python
import zlib data = b'Hello TutorialsPoint' compressed = zlib.compress(data) print ("Compressed:",compressed) decompressed = zlib.decompress(compressed) print ("Decompressed:", decompressed)
๐ŸŒ
GeeksforGeeks
geeksforgeeks.org โ€บ python โ€บ zlib-decompresss-in-python
zlib.decompress(s) in Python - GeeksforGeeks
March 6, 2020 - Example #1 : In this example we can see that by using zlib.decompress(s) method, we are able to decompress the compressed string in the byte format of string by using this method.
๐ŸŒ
Linux Hint
linuxhint.com โ€บ zlib-python
Linux Hint โ€“ Linux Hint
November 30, 2022 - Linux Hint LLC, [email protected] 1210 Kelly Park Circle, Morgan Hill, CA 95037 Privacy Policy and Terms of Use
๐ŸŒ
MicroPython
docs.micropython.org โ€บ en โ€บ latest โ€บ library โ€บ zlib.html
zlib โ€“ zlib compression & decompression โ€” MicroPython latest documentation
The output formats are โ€œrawโ€ DEFLATE (no header/footer), zlib, and gzip, where the latter two include a header and checksum. The low four bits of the absolute value of wbits set the base-2 logarithm of the DEFLATE dictionary window size. So for example, wbits=10, wbits=-10, and wbits=26 all set the window size to 1024 bytes.
๐ŸŒ
TutorialsPoint
tutorialspoint.com โ€บ compression-compatible-with-gzip-in-python-zlib
Python - Data Compression
June 25, 2020 - Python's standard library has a rich collection of modules for data compression and archiving. One can select whichever is suitable for his job. There are following modules related to data compression โˆ’
๐ŸŒ
Readthedocs
python-zlib-ng.readthedocs.io
Welcome to python-zlib-ngโ€™s documentation! โ€” python-zlib-ng 1.0.0 documentation
Python-zlib-ng can be installed via conda, for example using the miniconda installer with a properly setup conda-forge channel.
๐ŸŒ
Jon
jon.glass โ€บ blog โ€บ has-fun-with-zlib
Use Python zlib to recover Zip Files โ€“ Half Full of Security
November 25, 2014 - Now that we have our functions, let's try them out. To verify that zlib created the same deflated content as the Windows standard compression, I zipped the uncompressed.txt through the Explorer. Then I compared the compressed.bin file with the uncompressed.zip in a hex editor.
Top answer
1 of 4
36

According to RFC 1950 , the difference between the "OK" 0x789C and the "bad" 0x78DA is in the FLEVEL bit-field:

  FLEVEL (Compression level)
     These flags are available for use by specific compression
     methods.  The "deflate" method (CM = 8) sets these flags as
     follows:

        0 - compressor used fastest algorithm
        1 - compressor used fast algorithm
        2 - compressor used default algorithm
        3 - compressor used maximum compression, slowest algorithm

     The information in FLEVEL is not needed for decompression; it
     is there to indicate if recompression might be worthwhile.

"OK" uses 2, "bad" uses 3. So that difference in itself is not a problem.

To get any further, you might consider supplying the following information for each of compressing and (attempted) decompressing: what platform, what version of Python, what version of the zlib library, what was the actual code used to call the zlib module. Also supply the full traceback and error message from the failing decompression attempts. Have you tried to decompress the failing files with any other zlib-reading software? With what results? Please clarify what you have to work with: Does "Am I hosed?" mean that you don't have access to the original data? How did it get from a stream to a file? What guarantee do you have that the data was not mangled in transmission?

UPDATE Some observations based on partial clarifications published in your self-answer:

You are using Windows. Windows distinguishes between binary mode and text mode when reading and writing files. When reading in text mode, Python 2.x changes '\r\n' to '\n', and changes '\n' to '\r\n' when writing. This is not a good idea when dealing with non-text data. Worse, when reading in text mode, '\x1a' aka Ctrl-Z is treated as end-of-file.

To compress a file:

# imports and other superstructure left as a exercise
str_object1 = open('my_log_file', 'rb').read()
str_object2 = zlib.compress(str_object1, 9)
f = open('compressed_file', 'wb')
f.write(str_object2)
f.close()

To decompress a file:

str_object1 = open('compressed_file', 'rb').read()
str_object2 = zlib.decompress(str_object1)
f = open('my_recovered_log_file', 'wb')
f.write(str_object2)
f.close()

Aside: Better to use the gzip module which saves you having to think about nasssties like text mode, at the cost of a few bytes for the extra header info.

If you have been using 'rb' and 'wb' in your compression code but not in your decompression code [unlikely?], you are not hosed, you just need to flesh out the above decompression code and go for it.

Note carefully the use of "may", "should", etc in the following untested ideas.

If you have not been using 'rb' and 'wb' in your compression code, the probability that you have hosed yourself is rather high.

If there were any instances of '\x1a' in your original file, any data after the first such is lost -- but in that case it shouldn't fail on decompression (IOW this scenario doesn't match your symptoms).

If a Ctrl-Z was generated by zlib itself, this should cause an early EOF upon attempted decompression, which should of course cause an exception. In this case you may be able to gingerly reverse the process by reading the compressed file in binary mode and then substitute '\r\n' with '\n' [i.e. simulate text mode without the Ctrl-Z -> EOF gimmick]. Decompress the result. Edit Write the result out in TEXT mode. End edit

UPDATE 2 I can reproduce your symptoms -- with ANY level 1 to 9 -- with the following script:

import zlib, sys
fn = sys.argv[1]
level = int(sys.argv[2])
s1 = open(fn).read() # TEXT mode
s2 = zlib.compress(s1, level)
f = open(fn + '-ct', 'w') # TEXT mode
f.write(s2)
f.close()
# try to decompress in text mode
s1 = open(fn + '-ct').read() # TEXT mode
s2 = zlib.decompress(s1) # error -5
f = open(fn + '-dtt', 'w')
f.write(s2)
f.close()

Note: you will need a use a reasonably large text file (I used an 80kb source file) to ensure that the decompression result will contain a '\x1a'.

I can recover with this script:

import zlib, sys
fn = sys.argv[1]
# (1) reverse the text-mode write
# can't use text-mode read as it will stop at Ctrl-Z
s1 = open(fn, 'rb').read() # BINARY mode
s1 = s1.replace('\r\n', '\n')
# (2) reverse the compression
s2 = zlib.decompress(s1)
# (3) reverse the text mode read
f = open(fn + '-fixed', 'w') # TEXT mode
f.write(s2)
f.close()

NOTE: If there is a '\x1a' aka Ctrl-Z byte in the original file, and the file is read in text mode, that byte and all following bytes will NOT be included in the compressed file, and thus can NOT be recovered. For a text file (e.g. source code), this is no loss at all. For a binary file, you are most likely hosed.

Update 3 [following late revelation that there's an encryption/decryption layer involved in the problem]:

The "Error -5" message indicates that the data that you are trying to decompress has been mangled since it was compressed. If it's not caused by using text mode on the files, suspicion obviously(?) falls on your decryption and encryption wrappers. If you want help, you need to divulge the source of those wrappers. In fact what you should try to do is (like I did) put together a small script that reproduces the problem on more than one input file. Secondly (like I did) see whether you can reverse the process under what conditions. If you want help with the second stage, you need to divulge the problem-reproduction script.

2 of 4
4

I was looking for

python -c 'import sys,zlib;sys.stdout.write(zlib.decompress(sys.stdin.read()))'

wrote it myself; based on answers of zlib decompression in python

๐ŸŒ
GitHub
github.com โ€บ oliverbaileysmith โ€บ zlib-decompress
GitHub - oliverbaileysmith/zlib-decompress: Python implementation of ZLIB decompression ยท GitHub
A Python implementation of decompression of ZLIB compressed data. decompress.py contains the user-facing function decompress which takes in data compressed by zlib.compress and returns the decompressed data.
Author ย  oliverbaileysmith