🌐
Python
docs.python.org › 3 › library › zlib.html
zlib — Compression compatible with gzip
The zlib manual explains the semantics and usage of the library’s many functions. In case gzip (de)compression is a bottleneck, the python-isal package speeds up (de)compression with a mostly compatible API. compression.zstd — Compression compatible with the Zstandard format ... © Copyright 2001 Python Software Foundation. This page is licensed under the Python Software Foundation License Version 2. Examples...
🌐
Stack Abuse
stackabuse.com › python-zlib-library-tutorial
Python zlib Library Tutorial
July 17, 2023 - This example is very similar to the previous one in that we're decompressing data that originates from a file, except that in this case we're going back to using the one-off decompress method, which decompresses the data in a single method call. This is useful for when your data is small enough to easily fit in memory. ... import zlib compressed_data = open('compressed.dat', 'rb').read() decompressed_data = zlib.decompress(compressed_data) print(decompressed_data)
🌐
W3Schools
w3schools.com › python › ref_module_zlib.asp
Python zlib Module
Python Examples Python Compiler Python Exercises Python Quiz Python Challenges Python Practice Problems Python Server Python Syllabus Python Study Plan Python Interview Q&A Python Bootcamp Python Training ... import zlib text = b'Hello Tobias!
🌐
Effbot
effbot.org › librarybook › zlib.htm
The zlib module - Effbot
# File: zlib-example-1.py import zlib MESSAGE = "life of brian" compressed_message = zlib.compress(MESSAGE) decompressed_message = zlib.decompress(compressed_message) print "original:", repr(MESSAGE) print "compressed message:", repr(compressed_message) print "decompressed message:", repr(decompressed_message) $ python zlib-example-1.py original: 'life of brian' compressed message: 'x\234\313\311LKU\310OSH* \312L\314\003\000!\010\004\302' decompressed message: 'life of brian'
🌐
Python Module of the Week
pymotw.com › 2 › zlib
zlib – Low-level access to GNU zlib compression library - Python Module of the Week
import zlib import binascii ... decompressed = zlib.decompress(compressed) print 'Decompressed :', len(decompressed), decompressed · $ python zlib_memory.py Original : 26 This is the original text....
🌐
Javatpoint
javatpoint.com › python-zlib-library
Python zlib Library
Python zlib Library with python, tutorial, tkinter, button, overview, entry, checkbutton, canvas, frame, environment set-up, first python program, operators, etc.
🌐
Python Module of the Week
pymotw.com › 3 › zlib
zlib — GNU zlib Compression
December 18, 2016 - $ python3 zlib_server.py Client: Contacting server on 127.0.0.1:53658 Client: sending filename: 'lorem.txt' Server: client asked for: 'lorem.txt' Server: RAW b'Lorem ipsum dolor sit amet, consectetuer adipiscin g elit. Donec\n' Server: SENDING b'7801' Server: RAW b'egestas, enim et consectetuer ullamcorper, lectus ligula rutrum ' Server: BUFFERING Server: RAW b'leo, a\nelementum elit tortor eu quam.
🌐
GeeksforGeeks
geeksforgeeks.org › zlib-compresss-in-python
zlib.compress(s) in python - GeeksforGeeks
March 6, 2020 - Python3 1=1 · # import zlib and compress import zlib s = b'This is GFG author, and final year student.' print(len(s)) # using zlib.compress(s) method t = zlib.compress(s) print(len(t)) Output : 43 49 Example #2 : Python3 1=1 · # import zlib and compress import zlib s = b'GeeksForGeeks@12345678' print(len(s)) # using zlib.compress(s) method t = zlib.compress(s) print(len(t)) Output : 22 27 ·
Find elsewhere
🌐
Python
docs.python.org › 3.4 › library › zlib.html
13.1. zlib — Compression compatible with gzip — Python 3.4.10 documentation
June 16, 2019 - A bytes object that contains any data that was not consumed by the last decompress() call because it exceeded the limit for the uncompressed data buffer. This data has not yet been seen by the zlib machinery, so you must feed it (possibly with further data concatenated to it) back to a subsequent ...
🌐
Tutorialspoint
tutorialspoint.com › python › zlib_module.htm
The zlib Module - Python
import zlib data = b'Hello TutorialsPoint' compressed = zlib.compress(data) print ("Compressed:",compressed) decompressed = zlib.decompress(compressed) print ("Decompressed:", decompressed)
🌐
O'Reilly
oreilly.com › library › view › python-standard-library › 0596000960 › ch02s23.html
The zlib Module - Python Standard Library [Book]
May 10, 2001 - (Optional) The zlib module provides support for “zlib” compression. (This compression method is also known as “deflate.”) Example 2-43 shows how the compress and decompress functions take string arguments.
Author   Fredrik Lundh
Published   2001
Pages   304
🌐
GeeksforGeeks
geeksforgeeks.org › zlib-decompresss-in-python
zlib.decompress(s) in Python - GeeksforGeeks
March 6, 2020 - # using zlib.compress(s) method t = zlib.compress(s) print("Compressed String") print(t) print("\nDecompressed String") print(zlib.decompress(t)) Output : Compressed String b'x\x9c\x0b\xc9\xc8,V\x00"w7w\x85\xc4\xd2\x92\x8c\xfc"\x1d\x85\xc4\xbc\x14\x85\xb4\xcc\xbc\xc4\x1c\x85\xca\xd4\xc4"\x85\xe2\x92\xd2\x94\xd4\xbc\x12=\x00A\xc9\x0f\x0b' Decompressed String b'This is GFG author, and final year student.' Example #2 : Python3 1=1 ·
🌐
Shahid Yousuf
shahidyousuf.com › basic-data-compression-in-python-using-zlib
Basic Data Compression in Python Using zlib - Shahid Yousuf
December 12, 2022 - These examples demonstrate some of the more advanced features of the zlib library, such as using specific compression levels and buffer sizes, as well as calculating different types of checksums. For more information, you can refer to the zlib library documentation. Happy coding! ... This series provides practical articles and blog posts on everything related to Python ...
Top answer
1 of 2
4

Grant's answer is fine, but something here needs to be emphasized.

Doesn't the zlib need to compress ??

No! It does not, and cannot always compress. Any operations that losslessly compress and decompress and input must expand some, actually most, inputs, while compressing only some inputs. This is a simple and obvious consequence of counting.

The only thing that is guaranteed by a lossless compressor is that what you get out from decompression is what you put in to compression.

Any useful compression scheme is rigged to take advantage of the specific redundancies expected to be found in the particular kind of data being compressed. Language data, e.g. English, C code, data files, even machine code, which is a sequence of symbols with a specific frequency distribution and oft repeated strings, is compressed using models that are expecting and looking for those redundancies. Such schemes depend on gathering information on the data being compressed in the first, at least, 10's of Kbytes before the compression starts being really effective.

Your example is far too short to have the statistics needed, and has no repetition of any kind, and so will be expanded by any general compressor.

2 of 2
3

You're going to have a hard time compressing a string like that. It's rather short and contains 26 unique characters. Compressors work by assigning byte values to common words, characters, etc, so by having all unique characters you'll get poor performance.

You'll also get poor performance if the data is random.

Here's an example with a string of the same length which compresses.

>>> str2 = 'a'*26
>>> str2
'aaaaaaaaaaaaaaaaaaaaaaaaaa'
>>> sys.getsizeof(str2)
63
>>> sys.getsizeof(zlib.compress(str2))
48
🌐
Delft Stack
delftstack.com › home › howto › python › python zlib
How to Compress and Decompress Data Using Zlib in Python | Delft Stack
February 2, 2024 - The following code snippet demonstrates how to decompress the previously compressed data with Python’s zlib.decompress() function.
Top answer
1 of 4
36

According to RFC 1950 , the difference between the "OK" 0x789C and the "bad" 0x78DA is in the FLEVEL bit-field:

  FLEVEL (Compression level)
     These flags are available for use by specific compression
     methods.  The "deflate" method (CM = 8) sets these flags as
     follows:

        0 - compressor used fastest algorithm
        1 - compressor used fast algorithm
        2 - compressor used default algorithm
        3 - compressor used maximum compression, slowest algorithm

     The information in FLEVEL is not needed for decompression; it
     is there to indicate if recompression might be worthwhile.

"OK" uses 2, "bad" uses 3. So that difference in itself is not a problem.

To get any further, you might consider supplying the following information for each of compressing and (attempted) decompressing: what platform, what version of Python, what version of the zlib library, what was the actual code used to call the zlib module. Also supply the full traceback and error message from the failing decompression attempts. Have you tried to decompress the failing files with any other zlib-reading software? With what results? Please clarify what you have to work with: Does "Am I hosed?" mean that you don't have access to the original data? How did it get from a stream to a file? What guarantee do you have that the data was not mangled in transmission?

UPDATE Some observations based on partial clarifications published in your self-answer:

You are using Windows. Windows distinguishes between binary mode and text mode when reading and writing files. When reading in text mode, Python 2.x changes '\r\n' to '\n', and changes '\n' to '\r\n' when writing. This is not a good idea when dealing with non-text data. Worse, when reading in text mode, '\x1a' aka Ctrl-Z is treated as end-of-file.

To compress a file:

# imports and other superstructure left as a exercise
str_object1 = open('my_log_file', 'rb').read()
str_object2 = zlib.compress(str_object1, 9)
f = open('compressed_file', 'wb')
f.write(str_object2)
f.close()

To decompress a file:

str_object1 = open('compressed_file', 'rb').read()
str_object2 = zlib.decompress(str_object1)
f = open('my_recovered_log_file', 'wb')
f.write(str_object2)
f.close()

Aside: Better to use the gzip module which saves you having to think about nasssties like text mode, at the cost of a few bytes for the extra header info.

If you have been using 'rb' and 'wb' in your compression code but not in your decompression code [unlikely?], you are not hosed, you just need to flesh out the above decompression code and go for it.

Note carefully the use of "may", "should", etc in the following untested ideas.

If you have not been using 'rb' and 'wb' in your compression code, the probability that you have hosed yourself is rather high.

If there were any instances of '\x1a' in your original file, any data after the first such is lost -- but in that case it shouldn't fail on decompression (IOW this scenario doesn't match your symptoms).

If a Ctrl-Z was generated by zlib itself, this should cause an early EOF upon attempted decompression, which should of course cause an exception. In this case you may be able to gingerly reverse the process by reading the compressed file in binary mode and then substitute '\r\n' with '\n' [i.e. simulate text mode without the Ctrl-Z -> EOF gimmick]. Decompress the result. Edit Write the result out in TEXT mode. End edit

UPDATE 2 I can reproduce your symptoms -- with ANY level 1 to 9 -- with the following script:

import zlib, sys
fn = sys.argv[1]
level = int(sys.argv[2])
s1 = open(fn).read() # TEXT mode
s2 = zlib.compress(s1, level)
f = open(fn + '-ct', 'w') # TEXT mode
f.write(s2)
f.close()
# try to decompress in text mode
s1 = open(fn + '-ct').read() # TEXT mode
s2 = zlib.decompress(s1) # error -5
f = open(fn + '-dtt', 'w')
f.write(s2)
f.close()

Note: you will need a use a reasonably large text file (I used an 80kb source file) to ensure that the decompression result will contain a '\x1a'.

I can recover with this script:

import zlib, sys
fn = sys.argv[1]
# (1) reverse the text-mode write
# can't use text-mode read as it will stop at Ctrl-Z
s1 = open(fn, 'rb').read() # BINARY mode
s1 = s1.replace('\r\n', '\n')
# (2) reverse the compression
s2 = zlib.decompress(s1)
# (3) reverse the text mode read
f = open(fn + '-fixed', 'w') # TEXT mode
f.write(s2)
f.close()

NOTE: If there is a '\x1a' aka Ctrl-Z byte in the original file, and the file is read in text mode, that byte and all following bytes will NOT be included in the compressed file, and thus can NOT be recovered. For a text file (e.g. source code), this is no loss at all. For a binary file, you are most likely hosed.

Update 3 [following late revelation that there's an encryption/decryption layer involved in the problem]:

The "Error -5" message indicates that the data that you are trying to decompress has been mangled since it was compressed. If it's not caused by using text mode on the files, suspicion obviously(?) falls on your decryption and encryption wrappers. If you want help, you need to divulge the source of those wrappers. In fact what you should try to do is (like I did) put together a small script that reproduces the problem on more than one input file. Secondly (like I did) see whether you can reverse the process under what conditions. If you want help with the second stage, you need to divulge the problem-reproduction script.

2 of 4
4

I was looking for

python -c 'import sys,zlib;sys.stdout.write(zlib.decompress(sys.stdin.read()))'

wrote it myself; based on answers of zlib decompression in python

🌐
GitHub
github.com › oliverbaileysmith › zlib-decompress
GitHub - oliverbaileysmith/zlib-decompress: Python implementation of ZLIB decompression · GitHub
A Python implementation of decompression of ZLIB compressed data. decompress.py contains the user-facing function decompress which takes in data compressed by zlib.compress and returns the decompressed data.
Author   oliverbaileysmith
🌐
Linux Hint
linuxhint.com › zlib-python
Linux Hint – Linux Hint
November 30, 2022 - Linux Hint LLC, [email protected] 1210 Kelly Park Circle, Morgan Hill, CA 95037 Privacy Policy and Terms of Use