It is also possible to decompress it using standard shell-script + gzip, if you don't have, or want to use openssl or other tools.
The trick is to prepend the gzip magic number and compress method to the actual data from zlib.compress:
printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" |cat - /tmp/data |gzip -dc >/tmp/out
Edits:
@d0sboots commented: For RAW Deflate data, you need to add 2 more null bytes:
→ "\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x00"
This Q on SO gives more information about this approach. An answer there suggests that there is also an 8 byte footer.
Users @Vitali-Kushner and @mark-bessey reported success even with truncated files, so a gzip footer does not seem strictly required.
@tobias-kienzler suggested this function for the bashrc:
zlibd() (printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" | cat - "$@" | gzip -dc)
It is also possible to decompress it using standard shell-script + gzip, if you don't have, or want to use openssl or other tools.
The trick is to prepend the gzip magic number and compress method to the actual data from zlib.compress:
printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" |cat - /tmp/data |gzip -dc >/tmp/out
Edits:
@d0sboots commented: For RAW Deflate data, you need to add 2 more null bytes:
→ "\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x00"
This Q on SO gives more information about this approach. An answer there suggests that there is also an 8 byte footer.
Users @Vitali-Kushner and @mark-bessey reported success even with truncated files, so a gzip footer does not seem strictly required.
@tobias-kienzler suggested this function for the bashrc:
zlibd() (printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" | cat - "$@" | gzip -dc)
zlib-flate -uncompress < IN_FILE > OUT_FILE
I tried this and it worked for me.
zlib-flate can be found in package qpdf (in Debian Squeeze, Fedora 23, and brew on MacOS according to comments in other answers)
(Thanks to user @tino who provided this as a comment below the OpenSSL answer. Made into propper answer for easy access.)
Zlib uncompression for an noob
Looks like zlib compressed data, here's more info how to deflate it: https://unix.stackexchange.com/questions/22834/how-to-uncompress-zlib-data-in-unix
More on reddit.comred lang - How to decompress/deflate zlib data [rfc1951]? - Stack Overflow
blob - How can I DEFLATE with a command line tool to extract a Git object? - Stack Overflow
How can I decompress a gzip stream with zlib? - Stack Overflow
Videos
i was playing ctf challenges and i stuck in this point can someone explain to me(this compression) and how i can uncompress this file .
$ file out.zlib out.zlib: zlib compressed data
Something like the following will print the raw content, including the "$type $length\0" header:
perl -MCompress::Zlib -e 'undef $/; print uncompress(<>)' \
< .git/objects/27/de0a1dd5a89a94990618632967a1c86a82d577
You can do this with the OpenSSL command line tool:
openssl zlib -d < $IN > $OUT
Unfortunately, at least on Ubuntu, the zlib subcommand is disabled in the default build configuration (--no-zlib --no-zlib-dynamic), so you would need to compile openssl from source to use it. But it is enabled by default on Arch, for example.
Edit: Seems like the zlib command is no longer supported on Arch either. This answer might not be useful anymore :(
To decompress a gzip format file with zlib, call inflateInit2 with the windowBits parameter as 16+MAX_WBITS, like this:
inflateInit2(&stream, 16+MAX_WBITS);
If you don't do this, zlib will complain about a bad stream format. By default, zlib creates streams with a zlib header, and on inflate does not recognise the different gzip header unless you tell it so. Although this is documented starting in version 1.2.1 of the zlib.h header file, it is not in the zlib manual. From the header file:
windowBitscan also be greater than 15 for optional gzip decoding. Add 32 towindowBitsto enable zlib and gzip decoding with automatic header detection, or add 16 to decode only the gzip format (the zlib format will return aZ_DATA_ERROR). If a gzip stream is being decoded,strm->adleris a crc32 instead of an adler32.
python
zlib library supports:
- RFC 1950 (
zlibcompressed format) - RFC 1951 (
deflatecompressed format) - RFC 1952 (
gzipcompressed format)
The python zlib module will support these as well.
choosing windowBits
But zlib can decompress all those formats:
- to (de-)compress
deflateformat, usewbits = -zlib.MAX_WBITS - to (de-)compress
zlibformat, usewbits = zlib.MAX_WBITS - to (de-)compress
gzipformat, usewbits = zlib.MAX_WBITS | 16
See documentation in http://www.zlib.net/manual.html#Advanced (section inflateInit2)
examples
test data:
>>> deflate_compress = zlib.compressobj(9, zlib.DEFLATED, -zlib.MAX_WBITS)
>>> zlib_compress = zlib.compressobj(9, zlib.DEFLATED, zlib.MAX_WBITS)
>>> gzip_compress = zlib.compressobj(9, zlib.DEFLATED, zlib.MAX_WBITS | 16)
>>>
>>> text = '''test'''
>>> deflate_data = deflate_compress.compress(text) + deflate_compress.flush()
>>> zlib_data = zlib_compress.compress(text) + zlib_compress.flush()
>>> gzip_data = gzip_compress.compress(text) + gzip_compress.flush()
>>>
obvious test for zlib:
>>> zlib.decompress(zlib_data)
'test'
test for deflate:
>>> zlib.decompress(deflate_data)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
zlib.error: Error -3 while decompressing data: incorrect header check
>>> zlib.decompress(deflate_data, -zlib.MAX_WBITS)
'test'
test for gzip:
>>> zlib.decompress(gzip_data)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
zlib.error: Error -3 while decompressing data: incorrect header check
>>> zlib.decompress(gzip_data, zlib.MAX_WBITS|16)
'test'
the data is also compatible with gzip module:
>>> import gzip
>>> import StringIO
>>> fio = StringIO.StringIO(gzip_data)
>>> f = gzip.GzipFile(fileobj=fio)
>>> f.read()
'test'
>>> f.close()
automatic header detection (zlib or gzip)
adding 32 to windowBits will trigger header detection
>>> zlib.decompress(gzip_data, zlib.MAX_WBITS|32)
'test'
>>> zlib.decompress(zlib_data, zlib.MAX_WBITS|32)
'test'
using gzip instead
For gzip data with gzip header you can use gzip module directly; but please remember that under the hood, gzip uses zlib.
fh = gzip.open('abc.gz', 'rb')
cdata = fh.read()
fh.close()