I believe that's because gzip never operates over directories, it acts as a compression algorithm unlike zip and tar where we could compress directories. python's implementation of gzip is to operate on files. However recursive traversal of a directory tree is easy if we look at the os.walk call.
(I haven't tested this)
def gunzip(file_path,output_path):
with gzip.open(file_path,"rb") as f_in, open(output_path,"wb") as f_out:
shutil.copyfileobj(f_in, f_out)
def recurse_and_gunzip(root):
walker = os.walk(root)
for root,dirs,files in walker:
for f in files:
if fnmatch.fnmatch(f,"*.gz"):
gunzip(f,f.replace(".gz",""))
Answer from Addy on Stack OverflowI believe that's because gzip never operates over directories, it acts as a compression algorithm unlike zip and tar where we could compress directories. python's implementation of gzip is to operate on files. However recursive traversal of a directory tree is easy if we look at the os.walk call.
(I haven't tested this)
def gunzip(file_path,output_path):
with gzip.open(file_path,"rb") as f_in, open(output_path,"wb") as f_out:
shutil.copyfileobj(f_in, f_out)
def recurse_and_gunzip(root):
walker = os.walk(root)
for root,dirs,files in walker:
for f in files:
if fnmatch.fnmatch(f,"*.gz"):
gunzip(f,f.replace(".gz",""))
It may not answer this specific question, but for those looking to extract a gzipped directory structure: that would be a job for shutil.unpack_archive.
For example:
import shutil
shutil.unpack_archive(
filename='path/to/archive.tar.gz', extract_dir='where/to/extract/to'
)
How to unzip gz file using Python - Stack Overflow
compression - How do I compress a folder with the Python GZip module? - Stack Overflow
subprocess - How do I decompress a .gz file and save the decompressed file to a different directory in Python? - Stack Overflow
How to unzip a massive amount of files using Windows Python 2.7?
Why over engineer? Simply gunzip -r a directory of zipped files:
gunzip -r ./directory_of_zipped_files/
EDIT: Sorry ... now noted that you said windows. This is a *nix solution.
More on reddit.comThe code to compress a folder in to tar file is:
import tarfile
tar = tarfile.open("TarName.tar.gz", "w:gz")
tar.add("folder/location", arcname="TarName")
tar.close()
It works for me. Hope that works for you too.
I don't do UI, so you're on your own for getting the folder name from the user. Here's one way to make a gz-compressed tarfile. It does not recurse over subfolders, you'll need something like os.walk() for that.
# assume the path to the folder to compress is in 'folder_path'
import tarfile
import os
with tarfile.open( folder_path + ".tgz", "w:gz" ) as tar:
for name in os.listdir( folder_path ):
tar.add(name)
You can try couple of changes:
- in the subprocess, use 'gunzip' Unix command rather than 'zcat'
- place the 'gunzip' command in a shell script file, e.g. bash shell. subprocess.call() the script file instead of the command directly. This may be helpful if you need to do additional os level manipulations such as file copies or move to differently locations etc. Make sure to set the shell script file as executable with 'chmod' on the command line.
Good luck.
Seems like process dies because you are trying to load entire archive into memory. Watch memory usage to confirm this.
Because GzipFile constructs file-like object, it might be possible to run it through shutil.copyfileobj. Let's make function for this:
import gzip
import shutil
BUFFER_SIZE = 200 * 1024 * 1024 # 200 mb, arbitrary
def gunzip(source, destination, buffer_size=BUFFER_SIZE):
with gzip.open(source) as s:
with open(destination, 'wb') as d:
shutil.copyfileobj(s, d, buffer_size)
And use it:
gunzip("/directory1/file.txt.gz", "/directory2/file.txt")