tar -zcvf compressFileName.tar.gz folderToCompress
everything in folderToCompress will go to compressFileName
Edit: After review and comments I realized that people may get confused with compressFileName without an extension. If you want you can use .tar.gz extension(as suggested) with the compressFileName
Answer from amitchhajer on Stack Overflowtar -zcvf compressFileName.tar.gz folderToCompress
everything in folderToCompress will go to compressFileName
Edit: After review and comments I realized that people may get confused with compressFileName without an extension. If you want you can use .tar.gz extension(as suggested) with the compressFileName
there are lots of compression methods that work recursively command line and its good to know who the end audience is.
i.e. if it is to be sent to someone running windows then zip would probably be best:
zip -r file.zip folder_to_zip
unzip filenname.zip
for other linux users or your self tar is great
tar -cvzf filename.tar.gz folder
tar -cvjf filename.tar.bz2 folder # even more compression
#change the -c to -x to above to extract
One must be careful with tar and how things are tarred up/extracted, for example if I run
cd ~
tar -cvzf passwd.tar.gz /etc/passwd
tar: Removing leading `/' from member names
/etc/passwd
pwd
/home/myusername
tar -xvzf passwd.tar.gz
this will create /home/myusername/etc/passwd
unsure if all versions of tar do this:
Removing leading `/' from member names
I'd prefer gzip -r ./ which does the same thing but is shorter.
No need for loops or anything more than find and gzip:
find . -type f ! -name '*.gz' -exec gzip "{}" \;
This finds all regular files in and below the current directory whose names don't end with the .gz extension (that is, all files that are not already compressed). It invokes gzip on each file individually.
Edit, based on comment from user unknown:
The curly braces ({}) are replaced with the filename, which is passed directly, as a single word, to the command following -exec as you can see here:
$ touch foo
$ touch "bar baz"
$ touch xyzzy
$ find . -exec echo {} \;
./foo
./bar baz
./xyzzy
Can I zip an entire folder using gzip? - Unix & Linux Stack Exchange
How do I gzip thousand of files in one directory ?
Bash: Gzip files in Directory and it´s Subdirectories
linux - Unzip all gz files in all subdirectories in the terminal - Stack Overflow
gzip will always compress each file into a single .gz file when given a list of files on its command line.
For example
$ gzip -r log.2011
to recursively walk the log.2011 directory and compress all files it finds, or
$ gzip log.2011/*
to compress only the files in the log.2011 directory without descending into subdirectories.
This will output a gz archive for each matching file, files are replaced by the archive:
gzip fileprefix*
Have a look to the '-r' flag too.
No.
Unlike zip, gzip functions as a compression algorithm only.
Because of various reasons some of which hearken back to the era of tape drives, Unix uses a program named tar to archive data, which can then be compressed with a compression program like gzip, bzip2, 7zip, etc.
In order to "zip" a directory, the correct command would be
tar -zcvf archive.tar.gz directory/
This will tell tar to
compress it using the z (gzip) algorithm
c (create) an archive from the files in
directory(taris recursive by default)v (verbosely) list (on /dev/stderr so it doesn't affect piped commands) all the files it adds to the archive.
and store the output as a f (file) named
archive.tar.gz
The tar command offers gzip support (via the -z flag) purely for your convenience. The gzip command/lib is completely separate. The command above is effectively the same as
tar -cv directory | gzip > archive.tar.gz
To decompress and unpack the archive into the current directory you would use
tar -zxvf archive.tar.gz
That command is effectively the same as
gunzip < archive.tar.gz | tar -xv
tar has many, many, MANY other options and uses as well; I heartily recommend reading through its manpage sometime.
The gzip command will not recursively compress a directory into a single zip file, when using the -r switch. Rather it will walk that directory structure and zip each file that it finds into a separate file.
Example
before
$ tree dir1/
dir1/
|-- dir11
| |-- file11
| |-- file12
| `-- file13
|-- file1
|-- file2
`-- file3
now run the gzip command
$ gzip -r dir1
after
$ tree dir1/
dir1/
|-- dir11
| |-- file11.gz
| |-- file12.gz
| `-- file13.gz
|-- file1.gz
|-- file2.gz
`-- file3.gz
If you'd prefer to zip up the directory structure then you'll likely want to use the tar command, and then compress the resulting .tar file.
$ tar zcvf dir1.tar.gz dir1/
Example
$ tar zcvf dir1.tar.gz dir1/
dir1/
dir1/file1
dir1/file2
dir1/dir11/
dir1/dir11/file11.gz
dir1/dir11/file12.gz
dir1/dir11/file13.gz
dir1/file3
Which results in the following single file:
$ ls -l | grep tar
-rw-rw-r-- 1 saml saml 271 Oct 1 08:07 dir1.tar.gz
You can confirm its contents:
$ tar ztvf dir1.tar.gz
drwxrwxr-x saml/saml 0 2013-10-01 08:05 dir1/
-rw-rw-r-- saml/saml 0 2013-10-01 07:45 dir1/file1
-rw-rw-r-- saml/saml 0 2013-10-01 07:45 dir1/file2
drwxrwxr-x saml/saml 0 2013-10-01 08:04 dir1/dir11/
-rw-rw-r-- saml/saml 27 2013-10-01 07:45 dir1/dir11/file11.gz
-rw-rw-r-- saml/saml 27 2013-10-01 07:45 dir1/dir11/file12.gz
-rw-rw-r-- saml/saml 27 2013-10-01 07:45 dir1/dir11/file13.gz
-rw-rw-r-- saml/saml 0 2013-10-01 07:45 dir1/file3
No, gzip can't do this, -r just means "descend into subdirectories" but there is no option for "descend into subdirectories and then look for files matching this glob". The expansion of the *.vtu glob happens before grep is launched, and it is handled by the shell not grep, so grep is given a specific list of files: those files matching *.vtu in the current directory.
So yes, globstar is your best bet. As for the use of -r, that is explained in man gzip:
-r --recursive
Travel the directory structure recursively. If any of the file
names specified on the command line are directories, gzip will
descend into the directory and compress all the files it finds
there (or decompress them in the case of gunzip ).
So gzip -r foo means "descend into foo if foo is a directory and gzip any files in it". If foo matches both files and directories, if for example you had both file.vtu and my.vtu/ in the directory you ran gzip in, then the contents of my.vtu would also be compressed. Without it, you would get my.vtu is a directory -- ignored.
Other options include:
find . -name "*.vtu" -exec gzip {} +to compress all matching files.gzip **/*.vtuwithglobstarset.find . -name "*.vtu" | xargs gzip(as long as your names are sane and don't contain newlines)find . -name "*.vtu" -print0 | xargs -0 gzip(if your file names can contain newlines)
After the answer by terdon, and upon tinkering a bit, I came to the conclusion that the way -r works is the following:
- If what is matched is a file (only in the present directory) do
gzip. - If what is matched is a directory, enter that directory, and down there execute
gzip -r *.
For me, this is extremely weird (and therefore I would have never imagined this is how it works).
For instance, if in ./ I have
foo
foo.vtk
test.vtk/
test.vtk/another.vtk/
test.vtk/another.vtk/cake.vtk
test.vtk/another.vtk/dow.txt
test.vtk/cake.vtk
test.vtk/dow.txt
test.vtk/this/
test.vtk/this/cake.vtk
test.vtk/this/dow.txt
command gzip -r -v *.vtk would gzip all files except ./foo.
All files (not only *.vtk), in all subdirectories *.vtk (with depth=1) and * (with depth>1) would be gzipped.
I would use
find /path/to/dir \( -name '*.css' -o -name '*.html' \) -exec gzip --verbose --keep {} \;
Change name to iname if you want to match the extensions case-insensitively (i.e. include .CSS and/or .HTML extensions). You can omit the /path/to/dir if you want to start the recursive search from the current directory.
you can do that with a for loop to find every file then compress it:
for i in `find | grep -E "\.css$|\.html$"`; do gzip "$i" ; done
gunzip has -r option. From man gunzip :
-r --recursive
Travel the directory structure recursively. If any of the
file names specified on the command line are directories, gzip
will descend into the directory and compress all the files it finds
there (or decompress them in the case of gunzip ).
So, if you want to gunzip all compressed files (gunzip can currently decompress files created by gzip, zip, compress, compress -H or pack) inside the directory /foo/bar and all its subdirectories :
gunzip -r /foo/bar
This will handle file names with spaces too.
Using the commands below. Replace <path_of_your_zips> with the path to your ZIP files and <out> with your destination folder:
For GZ files
find <path_of_your_zips> -type f -name "*.gz" -exec tar xf {} -C <out> \;or
find <path_of_your_zips> -type f -name "*.gz" -print0 | xargs -0 -I{} tar xf {} -C <out>For ZIP files
find <path_of_your_zips> -type f -name "*.zip" -exec unzip {} -d <out> \;or
find <path_of_your_zips> -type f -name "*.zip" -print0 | xargs -0 -I{} unzip {} -d <out>
I have a directory which contains about 17GB of files. There's so many files that when I run gzip with a glob pattern, it would not proceed.
$ gzip *.json -bash: /bin/gzip: Argument list too long
I know the solution is probably to chunk the list of files. What would you suggest I do ?
If you want, for each of those, to launch "gzip -d" on them:
cd theparentdir && gzip -d $(find ./ -type f -name '*.gz')
and then, to gzip them back:
cd theparentdir && gzip $(find ./ -type f -name '*.csv')
This will however choke in many cases
- if filenames have some special characters (spaces, tabs, newline, etc) in them
- other similar cases
- or if there are TOO MANY files to be put after the gzip command!
A solution would be instead, if you have GNU find, to do :
find ... -print0 | xarsg -0 gzip -d # for the gunzip one, but still choke on files with "newline" in them
Another (arguably better?) solution, if you have GNU find at your disposal:
cd theparentdir && find ./ -type f -name '*.gz' -exec gzip -d '{}' '+'
and to re-zip all csv in that parentdir & all subdirs:
cd theparentdir && find ./ -type f -name '*.csv' -exec gzip '{}' '+'
"+" tells GNU find to try to put as many found files as it can on each gzip invocation (instead of doing 1 gzip incocation per file, very very ressource intensive and very innefficient and slow), similar to xargs, but with some benefits (1 command only, no pipe needed)
There is an option for recursivity (-r).
gzip -dr ./datasets
All archive will be decompressed in their own directory.
Example: gzip -dr ./a
a/b/c/test1.gz
a/b/d/test2.gz
a/e/test3.gz
After execution:
a/b/c/test1
a/b/d/test2
a/e/test3
find . -type f | \
while read -r x
do
gzip -c "$x" > "$x.gz"
done
The -c pushes the result to stdout and keeps the original alone. The disadvantage is, that you need to find the files yourself. For more sophisticated traversing, you can use find(1), however, like above: . searches starting from the current directory, and -type f returns the name of every regular file.
find . -type f -not \( -name '*.gz' -or -name '*[~#]' \) -exec sh -c 'gzip -c "{}" > "{}.gz"' \;
You could easily switch it around to include what you want to compress ( -name '*.txt -or -name '*.html etc.) instead of like now, excluding some files (already compressed, backup and temporary files).
Handles spaces in the filename just fine too.
Change gzip to echo gzip for testing. Or skip the -exec part all together.
Edit: Oh, I forgot to mention that this doesn't check if <target>.gz already exists. This may or may not be a problem.
Edit2: Ok, here we go with something that checks for existing file. If that may be wanted. Pardon the oneliney-ness.
while read file; do if [ ! -f "$file.gz" ]; then echo "Compressing $file"; gzip -c "$file" > "$file.gz"; else echo "Not overwriting $file.gz"; fi done < <(find . -type f -not \( -name '*.gz' -or -name '*[~#]' \))
My find-foo is maybe not what it could be, it may very well be possible to skip directly in find.
This small script seems to be your best option, given your requirements:
cd directory
for dir in */
do
base=$(basename "$dir")
tar -czf "${base}.tar.gz" "$dir"
done
It properly handles directories with spaces in their names.
How about this: find * -maxdepth 0 -type d -exec tar czvf {}.tar.gz {} \;
Explanation: You run a find on all items in the current directory. Maxdepth 0 makes it not recurse any lower than the arguments given. (In this case *, or all items in your current directory) The 'd' argument to "-type" only matches directories. Then exec runs tar on whatever matches. ({} is replaced by the matching file)