tar -zcvf compressFileName.tar.gz folderToCompress
everything in folderToCompress will go to compressFileName
Edit: After review and comments I realized that people may get confused with compressFileName without an extension. If you want you can use .tar.gz extension(as suggested) with the compressFileName
Answer from amitchhajer on Stack Overflowtar -zcvf compressFileName.tar.gz folderToCompress
everything in folderToCompress will go to compressFileName
Edit: After review and comments I realized that people may get confused with compressFileName without an extension. If you want you can use .tar.gz extension(as suggested) with the compressFileName
there are lots of compression methods that work recursively command line and its good to know who the end audience is.
i.e. if it is to be sent to someone running windows then zip would probably be best:
zip -r file.zip folder_to_zip
unzip filenname.zip
for other linux users or your self tar is great
tar -cvzf filename.tar.gz folder
tar -cvjf filename.tar.bz2 folder # even more compression
#change the -c to -x to above to extract
One must be careful with tar and how things are tarred up/extracted, for example if I run
cd ~
tar -cvzf passwd.tar.gz /etc/passwd
tar: Removing leading `/' from member names
/etc/passwd
pwd
/home/myusername
tar -xvzf passwd.tar.gz
this will create /home/myusername/etc/passwd
unsure if all versions of tar do this:
Removing leading `/' from member names
gzip will always compress each file into a single .gz file when given a list of files on its command line.
For example
$ gzip -r log.2011
to recursively walk the log.2011 directory and compress all files it finds, or
$ gzip log.2011/*
to compress only the files in the log.2011 directory without descending into subdirectories.
This will output a gz archive for each matching file, files are replaced by the archive:
gzip fileprefix*
Have a look to the '-r' flag too.
Can I zip an entire folder using gzip? - Unix & Linux Stack Exchange
linux - GZip every file separately - Stack Overflow
linux - How to gzip all files in all sub-directories in bash - Stack Overflow
How do I gzip thousand of files in one directory ?
No.
Unlike zip, gzip functions as a compression algorithm only.
Because of various reasons some of which hearken back to the era of tape drives, Unix uses a program named tar to archive data, which can then be compressed with a compression program like gzip, bzip2, 7zip, etc.
In order to "zip" a directory, the correct command would be
tar -zcvf archive.tar.gz directory/
This will tell tar to
compress it using the z (gzip) algorithm
c (create) an archive from the files in
directory(taris recursive by default)v (verbosely) list (on /dev/stderr so it doesn't affect piped commands) all the files it adds to the archive.
and store the output as a f (file) named
archive.tar.gz
The tar command offers gzip support (via the -z flag) purely for your convenience. The gzip command/lib is completely separate. The command above is effectively the same as
tar -cv directory | gzip > archive.tar.gz
To decompress and unpack the archive into the current directory you would use
tar -zxvf archive.tar.gz
That command is effectively the same as
gunzip < archive.tar.gz | tar -xv
tar has many, many, MANY other options and uses as well; I heartily recommend reading through its manpage sometime.
The gzip command will not recursively compress a directory into a single zip file, when using the -r switch. Rather it will walk that directory structure and zip each file that it finds into a separate file.
Example
before
$ tree dir1/
dir1/
|-- dir11
| |-- file11
| |-- file12
| `-- file13
|-- file1
|-- file2
`-- file3
now run the gzip command
$ gzip -r dir1
after
$ tree dir1/
dir1/
|-- dir11
| |-- file11.gz
| |-- file12.gz
| `-- file13.gz
|-- file1.gz
|-- file2.gz
`-- file3.gz
If you'd prefer to zip up the directory structure then you'll likely want to use the tar command, and then compress the resulting .tar file.
$ tar zcvf dir1.tar.gz dir1/
Example
$ tar zcvf dir1.tar.gz dir1/
dir1/
dir1/file1
dir1/file2
dir1/dir11/
dir1/dir11/file11.gz
dir1/dir11/file12.gz
dir1/dir11/file13.gz
dir1/file3
Which results in the following single file:
$ ls -l | grep tar
-rw-rw-r-- 1 saml saml 271 Oct 1 08:07 dir1.tar.gz
You can confirm its contents:
$ tar ztvf dir1.tar.gz
drwxrwxr-x saml/saml 0 2013-10-01 08:05 dir1/
-rw-rw-r-- saml/saml 0 2013-10-01 07:45 dir1/file1
-rw-rw-r-- saml/saml 0 2013-10-01 07:45 dir1/file2
drwxrwxr-x saml/saml 0 2013-10-01 08:04 dir1/dir11/
-rw-rw-r-- saml/saml 27 2013-10-01 07:45 dir1/dir11/file11.gz
-rw-rw-r-- saml/saml 27 2013-10-01 07:45 dir1/dir11/file12.gz
-rw-rw-r-- saml/saml 27 2013-10-01 07:45 dir1/dir11/file13.gz
-rw-rw-r-- saml/saml 0 2013-10-01 07:45 dir1/file3
You can use gzip *
Note:
- This will zip each file individually and DELETE the original.
- Use
-k(--keep) option to keep the original files. - This may not work if you have a huge number of files due to limits of the shell
- To run gzip in parallel see @MarkSetchell's answer below.
Easy and very fast answer that will use all your CPU cores in parallel:
parallel gzip ::: *
GNU Parallel is a fantastic tool that should be used far more in this world where CPUs are only getting more cores rather than more speed. There are loads of examples that we would all do well to take 10 minutes to read... here
I'd prefer gzip -r ./ which does the same thing but is shorter.
No need for loops or anything more than find and gzip:
find . -type f ! -name '*.gz' -exec gzip "{}" \;
This finds all regular files in and below the current directory whose names don't end with the .gz extension (that is, all files that are not already compressed). It invokes gzip on each file individually.
Edit, based on comment from user unknown:
The curly braces ({}) are replaced with the filename, which is passed directly, as a single word, to the command following -exec as you can see here:
$ touch foo
$ touch "bar baz"
$ touch xyzzy
$ find . -exec echo {} \;
./foo
./bar baz
./xyzzy
I would use
find /path/to/dir \( -name '*.css' -o -name '*.html' \) -exec gzip --verbose --keep {} \;
Change name to iname if you want to match the extensions case-insensitively (i.e. include .CSS and/or .HTML extensions). You can omit the /path/to/dir if you want to start the recursive search from the current directory.
you can do that with a for loop to find every file then compress it:
for i in `find | grep -E "\.css$|\.html$"`; do gzip "$i" ; done
Putting every file into a separate tar file doesn't make any sense in this scenario. You can use gzip to compress them directly:
gzip *
will result in file1.out.gz, file2.out.gz etc.
You would use tar only if you would need a compressed archive as a single file.
If you ineed need a tar archive for every file, you can create it like so:
for i in *; do tar -czf $i.tar.gz $i; done
To build on @SvenW's answer (which will only work on the current directory), if you have a HUGE number of files or want to do it on a recursive directory structure you can also use
find . -type f -exec gzip \{\} \;
and if you need to put the output into a different directory (in this example, ../target) and don't want to remove the originals, you can do something like:
find . -type f -print | while read fname ; do
mkdir -p "../target/`dirname \"$fname\"`"
gzip -c "$fname" > "../target/$fname.gz"
done
I have a directory which contains about 17GB of files. There's so many files that when I run gzip with a glob pattern, it would not proceed.
$ gzip *.json -bash: /bin/gzip: Argument list too long
I know the solution is probably to chunk the list of files. What would you suggest I do ?
Two questions: 1) What did I do wrong with the syntax?
Gzip only compresses individual files; it's not an archiving tool. It's usually used in combination with something like tar. In fact, some versions of tar will use gzip to automatically create a compressed archive if given appropriate flags. For example:
tar -cvz -f public_html.tar.gz /home/site/public_html/
This creates (-c) a gzip-compressed (-z) archive called public_html.tar.gz containing the contents of the specified public_html directory.
2) How can I revert all of the filenames back to what they were before (exactly what they are now, minus the .gz extension).
Just run gunzip on all the files. E.g:
gunzip -r /home/site/public_html/
Note that you can also simply use the zip command, which will create a single compressed archive.
gzip will not combine files.. It is not like zip. You need to use tar with gzip
tar cf file.tar dir
gzip file.tar
or
tar zcf file.tar.gz dir