You can use gzip *
Note:
- This will zip each file individually and DELETE the original.
- Use
-k(--keep) option to keep the original files. - This may not work if you have a huge number of files due to limits of the shell
- To run gzip in parallel see @MarkSetchell's answer below.
You can use gzip *
Note:
- This will zip each file individually and DELETE the original.
- Use
-k(--keep) option to keep the original files. - This may not work if you have a huge number of files due to limits of the shell
- To run gzip in parallel see @MarkSetchell's answer below.
Easy and very fast answer that will use all your CPU cores in parallel:
parallel gzip ::: *
GNU Parallel is a fantastic tool that should be used far more in this world where CPUs are only getting more cores rather than more speed. There are loads of examples that we would all do well to take 10 minutes to read... here
gzip will always compress each file into a single .gz file when given a list of files on its command line.
For example
$ gzip -r log.2011
to recursively walk the log.2011 directory and compress all files it finds, or
$ gzip log.2011/*
to compress only the files in the log.2011 directory without descending into subdirectories.
This will output a gz archive for each matching file, files are replaced by the archive:
gzip fileprefix*
Have a look to the '-r' flag too.
Can I zip an entire folder using gzip? - Unix & Linux Stack Exchange
linux - How to gzip all files in all sub-directories in bash - Stack Overflow
How do I gzip thousand of files in one directory ?
gzip - gunzip all .gz files in directory - Unix & Linux Stack Exchange
tar -zcvf compressFileName.tar.gz folderToCompress
everything in folderToCompress will go to compressFileName
Edit: After review and comments I realized that people may get confused with compressFileName without an extension. If you want you can use .tar.gz extension(as suggested) with the compressFileName
there are lots of compression methods that work recursively command line and its good to know who the end audience is.
i.e. if it is to be sent to someone running windows then zip would probably be best:
zip -r file.zip folder_to_zip
unzip filenname.zip
for other linux users or your self tar is great
tar -cvzf filename.tar.gz folder
tar -cvjf filename.tar.bz2 folder # even more compression
#change the -c to -x to above to extract
One must be careful with tar and how things are tarred up/extracted, for example if I run
cd ~
tar -cvzf passwd.tar.gz /etc/passwd
tar: Removing leading `/' from member names
/etc/passwd
pwd
/home/myusername
tar -xvzf passwd.tar.gz
this will create /home/myusername/etc/passwd
unsure if all versions of tar do this:
Removing leading `/' from member names
Putting every file into a separate tar file doesn't make any sense in this scenario. You can use gzip to compress them directly:
gzip *
will result in file1.out.gz, file2.out.gz etc.
You would use tar only if you would need a compressed archive as a single file.
If you ineed need a tar archive for every file, you can create it like so:
for i in *; do tar -czf
i; done
To build on @SvenW's answer (which will only work on the current directory), if you have a HUGE number of files or want to do it on a recursive directory structure you can also use
find . -type f -exec gzip \{\} \;
and if you need to put the output into a different directory (in this example, ../target) and don't want to remove the originals, you can do something like:
find . -type f -print | while read fname ; do
mkdir -p "../target/`dirname \"$fname\"`"
gzip -c "$fname" > "../target/$fname.gz"
done
No.
Unlike zip, gzip functions as a compression algorithm only.
Because of various reasons some of which hearken back to the era of tape drives, Unix uses a program named tar to archive data, which can then be compressed with a compression program like gzip, bzip2, 7zip, etc.
In order to "zip" a directory, the correct command would be
tar -zcvf archive.tar.gz directory/
This will tell tar to
compress it using the z (gzip) algorithm
c (create) an archive from the files in
directory(taris recursive by default)v (verbosely) list (on /dev/stderr so it doesn't affect piped commands) all the files it adds to the archive.
and store the output as a f (file) named
archive.tar.gz
The tar command offers gzip support (via the -z flag) purely for your convenience. The gzip command/lib is completely separate. The command above is effectively the same as
tar -cv directory | gzip > archive.tar.gz
To decompress and unpack the archive into the current directory you would use
tar -zxvf archive.tar.gz
That command is effectively the same as
gunzip < archive.tar.gz | tar -xv
tar has many, many, MANY other options and uses as well; I heartily recommend reading through its manpage sometime.
The gzip command will not recursively compress a directory into a single zip file, when using the -r switch. Rather it will walk that directory structure and zip each file that it finds into a separate file.
Example
before
$ tree dir1/
dir1/
|-- dir11
| |-- file11
| |-- file12
| `-- file13
|-- file1
|-- file2
`-- file3
now run the gzip command
$ gzip -r dir1
after
$ tree dir1/
dir1/
|-- dir11
| |-- file11.gz
| |-- file12.gz
| `-- file13.gz
|-- file1.gz
|-- file2.gz
`-- file3.gz
If you'd prefer to zip up the directory structure then you'll likely want to use the tar command, and then compress the resulting .tar file.
$ tar zcvf dir1.tar.gz dir1/
Example
$ tar zcvf dir1.tar.gz dir1/
dir1/
dir1/file1
dir1/file2
dir1/dir11/
dir1/dir11/file11.gz
dir1/dir11/file12.gz
dir1/dir11/file13.gz
dir1/file3
Which results in the following single file:
$ ls -l | grep tar
-rw-rw-r-- 1 saml saml 271 Oct 1 08:07 dir1.tar.gz
You can confirm its contents:
$ tar ztvf dir1.tar.gz
drwxrwxr-x saml/saml 0 2013-10-01 08:05 dir1/
-rw-rw-r-- saml/saml 0 2013-10-01 07:45 dir1/file1
-rw-rw-r-- saml/saml 0 2013-10-01 07:45 dir1/file2
drwxrwxr-x saml/saml 0 2013-10-01 08:04 dir1/dir11/
-rw-rw-r-- saml/saml 27 2013-10-01 07:45 dir1/dir11/file11.gz
-rw-rw-r-- saml/saml 27 2013-10-01 07:45 dir1/dir11/file12.gz
-rw-rw-r-- saml/saml 27 2013-10-01 07:45 dir1/dir11/file13.gz
-rw-rw-r-- saml/saml 0 2013-10-01 07:45 dir1/file3
I would use
find /path/to/dir \( -name '*.css' -o -name '*.html' \) -exec gzip --verbose --keep {} \;
Change name to iname if you want to match the extensions case-insensitively (i.e. include .CSS and/or .HTML extensions). You can omit the /path/to/dir if you want to start the recursive search from the current directory.
you can do that with a for loop to find every file then compress it:
for i in `find | grep -E "\.css$|\.html$"`; do gzip "$i" ; done
I'd prefer gzip -r ./ which does the same thing but is shorter.
No need for loops or anything more than find and gzip:
find . -type f ! -name '*.gz' -exec gzip "{}" \;
This finds all regular files in and below the current directory whose names don't end with the .gz extension (that is, all files that are not already compressed). It invokes gzip on each file individually.
Edit, based on comment from user unknown:
The curly braces ({}) are replaced with the filename, which is passed directly, as a single word, to the command following -exec as you can see here:
$ touch foo
$ touch "bar baz"
$ touch xyzzy
$ find . -exec echo {} \;
./foo
./bar baz
./xyzzy
I have a directory which contains about 17GB of files. There's so many files that when I run gzip with a glob pattern, it would not proceed.
$ gzip *.json -bash: /bin/gzip: Argument list too long
I know the solution is probably to chunk the list of files. What would you suggest I do ?
How about just this?
$ gunzip *.txt.gz
gunzip will create a gunzipped file without the .gz suffix and remove the original file by default (see below for details). *.txt.gz will be expanded by your shell to all the files matching.
This last bit can get you into trouble if it expands to a very long list of files. In that case, try using find and -exec to do the job for you.
From the man page gzip(1):
gunzip takes a list of files on its command line and replaces each file whose name ends with .gz, -gz, .z, -z, or _z (ignoring case) and which begins with the correct magic number with an uncompressed file without the original extension.
Note about 'original name'
gzip can store and restore the filename used at compression time. Even if you rename the compressed file, you can be surprised to find out it restores to the original name again.
From the gzip manpage:
By default, gzip keeps the original file name and timestamp in the compressed file. These are used when decompressing the file with the
-Noption. This is useful when the compressed file name was truncated or when the time stamp was not preserved after a file transfer.
And these file names stored in metadata can also be viewed with file:
$ echo "foo" > myfile_orig
$ gzip myfile_orig
$ mv myfile_orig.gz myfile_new.gz
$ file myfile_new.gz
myfile_new.gz: gzip compressed data, was "myfile_orig", last modified: Mon Aug 5 08:46:39 2019, from Unix
$ gunzip myfile_new.gz # gunzip without -N
$ ls myfile_*
myfile_new
$ rm myfile_*
$ echo "foo" > myfile_orig
$ gzip myfile_orig
$ mv myfile_orig.gz myfile_new.gz
# gunzip with -N
$ gunzip -N myfile_new.gz # gunzip with -N
$ ls myfile_*
myfile_orig
Use this command to gunzip (unzip gz files) all files in the current directory and keep the original ones:
gunzip -k *.gz
Two questions: 1) What did I do wrong with the syntax?
Gzip only compresses individual files; it's not an archiving tool. It's usually used in combination with something like tar. In fact, some versions of tar will use gzip to automatically create a compressed archive if given appropriate flags. For example:
tar -cvz -f public_html.tar.gz /home/site/public_html/
This creates (-c) a gzip-compressed (-z) archive called public_html.tar.gz containing the contents of the specified public_html directory.
2) How can I revert all of the filenames back to what they were before (exactly what they are now, minus the .gz extension).
Just run gunzip on all the files. E.g:
gunzip -r /home/site/public_html/
Note that you can also simply use the zip command, which will create a single compressed archive.
gzip will not combine files.. It is not like zip. You need to use tar with gzip
tar cf file.tar dir
gzip file.tar
or
tar zcf file.tar.gz dir
gzip will compress 1+ files, though not meant to function like an archive utility. The posted cmd-line would yield N compressed file images concatenated to stdout, redirected to the named output file; unfortunately stuff like filenames and any dirs would not be recorded. A pair like this should work:
(create)
tar -czvf dir.tar.gz <some-dir>
(extract)
tar -xzvf dir.tar.gz
As others have already mentioned, gzip is a file compression tool and not an archival tool. It cannot work with directories. When you run it with -r, it will find all files in a directory hierarchy and compress them, i.e. replacing path/to/file with path/to/file.gz. When you pass -c the gzip output is written to stdout instead of creating files. You have effectively created one big file which contains several gzip-compressed files.
Now, you could look for the gzip file header/magic number, which is 1f8b and then reconstruct your files manually.
The sensible thing to do now is to create backups (if you haven't already). Backups always help (especially with problems such as yours). Create a backup of your directory.gz file now. Then read on.
Fortunately, there's an easier way than manually reconstructing all files: using binwalk, a forensics utility which can be used to extract files from within other files. I tried it with a test file, which was created the same way as yours. Running binwalk -e file.gz will create a folder with all extracted files. It even manages to reconstruct the original file names. The hierarchy of the directories is probably lost. But at least you have your file contents and their names back. Good luck!
Remember: backups are essential.
(For completeness' sake: What you probably intended to run: tar czf directory.tar.gz directory and then tar xf directory.tar.gz)