Thanks for your help!
I just have found the solution:
Part 1
(Thanks to the answer of dessert)
git by design does never write to stdout but stderr. So I needed to redirect stderr, too, in order to get the output using
git clone XYZ &> git_clone.file
Part 2
Anyway this wasn't enough and I only received the "uninteresting" part of the output to the file but not the lines of the progress I really wanted.
Doing further reserach again in man git-clone I realized there exists an option
--progress
progress status is reported on the standard error stream by
default when it is attached to a terminal, unless -q is
specified. This flag forces progress status even if the standard
error stream is not directed to a terminal.
Though I'ld think it actually was already attached to a terminal, this now seems to force git to write the lines of the progress part I'm most interested in finally to stderr as well so I can now get them using
git clone --progress XYZ &> git_clone.file
Answer from derHugo on askubuntu.comThanks for your help!
I just have found the solution:
Part 1
(Thanks to the answer of dessert)
git by design does never write to stdout but stderr. So I needed to redirect stderr, too, in order to get the output using
git clone XYZ &> git_clone.file
Part 2
Anyway this wasn't enough and I only received the "uninteresting" part of the output to the file but not the lines of the progress I really wanted.
Doing further reserach again in man git-clone I realized there exists an option
--progress
progress status is reported on the standard error stream by
default when it is attached to a terminal, unless -q is
specified. This flag forces progress status even if the standard
error stream is not directed to a terminal.
Though I'ld think it actually was already attached to a terminal, this now seems to force git to write the lines of the progress part I'm most interested in finally to stderr as well so I can now get them using
git clone --progress XYZ &> git_clone.file
git clone uses stderr for the output, so just write that to the file:
git clone https://github.com/someRepository 2>git_clone.file
Alternatively you could redirect both stdout and stderr – that's not necessary in this particular, but this way you make sure every output produces by the command gets redirected:
git clone https://github.com/someRepository &>git_clone.file
In the case of git clone obviously there's a different output if you redirect it, the whole progress information running through in the terminal is not included in an output file. That's by design and IIRC you can't easily change that behaviour directly, however if you need the output in another script you may very well pipe it to it, which works just fine and gives you all the output:
git clone https://github.com/someRepository | cat
Inside your script you can get stdin with -, e.g. cat - to print stdin to stdout – see here for more: How to write a script that accepts input from a file or from stdin? and How to read from a file or stdin in Bash?.
Pass results of git status to command line argument
How do I export a Git log to a text file? - Stack Overflow
terminology - Understanding the output of git status with the short flag - Stack Overflow
beginner - Bash script for referencing git status output files - Code Review Stack Exchange
I'm trying to find a tool I ran across years ago that makes it easy to pass results of git status to command line arguments. For example I run git status to see what file has changed and then I want to open one of those files. This was a tool that made it easy to pass the file name as an argument for my next command. Or similar example for running checkout -- to reset the file.
"the README file is modified in the working directory but not yet staged": We're not tracking this file. Still, Git understands that it has been modified. From the last snapshot.
No, this is wrong: specifically, the file is staged (and the staged copy matches the HEAD copy, and this makes the file tracked). The tricky part with the staging-area is that it is normally mostly invisible. This leads people down the wrong path in trying to understand how it works.
First, let's tackle some Git terminology. There are three entities of interest at this point: the current commit, the staging-area—which actually has three names—and the work-tree. The three names for the staging-area are index, staging-area, and cache, and these three names reflect the low quality of Linus Torvald's original choice ("index") or the enormous importance of the invisible staging-area, or both. (I think both.) Let's look deeper at each one:
The current commit, which we can also name via the name
HEAD(in all capitals1), is of course a commit—it's a snapshot of all the files that were in the staging area when you (or whoever) rangit commit. This snapshot is permanent (mostly) and read-only (entirely). Its true name is notHEAD—that's just a symbolic name by which we can find it right now—but rather some big ugly hash ID. The hash ID appears random, but is in fact a cryptographic checksum of the complete contents of the commit. That's why the commit can't be changed—changing anything would change the checksum, resulting in a different commit.The files stored within2 the commit are also read-only. They are stored in a special, Git-only, compressed form. This particular compression has the nice property that if the contents of a file are the same from one commit to another, these commits all share the underlying compressed file-image. That means you can commit a big file millions of times, if you like, and not use any more space than committing that file once.
The index / staging-area / cache is this crazy almost-invisible data structure. It contains all the files at all times, in the same way that a commit contains all the files. The files in the index are also in this special compressed Git-only format. The key difference between a file copy in the index / staging-area, and a copy in a commit, is that the index one can be overwritten.
(The index also caches—hence the name "cache"—information about the work-tree, to make Git go faster. These two facts, that the index holds all the files all ready to go into the next commit, and that it caches stuff about the work-tree, are what make
git commitso insanely fast, compared to other similar version control systems.)The work-tree is the simplest of the three, but in a sense, also the one Git cares the least about. It's where you do your work on your files. These files are in the ordinary format that the rest of your computer programs understand. They are the most important to you, but the least important to Git: a
--barerepository has no work-tree, but Git can still function (in a more limited way of course).
The work-tree is the only one of these three things that you can see easily and directly. Simply use whatever command it is that lists files or views files: there they are, plain to see. Fortunately, commits are easy to see as well, by checking them out.
When you initially check out some particular commit—via git checkout master or git checkout develop, for instance—Git populates both your index / staging-area and you work-tree from that commit. It sets HEAD to be a symbolic name for the correct hash ID. That way, the index already has in it all the same files that are in the HEAD commit, and the work-tree has all the same files that are in the index.
If you modify a file in the work-tree, and then run git add on it, Git copies the work-tree version of that file into the index / staging-area. Now the HEAD commit version and the index version differ, but the index version and the work-tree version agree with each other.
If you modify a file in the work-tree but don't run git add on it, the HEAD and index versions agree, but the index version disagrees with the work-tree version.
If you modify a file in the work-tree, then (1) use git add to copy it to the index / staging-area and (2) modify it again, now all three versions of that file differ. This is where you will see an MM status.
What git status is doing is, in effect, running two diffs. The first one compares HEAD to the index. Whatever is different here is "staged for commit". The second diff compares the index to the work-tree. Whatever is different here is "not staged for commit". That's almost it—we're nearly done!
Last, let's take a look at the term tracked as applied to files. In Git, a file is tracked if and only if it is in the index / staging-area. It's really that simple! The tricky part is telling whether a file is in fact in the index, since it's normally so invisible there.
The git status command compares the index: first, it compares HEAD vs index. Suppose some file is in both HEAD and index and has the same contents in both. Then you won't see it here. Likewise, you won't see it here if it's the same in the index and the work-tree. So if the file is in the index, but matches both HEAD and work-tree versions, it's invisible.
Suppose some file isn't in the index. If it's in HEAD, git status will tell you that between HEAD and the index, the file got deleted—a D in the first column of the short output. So in that case you can tell: the file has gone away from the index, and is no longer tracked. It won't be in the next commit.
Suppose some file isn't in HEAD, but is in the index. In this case git status will tell you that between HEAD and the index, the file got added—an A in the first column of the short output. So in that case you can tell that the file is now tracked, and will be in the next commit.
The tricky case occurs when a file is both untracked and ignored, because now, if the file is not in the HEAD commit (and by definition it's not in the index—we just said it was untracked), the first column can't tell you anything: it's not in either of those two entities, so Git says nothing here. The second column could tell you that the index and work-tree don't match, if the file exists in the work-tree, but since you told Git that the untracked work-tree file should be ignored, git status won't mention it here either.
Finally, there are a few things worth mentioning:
You can actually view the index. Run
git ls-files --stageto see a quick view of most of what is actually in the staging-area. This is impractical in a big project, precisely because the staging-area holds a copy of every file—well, every file that will be committed. That can be tens of thousands of files. It's much more useful to view the difference between theHEADcommit and the index / staging-area, so that's whatgit statusdoes (in the first column of--shortoutput).You can also view the contents of a commit directly. Run
git ls-tree -r HEADto see all of the committed files. The output is similar togit ls-files --stage. (It adds the Git object type name and takes away the staging number, and uses a tree structure rather than the index's flattened-tree.) As withgit ls-files --stagethis is mainly useful for debugging Git or writing fancy new commands, not for regular work.
The key here is that git status summarizes the state of the three entities of interest, by comparing HEAD to the index, and then comparing the index to the work-tree. The two columns show you the differences between them, stripped down to just a letter code and a file name. Even though the next commit will be a snapshot of every file that is in the index / staging-area at that time, it's more useful to tell you what's different about that snapshot, as compared to the current snapshot, or the potential snapshot you could make by copying work-tree files into the index.
1On Windows and MacOS where opening a file named readme.txt opens an existing file named README.TXT (and vice versa), you can use lowercase, but Git has various places where it hard-codes the all-capitals HEAD string, so it's best to stick with that. If you don't like typing that much, the character @ is a synonym for HEAD.
2Technically, a commit stores the hash ID of a tree object. The tree object stores each file's name, mode (100644 or 100755), and content-hash-ID, along with names and hash IDs for subtrees as needed. Hence the file contents are not actually inside the commit, but rather laid out as blob objects, right alongside commit and tree objects. This is the mechanism by which commits—and the index!—share blob objects so that however many snapshots you have of a big file, you really only have one copy in the repository database.
You can read about what the letters means here :
https://www.git-scm.com/docs/git-status#_short_format
If "staging area", "added" or "unmerged" are unfarmiliar to you, maybe you need to deepen your git understanding. Or maybe just stop using the -s flag, which is really simpler to understand.
The git pro git is really git, but you can't pick what you want to read. Read everything, in the correct order. https://git-scm.com/book/en/v2
Nice try
That's a fantastic idea.
It's very annoying to run git status and then very often type some next command and copy paste selected files from the output.
I long wanted to have something like this but never made time to actually do it.
You gave me the final push, so thanks.
Setting variables dynamically
I don't think your script actually works :-)
At least on my computer, in Bash 3 or 4,
it's not possible to create variables dynamically with set,
for example:
m1=
c=1
set m$c=x
echo $m1
This outputs nothing, because set m$c=x does not actually perform m1=x.
After some research, I found that this works with declare:
m1=
c=1
declare m$c=x
echo $m1
This outputs x alright. Unfortunately, according to help declare:
[...] When used in a function, makes NAMEs local, as with the `local' command.
That is, the variables created dynamically this way inside the function will not be visible outside, so this is not usable for this purpose.
One obvious option is eval, but not a good one, because eval is evil.
A well-crafted filename in the working tree could wreak havoc.
Even if this is unlikely, I would rather not resort to such option.
Finally, there's a trick with printf -v that will work well:
m1=
c=1
printf -v m$c x
echo $m1
This outputs x, and even when used in a function, m1 will be visible outside.
The only downside is that printf is not POSIX compliant,
but that's probably not a big problem in practice.
Unnecessary repeated processing of git status -s
This is really quite wasteful:
m=$(echo "$gitsall" | grep "^ M") d=$(echo "$gitsall" | grep "^ D") u=$(echo "$gitsall" | grep "^??") a=$(echo "$gitsall" | grep "^A ") s=$(echo "$gitsall" | grep "^M ")
It would be better to run git status -s just once,
loop over the output,
use a case statement to decide the type of the change,
and set the m, d, u, a, s variables accordingly.
When you go this way, you can also directly create the variables with a count, so that you don't need all those repetitive while loops for each status type.
Arithmetic context
When within $((...)), you don't need to write $ to access the values of variables, for example instead of:
count=$(($count+1))
You can write:
count=$((count+1))
An even shorter equivalent:
((count++))
Current working directory
Instead of $(pwd) it's better to use the $PWD variable.
Local variables
Instead of using the unset command at the end of the function to clear the variables used, it's much better to use local variables, for example:
local m d u a s
Not only this will make sure you don't clear the variables,
it also makes your use correct.
The current use is not correct,
because if some of these variables had value before calling the function,
the values get reset.
When using local, the original values of those variables remain intact in the calling environment.
Alternative implementation
Putting the above together, the function could be written simpler as:
gits() {
git status
mc=1 dc=1 uc=1 ac=1 sc=1
local line status path name
while read -r line; do
[ "$line" ] || continue
status=${line:0:2}
path=${line:3}
case "$status" in
" M") name=m$((mc++)) ;;
" D") name=d$((dc++)) ;;
"??") name=u$((uc++)) ;;
"A ") name=a$((ac++)) ;;
"M ") name=s$((sc++)) ;;
*) echo unsupported status on line: $line
esac
printf -v $name "$path"
done <<< "$(git status -s)"
}
I'll limit myself to points not addressed by janos' critique.
Firstly, when parsing lines using while-read loops, you should ask yourself if you want bash to strip whitespace in the line. In this case, given that we may have lines like M file, the answer is no, and IFS should be set to an empty string. I have edited janos' answer to reflect this critique, as it was a showstopping bug as written.
The other concern I have is the use of dynamically-named variables. While your hack can be made to work, it is pretty magic and will make it harder for the next person reading to understand.
Moreover, I'm not sure what's to be gained by trying to adhere to POSIX -- you're already firmly in Bash territory due to your use of substring indexing, and would be going further there by using printf -v and local as janos suggests (thanks Shellcheck for flagging these!).
Accepting that, we gain access to Bash's data structures, specifically arrays:
gits() {
git status
m=() d=() u=() a=() s=()
local line status path name
while IFS= read -r line; do
[ "$line" ] || continue
status=${line:0:2}
path=${line:3}
case "$status" in
" M") m+=("$path") ;;
" D") d+=("$path") ;;
"??") u+=("$path") ;;
"A ") a+=("$path") ;;
"M ") s+=("$path") ;;
*) echo unsupported status on line: "$line"
esac
done < <(git status -s)
}
Now, we can access the first unstaged modified file as "${m[0]}", no dynamic variables needed (though, given the likely unpredictability of these indeces, I'm guessing you'll probably be using "${m[@]}" instead -- it expands to every entry of m, so can be used eg to populate the list of files to open in vim)
However, I appreciate the reminder of the power of dynamic variable names. Kudos on figuring that one out!
The output of git status --porcelain, designed to be easy to parse in a script, outputs the full paths rather than relative paths regardless of where your current directory is within the tree.
Each line output by git status --porcelain has two leading characters indicating the status of the file (e.g. whether it's untracked, modified, new, deleted, etc.) followed by a space, so if you just want the full paths of everything that would be mentioned in the output of git status you can do:
git status --porcelain | sed s/^...//
I think cut is good for this.
git status -s | cut -c4-