You can use: GitPython
Or 'native way':
from subprocess import Popen, PIPE
cmd = 'git diff --name-only'
p = Popen(cmd, stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate()
files_changed_list = stdout.decode("utf-8").splitlines()
Answer from David Meu on Stack OverflowYou can use: GitPython
Or 'native way':
from subprocess import Popen, PIPE
cmd = 'git diff --name-only'
p = Popen(cmd, stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate()
files_changed_list = stdout.decode("utf-8").splitlines()
You could also use Python's os module.
import os
os.system("git diff --name-only | tee status")
would put the names of the changed files into a file called status.
Erm I think the other answer is better though.
An easier solution would be to use the Python subprocess module to call git. In your case, this would pull the latest code and build:
import subprocess
subprocess.call(["git", "pull"])
subprocess.call(["make"])
subprocess.call(["make", "test"])
Docs:
- subprocess - Python 2.x
- subprocess - Python 3.x
So with Python 3.5 and later, the .call() method has been deprecated.
https://docs.python.org/3.6/library/subprocess.html#older-high-level-api
The current recommended method is to use the .run() method on subprocess.
import subprocess
subprocess.run(["git", "pull"])
subprocess.run(["make"])
subprocess.run(["make", "test"])
Adding this as when I went to read the docs, the links above contradicted the accepted answer and I had to do some research. Adding my 2 cents to hopefully save someone else a bit of time.
» pip install git-adapter
Have you considered using GitPython? It's designed to handle all this nonsense for you.
import git # pip install gitpython
g = git.cmd.Git(git_dir)
g.pull()
To install the module use pip install GitPython
Project can be found here, GitPython
subprocess.Popen expects a list of the program name and arguments. You're passing it a single string, which is (with the default shell=False) equivalent to:
['git pull']
That means that subprocess tries to find a program named literally git pull, and fails to do so: In Python 3.3, your code raises the exception FileNotFoundError: [Errno 2] No such file or directory: 'git pull'. Instead, pass in a list, like this:
import subprocess
process = subprocess.Popen(["git", "pull"], stdout=subprocess.PIPE)
output = process.communicate()[0]
By the way, in Python 2.7+, you can simplify this code with the check_output convenience function:
import subprocess
output = subprocess.check_output(["git", "pull"])
Also, to use git functionality, it's by no way necessary (albeit simple and portable) to call the git binary. Consider using git-python or Dulwich.
GitPython uses the module logging. By adding logging.basicConfig(level=logging.DEBUG) before your code, it prints logs like
DEBUG:git.cmd:Popen(['git', 'fetch'], cwd=E:\path\foo, universal_newlines=False, shell=None)
If you want it to print formatted logs as you expect, you could modify GitPython's source code. I'm using Python 2.7 and my GitPython is installed at C:\Python27\lib\site-packages\git. You can run git.__file__ in REPL to find your installation directory. In the file C:\Python27\lib\site-packages\git\cmd.py, you can find a method def execute. It calls subprocess.Popen to run the command. You can modify the line log.debug before Popen or just insert print(' '.join(cmd)) at a proper line. If you are using Python 3.3 or newer, you could also refer to this answer on how to print the command with subprocess.
You could also:
pull = self.repo.git.pull('--rebase')
log.info(pull)
When not writing to a terminal, git clone doesn't have any output to either stdout or stderr, except on error.
When writing to a terminal, of course, it has lots of output—but that output is progress bars that are continually overwritten. Usually, you don't want that—it's going to be a big mess of control characters and repeated lines.
But if you do want it, there are two options.
First, you can use a PTY (Pseudo-TTY). You can create a PTY with os.openpty, then hand the PTY off explicitly to the child process. Or you can use os.forkpty, which handles forking and automatically hooking up the PTY so all you have to do is call one of the os.exec functions. Or you can use the pty module. (It's not entirely clear which is more portable; openpty and forkpty claim that pty is more portable, and conceptually it's designed that way… but it's also only really tested on Linux.)
Note that git wants the PTY as its stderr, not its stdout.
Alternatively, most git commands have a --progress flag that causes them to write progress to stderr even if it's not a terminal. At least as of the version documented here, this includes clone, but of course you should check the man for your local version. So, that may be all you need. (Also see the --verbose flag.)
However, this may not be as nice. For me, when I provide a PTY with no attached termcaps, I get each line followed by a \r without \n to overwrite it; when I use the --progress option, git detects the termcaps of whatever terminal my script happens to be running it, which means I end up getting ANSI color codes as well as \rs.
Of course either way I'm getting hundreds of useless lines that I have no interest in, but I assume that's what you wanted? (Or maybe you want to use universal_newlines='\r' to translate the '\r' to '\n'? That's slightly cheating, because this is self-overwriting Unix terminal output, and you're pretending it's classic-Mac output… but it works.)
I had a similar issue, but I also wanted real-time progress because the git clone was taking a long time (like 5 minutes), since it was a large git repo. And I wanted to provide real-time feedback to the user.
Here is a Python 3.7 working example:
# This will print stdout/stderr as it comes in
def run_shell_command_with_realtime_output(shell_command_string, working_dir='.'):
# print the command to be executed, just for fun
print("run_shell_command >", shell_command_string)
# run it, this will NOT block
sub_process = subprocess.Popen(shell_command_string,
shell=True,
cwd=working_dir, universal_newlines=True,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
# print the stdout/stderr as it comes in
while True:
# The readline() will block until...
# it reads and returns a string that ends in a '\n',
# or until the process has ended which will result in '' string
output = sub_process.stdout.readline()
if output:
print(output.strip())
elif sub_process.poll() is not None:
break
# get the return code
return_code = sub_process.wait()
# Was there an error?
if return_code != 0:
print("FYI, the subprocess had an error, you might want to do something special...")
# return the sub_process, in case the caller wants to check exit/return codes
return sub_process
It turns out git clone does not seem to write to stdout/stderr like we normally are used to. Instead it uses a pager, which is how it updates the same line, like when it is cloning, and the % is incrementing on the same line.
So you need to call it like this.
# prepare to clone the git repo
git_clone_url_with_credentials = "https://<username>:<password>@bitbucket.org/myreponame.git"
git_clone_with_progress_cmd = "git --no-pager clone --progress {}".format(git_clone_url_with_credentials)
helpers_misc.run_shell_command_with_progress(git_clone_with_progress_cmd)
And that should do it...