The subprocess module is a very good solution.
import subprocess
p = subprocess.Popen([command, argument1,...], cwd=working_directory)
p.wait()
It has also arguments for modifying environment variables, redirecting input/output to the calling program, etc.
Answer from hynekcer on Stack OverflowThe subprocess module is a very good solution.
import subprocess
p = subprocess.Popen([command, argument1,...], cwd=working_directory)
p.wait()
It has also arguments for modifying environment variables, redirecting input/output to the calling program, etc.
Try to os.chdir(path) before invoking the command.
From here:
os.chdir(path) Change the current working directory to path.
Availability: Unix, Windows
EDIT
This will change the current working dir, you can get the current working by:
os.getcwd()
If you want to save it and restore it later, if you need to do some work in the original working dir.
EDIT 2
In any case you should probably move to subprocess (doc) as suggested here. If you use subprocess's Popen you have the choice of providing cwd parameter to specify the working directory for the subprocess: read this.
subprocess.Popen(args, bufsize=0, executable=None, stdin=None,
stdout=None, stderr=None, preexec_fn=None, close_fds=False,
shell=False, cwd=None, env=None, universal_newlines=False,
startupinfo=None, creationflags=0)
...
If cwd is not None, the child’s current directory will be changed to cwd before it is executed. Note that this directory is not considered when searching the executable, so you can’t specify the program’s path relative to cwd.
Can't tell where I messed this up... I am using pycharm and the folder with all of the files on it is on my desktop....
os.system("cd C:\\softwares\\someDirectory\\subdirectory\\bin execute.sql")Am I doing this correctly? If I open up the command prompt and just cd to the directory and run the command it runs... but when I try to CD and run it from my python script it spits out the following error
The system cannot find the path specified.
thoughts?
python - How to change the working directory for a shell script - Unix & Linux Stack Exchange
subprocess - Python run shell command over specific folder - Stack Overflow
How to run shell commands in different directory using python subprocess command? - Stack Overflow
Calling a System/Shell command with Python - Using the Subprocess module
Quick and dirty:
In your start up script instead of just executing the python script, use cd first.
#!/bin/sh
cd /home/username/projectname &&
python ./scriptname.py
There are a couple of ways around this directly in your Python script.
If your script is always going to be in "/home/username/projectname/subfolder", you can simply add that to your search path inside Python:
import sys sys.path.append("/home/username/projectname/subfolder")I suspect, however, that you might have this in multiple "projectname" directories, so a more generic solution is something like this:
import sys import os sys.path.append(os.path.join(os.path.dirname(sys.argv[0]), "subfolder"))This finds the directory where the Python script is (in
sys.argv[0]), extracts the directory part, appends "subfolder" onto it, and puts it into the search path.Note that some operating systems may only give the executable name in
sys.argv[0]. I don't have a good solution for this case, perhaps someone else does. You may also need to inject aos.path.abspath()call in there ifsys.argv[0]has a relative path, but play around with it a bit and you should be able to get it working.Similar to the above answer, you can have the Python script change directories all by itself with no need for a wrapper script:
import os os.chdir("/home/username/projectname")
First, you should pass a list to Popen, not a string. Each element of the list is one fragment of the commands you want to run. In your case it should be:
proc = subprocess.Popen(['cd', 'path/to/folder', '&', 'command'])
Second, if your command is a system command like cmd. You need to tell Python to use the system shell.
proc = subprocess.Popen(['cd', 'path/to/folder', '&', 'command'], shell=True)
Third, it looks like you want to capture the output from the command. Right now anything that the command does will be written to screen as if you had run the command from the shell. To get Python to capture the outputs, we need to redirect them from writing to the screen back to Python. We do this using subprocess.PIPE for the stdout and stderr streams.
proc = subprocess.Popen(['cd', 'path/to/folder', '&', 'command'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
Lastly, using Popen returns a Popen object. You don't get the results right away, because the system might be churning away behind the scenes. Using Popen allows your code to keep running (Popen is non-blocking). To get the output, you need to run the communicate method. This returns the output from the output and error streams.
out, err = proc.communicate()
Example:
Change directory and list contents.
proc = subprocess.Popen(['cd', 'Documents/Projects', '&', 'dir'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
out, err = proc.communicate()
print(out.decode('latin-1'))
# prints
Volume in drive C is OS
Volume Serial Number is DE6D-0D88
Directory of C:\Users\james\Documents\Projects
01/04/2020 01:50 PM <DIR> .
01/04/2020 01:50 PM <DIR> ..
11/15/2019 09:05 PM <DIR> .vscode
01/09/2020 11:15 PM <DIR> CondaDeps
01/03/2020 09:22 PM <DIR> Django
12/21/2019 10:52 PM <DIR> FontCluster
12/20/2019 03:56 PM 70 fontcluster.code-workwspace.code-workspace
11/08/2019 03:01 PM <DIR> ScholarCrawler
12/30/2019 10:48 AM 56 scholarcrawler.code-workspace
07/24/2019 09:56 PM <DIR> ThinkStats2
2 File(s) 126 bytes
8 Dir(s) 415,783,694,336 bytes free
The Popen has a cwd keyword argument .
result = subprocess.Popen("command", cwd="path/to/folder")
And the error you are getting is because the Popen's shell keyword argument is set to false by default, so it doesn't know the cd command.
The subprocess methods all accept a cwd keyword argument.
import subprocess
d = subprocess.check_output(
['ls'], cwd='/home/you/Desktop')
Obviously, replace /home/you/Desktop with the actual directory you want.
Most well-written shell commands will not require you to run them in any particular directory, but if that's what you want, this is how you do it.
If this doesn't solve your problem, please update your question to include the actual code which doesn't behave like you expect.
(Of course, a subprocess is a really poor way to get a directory listing, and ls is a really poor way to get a directory listing if you really want to use a subprocess. Probably try os.listdir('/home/you/Desktop') if that's what you actually want. But I'm guessing you are just providing ls as an example of an external command.)
To add to tripleees excellent answer, you can solve this in 3 ways:
- Use subprocess'
cwdargument - Change dir before, using
os.chdir - Run the shell command to the exact dir you want, e.g.
ls /path/to/dirORcd /path/to/dir; ls, but note that some shell directives (e.g. &&, ;) cannot be used without addingshell=Trueto the method call
PS as tripleee commented, using shell=True is not encouraged and there are a lot of things that should be taken into consideration when using it
Use subprocess.run:
import subprocess
subprocess.run(["ls", "-l"])
Another common way is os.system but you shouldn't use it because it is unsafe if any parts of the command come from outside your program or can contain spaces or other special characters, also subprocess.run is generally more flexible (you can get the stdout, stderr, the "real" status code, better error handling, etc.). Even the documentation for os.system recommends using subprocess instead.
On Python 3.4 and earlier, use subprocess.call instead of .run:
subprocess.call(["ls", "-l"])
Here is a summary of ways to call external programs, including their advantages and disadvantages:
os.systempasses the command and arguments to your system's shell. This is nice because you can actually run multiple commands at once in this manner and set up pipes and input/output redirection. For example:os.system("some_command < input_file | another_command > output_file")However, while this is convenient, you have to manually handle the escaping of shell characters such as spaces, et cetera. On the other hand, this also lets you run commands which are simply shell commands and not actually external programs.
os.popenwill do the same thing asos.systemexcept that it gives you a file-like object that you can use to access standard input/output for that process. There are 3 other variants of popen that all handle the i/o slightly differently. If you pass everything as a string, then your command is passed to the shell; if you pass them as a list then you don't need to worry about escaping anything. Example:print(os.popen("ls -l").read())subprocess.Popen. This is intended as a replacement foros.popen, but has the downside of being slightly more complicated by virtue of being so comprehensive. For example, you'd say:print subprocess.Popen("echo Hello World", shell=True, stdout=subprocess.PIPE).stdout.read()instead of
print os.popen("echo Hello World").read()but it is nice to have all of the options there in one unified class instead of 4 different popen functions. See the documentation.
subprocess.call. This is basically just like thePopenclass and takes all of the same arguments, but it simply waits until the command completes and gives you the return code. For example:return_code = subprocess.call("echo Hello World", shell=True)subprocess.run. Python 3.5+ only. Similar to the above but even more flexible and returns aCompletedProcessobject when the command finishes executing.os.fork,os.exec,os.spawnare similar to their C language counterparts, but I don't recommend using them directly.
The subprocess module should probably be what you use.
Finally, please be aware that for all methods where you pass the final command to be executed by the shell as a string and you are responsible for escaping it. There are serious security implications if any part of the string that you pass can not be fully trusted. For example, if a user is entering some/any part of the string. If you are unsure, only use these methods with constants. To give you a hint of the implications consider this code:
print subprocess.Popen("echo %s " % user_input, stdout=PIPE).stdout.read()
and imagine that the user enters something "my mama didnt love me && rm -rf /" which could erase the whole filesystem.
I'm looking to fun httrack to mirror a list of websites from a json file with python