The subprocess module is a very good solution.
import subprocess
p = subprocess.Popen([command, argument1,...], cwd=working_directory)
p.wait()
It has also arguments for modifying environment variables, redirecting input/output to the calling program, etc.
Answer from hynekcer on Stack OverflowCan't tell where I messed this up... I am using pycharm and the folder with all of the files on it is on my desktop....
os.system("cd C:\\softwares\\someDirectory\\subdirectory\\bin execute.sql")Am I doing this correctly? If I open up the command prompt and just cd to the directory and run the command it runs... but when I try to CD and run it from my python script it spits out the following error
The system cannot find the path specified.
thoughts?
Python: run shell command in a specific directory - Stack Overflow
python - How to change the working directory for a shell script - Unix & Linux Stack Exchange
Execute shell commands in Python - Unix & Linux Stack Exchange
Can I do a "dir" (OS command) from within the Python shell?
Videos
The subprocess module is a very good solution.
import subprocess
p = subprocess.Popen([command, argument1,...], cwd=working_directory)
p.wait()
It has also arguments for modifying environment variables, redirecting input/output to the calling program, etc.
Try to os.chdir(path) before invoking the command.
From here:
os.chdir(path) Change the current working directory to path.
Availability: Unix, Windows
EDIT
This will change the current working dir, you can get the current working by:
os.getcwd()
If you want to save it and restore it later, if you need to do some work in the original working dir.
EDIT 2
In any case you should probably move to subprocess (doc) as suggested here. If you use subprocess's Popen you have the choice of providing cwd parameter to specify the working directory for the subprocess: read this.
subprocess.Popen(args, bufsize=0, executable=None, stdin=None,
stdout=None, stderr=None, preexec_fn=None, close_fds=False,
shell=False, cwd=None, env=None, universal_newlines=False,
startupinfo=None, creationflags=0)
...
If cwd is not None, the child’s current directory will be changed to cwd before it is executed. Note that this directory is not considered when searching the executable, so you can’t specify the program’s path relative to cwd.
Quick and dirty:
In your start up script instead of just executing the python script, use cd first.
#!/bin/sh
cd /home/username/projectname &&
python ./scriptname.py
There are a couple of ways around this directly in your Python script.
If your script is always going to be in "/home/username/projectname/subfolder", you can simply add that to your search path inside Python:
import sys sys.path.append("/home/username/projectname/subfolder")I suspect, however, that you might have this in multiple "projectname" directories, so a more generic solution is something like this:
import sys import os sys.path.append(os.path.join(os.path.dirname(sys.argv[0]), "subfolder"))This finds the directory where the Python script is (in
sys.argv[0]), extracts the directory part, appends "subfolder" onto it, and puts it into the search path.Note that some operating systems may only give the executable name in
sys.argv[0]. I don't have a good solution for this case, perhaps someone else does. You may also need to inject aos.path.abspath()call in there ifsys.argv[0]has a relative path, but play around with it a bit and you should be able to get it working.Similar to the above answer, you can have the Python script change directories all by itself with no need for a wrapper script:
import os os.chdir("/home/username/projectname")
You can use os.system(), like this:
import os
os.system('ls')
Or in your case:
os.system('echo 1 > /proc/sys/net/ipv4/ip_forward')
os.system('iptables -t nat -A PREROUTING -p tcp --destination-port 80 -j REDIRECT --to-port 8080')
Better yet, you can use subprocess's call, it is safer, more powerful and likely faster:
from subprocess import call
call('echo "I like potatos"', shell=True)
Or, without invoking shell:
call(['echo', 'I like potatos'])
If you want to capture the output, one way of doing it is like this:
import subprocess
cmd = ['echo', 'I like potatos']
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
o, e = proc.communicate()
print('Output: ' + o.decode('ascii'))
print('Error: ' + e.decode('ascii'))
print('code: ' + str(proc.returncode))
I highly recommend setting a timeout in communicate, and also to capture the exceptions you can get when calling it. This is a very error-prone code, so you should expect errors to happen and handle them accordingly.
https://docs.python.org/3/library/subprocess.html
The first command simply writes to a file. You wouldn't execute that as a shell command because python can read and write to files without the help of a shell:
with open('/proc/sys/net/ipv4/ip_forward', 'w') as f:
f.write("1")
The iptables command is something you may want to execute externally. The best way to do this is to use the subprocess module.
import subprocess
subprocess.check_call(['iptables', '-t', 'nat', '-A',
'PREROUTING', '-p', 'tcp',
'--destination-port', '80',
'-j', 'REDIRECT', '--to-port', '8080'])
Note that this method also does not use a shell, which is unnecessary overhead.
Use subprocess.run:
import subprocess
subprocess.run(["ls", "-l"])
Another common way is os.system but you shouldn't use it because it is unsafe if any parts of the command come from outside your program or can contain spaces or other special characters, also subprocess.run is generally more flexible (you can get the stdout, stderr, the "real" status code, better error handling, etc.). Even the documentation for os.system recommends using subprocess instead.
On Python 3.4 and earlier, use subprocess.call instead of .run:
subprocess.call(["ls", "-l"])
Here is a summary of ways to call external programs, including their advantages and disadvantages:
os.systempasses the command and arguments to your system's shell. This is nice because you can actually run multiple commands at once in this manner and set up pipes and input/output redirection. For example:os.system("some_command < input_file | another_command > output_file")However, while this is convenient, you have to manually handle the escaping of shell characters such as spaces, et cetera. On the other hand, this also lets you run commands which are simply shell commands and not actually external programs.
os.popenwill do the same thing asos.systemexcept that it gives you a file-like object that you can use to access standard input/output for that process. There are 3 other variants of popen that all handle the i/o slightly differently. If you pass everything as a string, then your command is passed to the shell; if you pass them as a list then you don't need to worry about escaping anything. Example:print(os.popen("ls -l").read())subprocess.Popen. This is intended as a replacement foros.popen, but has the downside of being slightly more complicated by virtue of being so comprehensive. For example, you'd say:print subprocess.Popen("echo Hello World", shell=True, stdout=subprocess.PIPE).stdout.read()instead of
print os.popen("echo Hello World").read()but it is nice to have all of the options there in one unified class instead of 4 different popen functions. See the documentation.
subprocess.call. This is basically just like thePopenclass and takes all of the same arguments, but it simply waits until the command completes and gives you the return code. For example:return_code = subprocess.call("echo Hello World", shell=True)subprocess.run. Python 3.5+ only. Similar to the above but even more flexible and returns aCompletedProcessobject when the command finishes executing.os.fork,os.exec,os.spawnare similar to their C language counterparts, but I don't recommend using them directly.
The subprocess module should probably be what you use.
Finally, please be aware that for all methods where you pass the final command to be executed by the shell as a string and you are responsible for escaping it. There are serious security implications if any part of the string that you pass can not be fully trusted. For example, if a user is entering some/any part of the string. If you are unsure, only use these methods with constants. To give you a hint of the implications consider this code:
print subprocess.Popen("echo %s " % user_input, stdout=PIPE).stdout.read()
and imagine that the user enters something "my mama didnt love me && rm -rf /" which could erase the whole filesystem.