To launch a program in a non blocking way but still being able to see the output of program, the program has to be launched in a separate thread or process. Ryan has posted a nice code sample here: Python Subprocess.Popen from a thread

Keep in mind that the last line print myclass.stdout will print the output how it appears at that time. If program is just being launched, it might not have output anything at all, so your code should probably read from myclass.stdout until it receives the line you need.

Answer from fest on Stack Overflow
Top answer
1 of 2
1

To launch a program in a non blocking way but still being able to see the output of program, the program has to be launched in a separate thread or process. Ryan has posted a nice code sample here: Python Subprocess.Popen from a thread

Keep in mind that the last line print myclass.stdout will print the output how it appears at that time. If program is just being launched, it might not have output anything at all, so your code should probably read from myclass.stdout until it receives the line you need.

2 of 2
0

You can run it in a thread (so that it doesn't block your code from running), and get the output until you get the second line, then wait for it to terminate. This is an example that will read the output from the command dir /s on Windows to get all the directory listing.

import subprocess, thread, time

def run():
    global can_break

    args = ["dir", "/s"]
    shell = True

    count = 0
    popen = subprocess.Popen(args, shell=shell, stdout=subprocess.PIPE)
    while True:
        line = popen.stdout.readline()
        if line == "": continue
        count += 1
        if count == 2:
            do_something_with(line)
            break

    print "We got our line, we are waiting now"
    popen.wait()
    print "Done."
    can_break = True

def do_something_with(line):
    print '>>> This is it:', line

thread.start_new_thread(run, tuple())

can_break = False
while not can_break:
    print 'Wait'
    time.sleep(1)

print 'Okay!'

Output will look like:

Wait
>>> This is it:  Volume Serial Number is XXXX-XXXX

We got our line, we are waiting now
Wait
Wait
Wait
.
.
.
Done.
Wait
Okay!
🌐
GitHub
gist.github.com › keymon › 1498154
Implements run a process in background from python, capturing the output in a buffer and allowing set callbacks for the output. · GitHub
Save keymon/1498154 to your computer and use it in GitHub Desktop. ... Implements run a process in background from python, capturing the output in a buffer and allowing set callbacks for the output.
🌐
Reddit
reddit.com › r/learnpython › how can i make use subprocess to run process in background and write output to a file line by line?
r/learnpython on Reddit: How can I make use subprocess to run process in background and write output to a file line by line?
February 13, 2019 -

I am wondering if this can be achieved with subprocess. Run the process and write output

I have two files:

main.py:

import subprocess
import shlex


def main():
	command = 'python test_output.py'
	logfile =  open('output', 'w')
	proc = subprocess.Popen(shlex.split(command), stdout=logfile)


if __name__ == "__main__":
	main()

and test_output.py:

from time import sleep
import os

for i in range(0, 30):
	print("Slept for => ", i+1, "s")
	sleep(1)
os.system("notify-send completed -t 1500")

The output of the process is written to logfile once the child process is completed. Is there any way to:

  • Start child process from main and exit it (like it does now).

  • Keep running the child process in background.

  • As child process produces an output, write it immediately to logfile. (Don't wait for the child process to finish, as it does now.)

  • Or write to Named Pipes alternatively but don't keep it waiting to read (non-blocking)

Is it possible to do everything in background, without keeping main.py waiting?

🌐
Janakiev
janakiev.com › blog › python-background
Running a Python Script in the Background - njanakiev
April 12, 2022 - You can also hardcode the path of your Python interpreter (e.g. #!/usr/bin/python3), but this is not flexible and not portable on other machines. Next, you’ll need to set the permissions of the file to allow execution: ... Now you can run the script with nohup which ignores the hangup signal. This means that you can close the terminal without stopping the execution. Also, don’t forget to add & so the script runs in the background: ... It is also possible to kill the process by using pkill, but make sure you check if there is not a different script running with the same name:
🌐
GeeksforGeeks
geeksforgeeks.org › python › running-python-program-in-the-background
Running Python program in the background - GeeksforGeeks
August 21, 2020 - If you close terminal before the end of the program, all processes executed by the terminal will stop, A hangup situation is arising in order to counter problem you need to use nohup command as shown below nohup will assure that process is running ...
Find elsewhere
🌐
Medium
medium.com › @maheshwar.ramkrushna › running-python-scripts-as-background-processes-using-subprocess-pythonw-exe-and-other-methods-ed5316dd5256
Running Python Scripts as Background Processes: Using subprocess, pythonw.exe, and Other Methods | by Ramkrushna Maheshwar | Medium
February 5, 2025 - In this example, both standard output and error messages from script.py will be written to script_output.log instead of being shown in the terminal. There are various ways to run Python scripts in the background on Windows, each serving a different need:
🌐
Dataquest
dataquest.io › home › blog › python subprocess: the simple beginner's tutorial
Python Subprocess: The Simple Beginner's Tutorial (2023)
February 19, 2025 - The run function of the subprocess module in Python is a great way to run commands in the background without worrying about opening a new terminal or running the command manually.
🌐
Indigodomo
forums.indigodomo.com › indigo home › board index › indigo help › extending indigo › plugin sdk and development
Starting and communicating with a background process - Indigo Domotics
January 11, 2021 - It appears that you need to tell the spawned python process to not buffer its output by passing the "-u" flag. My demo below works with the "-u" and it does nothing at all without the flag. Here is my quick example: ... [Aarons-MacBook-Pro:~/experiments/python_proc_comms] Aaron% cat runner.py import time def main(): while True: print(time.time()) time.sleep(1) if __name__ == "__main__": main()
🌐
Delft Stack
delftstack.com › home › howto › python › python background process
Background Process in Python | Delft Stack
October 10, 2023 - Now, if we created an important script for which we want to check how it is working and get an update about the script that we are running as a background process, we can log or flush the output from that script inside another file using the simple command. The command to log the output from our background process is shown below. ... If we want to log the output from our background process in Linux or Mac devices, we can use the following command below. # python python backgroundprocess.py > backgroundlog &
🌐
Medium
aws.plainenglish.io › set-up-background-execution-for-your-python-scripts-in-just-a-few-steps-de59a42d5b61
Set Up Background Execution for Your Python Scripts in Just a Few Steps | by The Devops Girl | AWS in Plain English
February 3, 2025 - The simplest way to run your Python script in the background is by appending `&` at the end of your command. This command allows the script to execute in the background, freeing up your terminal for other tasks. ... - What Happens? The script starts running in the background, and you’ll ...
Top answer
1 of 11
535

While jkp's solution works, the newer way of doing things (and the way the documentation recommends) is to use the subprocess module. For simple commands its equivalent, but it offers more options if you want to do something complicated.

Example for your case:

import subprocess
subprocess.Popen(["rm","-r","some.file"])

This will run rm -r some.file in the background. Note that calling .communicate() on the object returned from Popen will block until it completes, so don't do that if you want it to run in the background:

import subprocess
ls_output=subprocess.Popen(["sleep", "30"])
ls_output.communicate()  # Will block for 30 seconds

See the documentation here.

Also, a point of clarification: "Background" as you use it here is purely a shell concept; technically, what you mean is that you want to spawn a process without blocking while you wait for it to complete. However, I've used "background" here to refer to shell-background-like behavior.

2 of 11
107

Note: This answer is less current than it was when posted in 2009. Using the subprocess module shown in other answers is now recommended in the docs

(Note that the subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using these functions.)


If you want your process to start in the background you can either use system() and call it in the same way your shell script did, or you can spawn it:

import os
os.spawnl(os.P_DETACH, 'some_long_running_command')

(or, alternatively, you may try the less portable os.P_NOWAIT flag).

See the documentation here.