If you want to do the redirection within the Python script, setting sys.stdout to a file object does the trick:
# for python3
import sys
with open('file', 'w') as sys.stdout:
print('test')
A far more common method is to use shell redirection when executing (same on Windows and Linux):
$ python3 foo.py > file
Answer from moinudin on Stack OverflowIf you want to do the redirection within the Python script, setting sys.stdout to a file object does the trick:
# for python3
import sys
with open('file', 'w') as sys.stdout:
print('test')
A far more common method is to use shell redirection when executing (same on Windows and Linux):
$ python3 foo.py > file
There is contextlib.redirect_stdout() function in Python 3.4+:
from contextlib import redirect_stdout
with open('help.txt', 'w') as f:
with redirect_stdout(f):
print('it now prints to `help.text`')
It is similar to:
import sys
from contextlib import contextmanager
@contextmanager
def redirect_stdout(new_target):
old_target, sys.stdout = sys.stdout, new_target # replace sys.stdout
try:
yield new_target # run some code with the replaced stdout
finally:
sys.stdout = old_target # restore to the previous value
that can be used on earlier Python versions. The latter version is not reusable. It can be made one if desired.
It doesn't redirect the stdout at the file descriptors level e.g.:
import os
from contextlib import redirect_stdout
stdout_fd = sys.stdout.fileno()
with open('output.txt', 'w') as f, redirect_stdout(f):
print('redirected to a file')
os.write(stdout_fd, b'not redirected')
os.system('echo this also is not redirected')
b'not redirected' and 'echo this also is not redirected' are not redirected to the output.txt file.
To redirect at the file descriptor level, os.dup2() could be used:
import os
import sys
from contextlib import contextmanager
def fileno(file_or_fd):
fd = getattr(file_or_fd, 'fileno', lambda: file_or_fd)()
if not isinstance(fd, int):
raise ValueError("Expected a file (`.fileno()`) or a file descriptor")
return fd
@contextmanager
def stdout_redirected(to=os.devnull, stdout=None):
if stdout is None:
stdout = sys.stdout
stdout_fd = fileno(stdout)
# copy stdout_fd before it is overwritten
#NOTE: `copied` is inheritable on Windows when duplicating a standard stream
with os.fdopen(os.dup(stdout_fd), 'wb') as copied:
stdout.flush() # flush library buffers that dup2 knows nothing about
try:
os.dup2(fileno(to), stdout_fd) # $ exec >&to
except ValueError: # filename
with open(to, 'wb') as to_file:
os.dup2(to_file.fileno(), stdout_fd) # $ exec > to
try:
yield stdout # allow code to be run with the redirected stdout
finally:
# restore stdout to its previous value
#NOTE: dup2 makes stdout_fd inheritable unconditionally
stdout.flush()
os.dup2(copied.fileno(), stdout_fd) # $ exec >&copied
The same example works now if stdout_redirected() is used instead of redirect_stdout():
import os
import sys
stdout_fd = sys.stdout.fileno()
with open('output.txt', 'w') as f, stdout_redirected(f):
print('redirected to a file')
os.write(stdout_fd, b'it is redirected now\n')
os.system('echo this is also redirected')
print('this is goes back to stdout')
The output that previously was printed on stdout now goes to output.txt as long as stdout_redirected() context manager is active.
Note: stdout.flush() does not flush
C stdio buffers on Python 3 where I/O is implemented directly on read()/write() system calls. To flush all open C stdio output streams, you could call libc.fflush(None) explicitly if some C extension uses stdio-based I/O:
try:
import ctypes
from ctypes.util import find_library
except ImportError:
libc = None
else:
try:
libc = ctypes.cdll.msvcrt # Windows
except OSError:
libc = ctypes.cdll.LoadLibrary(find_library('c'))
def flush(stream):
try:
libc.fflush(None)
stream.flush()
except (AttributeError, ValueError, IOError):
pass # unsupported
You could use stdout parameter to redirect other streams, not only sys.stdout e.g., to merge sys.stderr and sys.stdout:
def merged_stderr_stdout(): # $ exec 2>&1
return stdout_redirected(to=sys.stdout, stdout=sys.stderr)
Example:
from __future__ import print_function
import sys
with merged_stderr_stdout():
print('this is printed on stdout')
print('this is also printed on stdout', file=sys.stderr)
Note: stdout_redirected() mixes buffered I/O (sys.stdout usually) and unbuffered I/O (operations on file descriptors directly). Beware, there could be buffering issues.
To answer, your edit: you could use python-daemon to daemonize your script and use logging module (as @erikb85 suggested) instead of print statements and merely redirecting stdout for your long-running Python script that you run using nohup now.
This is happening because normally when process STDOUT is redirected to something other than a terminal, then the output is buffered into some OS-specific-sized buffer (perhaps 4k or 8k in many cases). Conversely, when outputting to a terminal, STDOUT will be line-buffered or not buffered at all, so you'll see output after each \n or for each character.
You can generally change the STDOUT buffering with the stdbuf utility:
stdbuf -oL python script.py > log
Now if you tail -F log, you should see each line output immediately as it is generated.
Alternatively explicit flushing of the output stream after each print should achieve the same. It looks like sys.stdout.flush() should achieve this in Python. If you are using Python 3.3 or newer, the print function also has a flush keyword that does this: print('hello', flush=True).
This should do the job:
import time, sys
for i in range(10):
print('bla')
sys.stdout.flush()
time.sleep(5)
As Python will buffer the stdout by default, here i have used sys.stdout.flush() to flush the buffer.
Another solution would be to use the -u(unbuffered) switch of python. So, the following will do too:
python -u script.py >> log
python - How to redirect stdout to both file and console with scripting? - Stack Overflow
How do I output my print statements to a text file?
Is stdout a file?
How to use Popen to Capture Output from a Console Window (Windows OS)?
Videos
You can use shell redirection while executing the Python file:
python foo_bar.py > file
This will write all results being printed on stdout from the Python source to file to the logfile.
Or if you want logging from within the script:
import sys
class Logger(object):
def __init__(self):
self.terminal = sys.stdout
self.log = open("logfile.log", "a")
def write(self, message):
self.terminal.write(message)
self.log.write(message)
def flush(self):
# this flush method is needed for python 3 compatibility.
# this handles the flush command by doing nothing.
# you might want to specify some extra behavior here.
pass
sys.stdout = Logger()
Now you can use:
print "Hello"
This will write "Hello" to both stdout and the logfile.
I got the way to redirect the out put to console as well as to a text file as well simultaneously:
te = open('log.txt','w') # File where you need to keep the logs
class Unbuffered:
def __init__(self, stream):
self.stream = stream
def write(self, data):
self.stream.write(data)
self.stream.flush()
te.write(data) # Write the data of stdout here to a text file as well
sys.stdout=Unbuffered(sys.stdout)
I've got a program that runs and prints statements to the screen that I'd like to output to a file for posterity... I've googled the question, but nothing seems to make sense or help.
A lot of people say to use the write statement, but I keep getting function errors (only takes one, you gave it two).
some people say that you can simply do a >> after the print statement, but how do I know what file that outputs to?
Any advice (spelled out like I'm five) would be greatly appreciated.