Hey everyone, I just want to talk about reading in data from standard input and the 4 main ways it can be done.

I'm not going to talk about the input() or raw_input() functions today, instead ill be talking about how to read from standard input using the sys module.

To get access to the sys module we first need to import it

import sys

Ok now we have access to this module, there are 3 ways to read from standard input:

1. sys.stdin.read([size])

2. sys.stdin.readline()

3. sys.stdin.readlines()

Lets look at how all of these work first and the ways to use them.

First off we can read lines directly from the console, this will look something like this

lines = sys.stdin.read()
print(lines)

$ python3 stdin.py
Line1
Line 2
**END**
Line 1
Line 2

Our lines variable looks like this: "Line1\nLine2"

Here when we run our program, it waits until it we pass some data through the console window. We specify end of input using ctrl+z on windows and I believe ctrl+d on linux.

The sys.stdin.read() function also has an optional parameter for the size of the data we want to read. For example if we pass 10 then it reads 10 characters including any newline characters.

The read() function will read everything, or the size of data specified, and return it as one string. This is useful for small amounts of data but if we read large files this way, it can use up a lot of memory.

The second way is sys.stdin.readline() which is self explanatory and reads a single line from standard input with a newline character at the end.

line = sys.stdin.readline()
print(line)

$ python3 stdin.py
hello
hello

The next way is sys.stdin.readlines(). I find myself using this way most often. With this way, we read lines from the console and are returned a list containing all the lines we entered.

lines = sys.stdin.readlines()
print(lines)

$ python3 stdin.py
line1
line2
line3
['line1\n', 'line2\n', 'line3\n']

This is very useful if we wish to process a file line by line although, we do have a large list sitting in memory which we may not want with large files. I will show you how to read from files in a moment.

Reading from files:

To read from a file we can do this a couple of ways, we can open and read the file within our program.

with open('FILENAME', [rw]) as our_file:
    for line in our_file:
        print(line)

The optional [rw] specifies whether we wish to open the file for reading, r or writing, w. This will work depending on the access permission on the file. You can check this on linux from the command line by navigating to your directory where the file is and typing:

$ ls -l

This will display the access permissions of the file in that directory.

An error will be thrown if you try to read or write without having permission to do so.

If the file name you entered doesn't exist, an empty file will be created for you.

The use of with open() here is very useful as it closes our file for us when we are finished.

Another way to read a file is passing it at the command line

$ python3 stdin.py < FILENAME.txt

Presuming FILENAME.txt looks like this:

Line 1
Line 2
Line 3

Running the following program, we get the following output:

import sys

lines = sys.stdin.readlines()
print(lines)

$ python3 stdin.py < FILENAME.txt
['Line 1\n', 'Line 2\n', 'Line 3']

I dont want to talk to much about the different ways of reading and writing files as I only wanted to talk about the different methods we have available to use for reading so I wont discuss any further ways of reading.

If we wish to strip the newline characters from our lines we can use the strip() method, I'm going to use a list comprehension here as it is a good example of their usage:

lines = [line.strip() for line in sys.stdin.readlines()]
print(lines)

$ python3 stdin.py < FILENAME.txt
['Line 1', Line 2', 'Line 3']

Whats the list comprehension doing? It uses a for loop to loop through each line in standard input, takes each line and strips it then appends it to our list, lines.

Now our newline characters are gone.

We covered a fair bit of stuff here and got the chance to see some extra things in use such as list comprehensions. If you found anything here confusing, play around with it yourself, after all its one of the best ways to learn.

Hi, just wondering if using input() or sys.stdin is the preferred method for reading from stdin in python. What are the differences and what is more readable/pythonesque?

# Using input()
while True:
    try:
        line = input()
        ...
    except EOFError:
        break

# Using sys.stdin
for line in sys.stdin:
    line = line.strip()
    ...