Hey everyone, I just want to talk about reading in data from standard input and the 4 main ways it can be done.

I'm not going to talk about the input() or raw_input() functions today, instead ill be talking about how to read from standard input using the sys module.

To get access to the sys module we first need to import it

import sys

Ok now we have access to this module, there are 3 ways to read from standard input:

1. sys.stdin.read([size])

2. sys.stdin.readline()

3. sys.stdin.readlines()

Lets look at how all of these work first and the ways to use them.

First off we can read lines directly from the console, this will look something like this

lines = sys.stdin.read()
print(lines)

$ python3 stdin.py
Line1
Line 2
**END**
Line 1
Line 2

Our lines variable looks like this: "Line1\nLine2"

Here when we run our program, it waits until it we pass some data through the console window. We specify end of input using ctrl+z on windows and I believe ctrl+d on linux.

The sys.stdin.read() function also has an optional parameter for the size of the data we want to read. For example if we pass 10 then it reads 10 characters including any newline characters.

The read() function will read everything, or the size of data specified, and return it as one string. This is useful for small amounts of data but if we read large files this way, it can use up a lot of memory.

The second way is sys.stdin.readline() which is self explanatory and reads a single line from standard input with a newline character at the end.

line = sys.stdin.readline()
print(line)

$ python3 stdin.py
hello
hello

The next way is sys.stdin.readlines(). I find myself using this way most often. With this way, we read lines from the console and are returned a list containing all the lines we entered.

lines = sys.stdin.readlines()
print(lines)

$ python3 stdin.py
line1
line2
line3
['line1\n', 'line2\n', 'line3\n']

This is very useful if we wish to process a file line by line although, we do have a large list sitting in memory which we may not want with large files. I will show you how to read from files in a moment.

Reading from files:

To read from a file we can do this a couple of ways, we can open and read the file within our program.

with open('FILENAME', [rw]) as our_file:
    for line in our_file:
        print(line)

The optional [rw] specifies whether we wish to open the file for reading, r or writing, w. This will work depending on the access permission on the file. You can check this on linux from the command line by navigating to your directory where the file is and typing:

$ ls -l

This will display the access permissions of the file in that directory.

An error will be thrown if you try to read or write without having permission to do so.

If the file name you entered doesn't exist, an empty file will be created for you.

The use of with open() here is very useful as it closes our file for us when we are finished.

Another way to read a file is passing it at the command line

$ python3 stdin.py < FILENAME.txt

Presuming FILENAME.txt looks like this:

Line 1
Line 2
Line 3

Running the following program, we get the following output:

import sys

lines = sys.stdin.readlines()
print(lines)

$ python3 stdin.py < FILENAME.txt
['Line 1\n', 'Line 2\n', 'Line 3']

I dont want to talk to much about the different ways of reading and writing files as I only wanted to talk about the different methods we have available to use for reading so I wont discuss any further ways of reading.

If we wish to strip the newline characters from our lines we can use the strip() method, I'm going to use a list comprehension here as it is a good example of their usage:

lines = [line.strip() for line in sys.stdin.readlines()]
print(lines)

$ python3 stdin.py < FILENAME.txt
['Line 1', Line 2', 'Line 3']

Whats the list comprehension doing? It uses a for loop to loop through each line in standard input, takes each line and strips it then appends it to our list, lines.

Now our newline characters are gone.

We covered a fair bit of stuff here and got the chance to see some extra things in use such as list comprehensions. If you found anything here confusing, play around with it yourself, after all its one of the best ways to learn.

I recently took a coding assessment where I was tasked to compute the square of the input which is relatively easy. The issue was that the input was a multiple line input:
7
16

and the expected output is
49
256

The given code was

for line in sys.stdin:
    # your code here
    print(line, end="")

I tried to do ls = list(line.split()) but ls[0] is
['7']
['16']
and ls[1] is None

I also tried ls = list(line.split('\n')) but ls is
['7', '']
['16', '']

So how was I supposed to process the input to get ['7', '16'] rather than a 2 dimensional list?

From there I know how continue with make it an integer using map, creating a for loop for each item of the list and printing the square of each item.

I dont have a picture of the question since I was monitored on webcam but this is roughly what I remembered from the question.

edit: It was an online assessment platform so I am not sure exactly how the input was written as (like the raw input). Also I can only modify the code inside for line in sys.stdin:

Also, does anyone know how to write the input for sys.stdin using jupyternotebook such that I can practice this problem?

I'm trying to feed a prompt to a GPT-2 model, which is trained on song lyrics. With this model, you can feed it a prompt and it will attempt to finish it. The problem I'm having is that I need for the prompt to be formatted correctly like song lyrics - with each lyric on a newline. Since this model has been trained on formatted lyrics, the prompt needs to look the same to produce above average results.

input() doesn't work, because hitting return submits the text. I've also tried sys.stdin.read() as well and nothing seems to happen, just returns ' '.

Is it because I'm running it in a notebook? I'm only like 8 months deep into this python hobby, so I could be asking for something that can't happen.

Here's the code, would appreciate any ideas you've got.

prompt = input('Input Prompt:')
gpt2.generate(sess,
              length=200,
              temperature=.99,
              prefix= prompt, #this is where the prompt goes
              nsamples=5,
              top_p=.9,
              batch_size=5,
              run_name='run1'
              )

Hi, I'm trying to read a block of line text and stick it all into a list.

An example of the lines to read in is...

atcay

ittenkay

oopslay

Here's my code...

newlist = []

x = sys.stdin.readlines()

newwords.append(x) 


for k in newwords:

    if k in words:

        print(words[k])


    if k not in words:

        print("eh")

As I'm sure you all can already see, this is a run-time error. Can someone help me get it to run please?