The following should just work.
import sys
for line in sys.stdin:
# whatever
Rationale:
The code will iterate over lines in stdin as they come in. If the stream is still open, but there isn't a complete line then the loop will hang until either a newline character is encountered (and the whole line returned) or the stream is closed (and the whatever is left in the buffer is returned).
Once the stream has been closed, no more data can be written to or read from stdin. Period.
The reason that your code was overloading your cpu is that once the stdin has been closed any subsequent attempts to iterate over stdin will return immediately without doing anything. In essence your code was equivalent to the following.
for line in sys.stdin:
# do something
while 1:
pass # infinite loop, very CPU intensive
Maybe it would be useful if you posted how you were writing data to stdin.
EDIT:
Python will (for the purposes of for loops, iterators and readlines() consider a stream closed when it encounters an EOF character. You can ask python to read more data after this, but you cannot use any of the previous methods. The python man page recommends using
import sys
while True:
line = sys.stdin.readline()
# do something with line
When an EOF character is encountered readline will return an empty string. The next call to readline will function as normal if the stream is still open. You can test this out yourself by running the command in a terminal. Pressing ctrl+D will cause a terminal to write the EOF character to stdin. This will cause the first program in this post to terminate, but the last program will continue to read data until the stream is actually closed. The last program should not 100% your CPU as readline will wait until there is data to return rather than returning an empty string.
I only have the problem of a busy loop when I try readline from an actual file. But when reading from stdin, readline happily blocks.
Answer from Dunes on Stack OverflowThe following should just work.
import sys
for line in sys.stdin:
# whatever
Rationale:
The code will iterate over lines in stdin as they come in. If the stream is still open, but there isn't a complete line then the loop will hang until either a newline character is encountered (and the whole line returned) or the stream is closed (and the whatever is left in the buffer is returned).
Once the stream has been closed, no more data can be written to or read from stdin. Period.
The reason that your code was overloading your cpu is that once the stdin has been closed any subsequent attempts to iterate over stdin will return immediately without doing anything. In essence your code was equivalent to the following.
for line in sys.stdin:
# do something
while 1:
pass # infinite loop, very CPU intensive
Maybe it would be useful if you posted how you were writing data to stdin.
EDIT:
Python will (for the purposes of for loops, iterators and readlines() consider a stream closed when it encounters an EOF character. You can ask python to read more data after this, but you cannot use any of the previous methods. The python man page recommends using
import sys
while True:
line = sys.stdin.readline()
# do something with line
When an EOF character is encountered readline will return an empty string. The next call to readline will function as normal if the stream is still open. You can test this out yourself by running the command in a terminal. Pressing ctrl+D will cause a terminal to write the EOF character to stdin. This will cause the first program in this post to terminate, but the last program will continue to read data until the stream is actually closed. The last program should not 100% your CPU as readline will wait until there is data to return rather than returning an empty string.
I only have the problem of a busy loop when I try readline from an actual file. But when reading from stdin, readline happily blocks.
This actually works flawlessly (i.e. no runnaway CPU) - when you call the script from the shell, like so:
tail -f input-file | yourscript.py
Obviously, that is not ideal - since you then have to write all relevant stdout to that file -
but it works without a lot of overhead!
Namely because of using readline() - I think:
while 1:
line = sys.stdin.readline()
It will actually stop and wait at that line until it gets more input.
Hope this helps someone!
Hi,
I'm fairly new to python, but not to programming as a whole. I'm trying to read input via standard in:
numList = list(map(int, sys.stdin.read().split()))
Basically I'm expecting a long string of integers that i want to store as a list. This line, however, doesn't work, as the terminal keeps waiting for input endlessly. Isn't stdin.read() supposed to read until EOF? This exact line has worked for me in the past, until I had to reinstall windows, whereafter the problem started occuring. Any tips as of how to fix this would be greatly appreciated! Perhaps it's some vs code extension that is missing? Though it seems unlikely.
Python async: Waiting for stdin input while doing other stuff - Stack Overflow
Should ast wait for stdin if we didn't pass anthing in `python3 -m ast`
io - Python reading from stdin while doing other tasks - Software Engineering Stack Exchange
python - How do I wait for a pressed key? - Stack Overflow
You can use the aioconsole third-party package to interact with stdin in an asyncio-friendly manner:
line = await aioconsole.ainput('Is this your line? ')
Borrowing heavily from aioconsole, if you would rather avoid using an external library you could define your own async input function:
async def ainput(string: str) -> str:
await asyncio.get_event_loop().run_in_executor(
None, lambda s=string: sys.stdout.write(s+' '))
return await asyncio.get_event_loop().run_in_executor(
None, sys.stdin.readline)
This is a perfect example to use multithreading.
Create a 'worker' thread and another thread, which waits for user input. When the 'input' thread receives input, it can add an item to a 'todo' list (with some sort of locking to exclude race conditions).
Then, the worker thread can fetch the topmost element and execute it. In your case it should also add another item for 'unbanning' the IP later, which means each task should have a 'time' tag of some sort.
A reasonable structure to implement this is a red-black tree or a binary heap with sorting by the 'time' value.
Then, your worker thread checks if the time has come to execute the top object in the 'todo' list, executes it and sleeps some reasonable amount of time (say 100 ms).
On the other hand, the input thread waits for input, and when presented with it, adds a task with the current time (which guarantees it would be almost immediately picked up by the worker thread).
Threads are fine, and there's really nothing wrong with using them here. For completeness sake, there's also select(), which allows you to do a blocking check for pending input on a file descriptor, with a timeout. If you use this function in a loop, interspersed with the timed activity, you'll get roughly the same concurrent behavior, but implemented on a single thread. This is the python binding for select() and its friends, btw.
In Python 3, use input():
input("Press Enter to continue...")
In Python 2, use raw_input():
raw_input("Press Enter to continue...")
This only waits for the user to press enter though.
On Windows/DOS, one might want to use msvcrt. The msvcrt module gives you access to a number of functions in the Microsoft Visual C/C++ Runtime Library (MSVCRT):
import msvcrt as m
def wait():
m.getch()
This should wait for a key press.
Notes:
In Python 3, raw_input() does not exist.
In Python 2, input(prompt) is equivalent to eval(raw_input(prompt)).
In Python 3, use input():
input("Press Enter to continue...")
In Python 2, use raw_input():
raw_input("Press Enter to continue...")