Hi everyone,

I am trying to use subprocess.Popen to open Powershell, and from this instance, open a secondary application (such that I can send it asynchronous Powershell commands from my Python code).

However, the subprocess.communicate() method seems to permanently close the stdin/stdout stream of the subprocess, which prevents me from sending it subsequent commands:

class _CLI:
def __init__(cls, appExe, appArgs, cmdlet, shell=True):
    cls._interface = subprocess.Popen(f"{appExe} {appArgs}", stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=shell)

def _execute(cls, cmd)
    # ONLY WORKS ONCE
    cls._interface.stdin.write(f"{cmd}\n".encode(encoding))
    output, error = self._interface.communicate()
    return output, error
# ...

max = _CLI(appExe="powershell.exe", cmd="3dsMax", shell=True)
max._execute("SOMETHING COOL")

When I create a new subprocess for each command, that seems to work, but I specifically don't want to open a new application each time I run a command:

class _CLI:
def init(cls, appExe, appArgs, cmdlet, shell=True):
    cls._interface = None
    cls.appExe = appExe
    cls.appArgs = appArgs

def _execute(cls, cmd)
    # WORKS FOR SUBSEQUENT CALLS
    cls._interface = subprocess.Popen(f"{cls.appExe} {cls.appArgs}", stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=shell) cls._interface.stdin.write(f"{cmd}\n".encode(encoding)) 
    output, error = self._interface.communicate()
    return output, error
#...

max = _CLI(appExe="powershell.exe", cmd="3dsMax", shell=True)
max._execute("SOMETHING COOL")

It seems that the status of stdin/stdout is a protected property, and subprocess doesn't have any accessible methods to re-open it. I noticed that subprocess.execute() has a timeout parameter, and I figured that maybe it was just waiting for an unspecified timeout (and hanging indefinitely since it wasn't set). However, adjusting that value didn't seem to do anything.

If anyone has any suggestions, I'd really appreciate it!

EDIT: fixed the busted formatting

The Popen docs warn against using Popen.stdin (etc), saying:

Warning: Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

Is there further explanation somewhere? What does this warning mean practically, for the use of process.stdin.write and .flush in the example code below? Is there another way to send input to a running process, which doesn't assume the process will exit after receiving input?

For context, I've got several examples of executables that accept input on stdin continually, processing lines as they come in. I'd like to run one, send input to the process, and do other things in python before sending more input. The recommended Popen.communicate doesn't support this requirement, since it waits for the process to close after sending input. I haven't found any way to do it.

The following trimmed-down example code works with cat; the real example executables have more significant processing of their input.

import subprocess
import time

duration=1
items=[
    'spam',
    'spam & spam',
    'spam, spam, spam, eggs & spam'
    ]

with open( 'outfile', 'a' ) as outfile:
    with subprocess.Popen(
            [ 'cat' ],
            stdin=subprocess.PIPE,
            stdout=outfile,
            stderr=subprocess.DEVNULL
            ) as process:
        for item in items:
            process.stdin.write(\
                    ('order "' + item + '"\n')\
                    .encode( 'utf-8' ) )
            process.stdin.flush()
            time.sleep( duration )

(edit: this code actually works with the example executables, so I'm wondering when / if the deadlocks will actually affect me)