Table of Contents

• Invoke a blocking process
• Invoke a non-blocking process
• Using PIPE
• When to use shell=True
• References

Invoke a blocking process howto

>> subprocess.run(["ls", "-l"])  # doesn't capture output
CompletedProcess(args=['ls', '-l'], returncode=0)

>> subprocess.run(["ls", "-l", "/dev/null"], stdout=subprocess.PIPE)
CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0,
stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n')

• Added in Python 3.5. For older versions, use subprocess.call() instead.

Invoke a non-blocking process howto

proc = Popen(['ls', '-l'], stdout=PIPE)
# Read the subprocess's stdout, stdderr, and blocks until it exits
outs, errs = proc.communicate(timeout=15)

• Not using Popen.communicate() or call() will result in a zombie process.

Using PIPE discussion

• Always use communicate(), Don't use wait()
• Use communicate() rather than .stdin.write, .stdout.read or .stderr.read
• bufsize only works on file descriptors on the parent process side.
  • Popen uses os.pipe() internally.
  • it won't make the child process flush.
  • To make the child process flush, there are several ways like pexpect

as specified in open

Warning This will deadlock when using stdout=PIPE and/or stderr=PIPE and the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data. Use communicate() to avoid that.

Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffersfilling up and blocking the child process.

When to use shell=True discussion

• with shell=False, the first argument should be a list.
• with shell=True, the first argument should be a string.
  • The string for the first argument is like the command you put into the shell prompt.
  • The command can use environment variables, globs, pipes.
  • It's very dangerous, not recommended.

References