Capturing Output from External Commands in Python

In this tutorial, we will explore how to capture the output of external commands executed using Python’s subprocess module. The subprocess module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes.

Introduction to Subprocess Module

The subprocess module provides a high-level interface for running subprocesses. It was designed to replace several other modules and functions for running subprocesses, including os.system(), os.popen(), and popen2.*.

Running External Commands

To run an external command using the subprocess module, you can use the subprocess.run() function (available in Python 3.5 and later) or the subprocess.Popen() class. Here’s a basic example of running a command using subprocess.run():

import subprocess

command = ['echo', 'hello']
result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
print(result.returncode, result.stdout, result.stderr)

In this example, the subprocess.run() function runs the command echo hello and captures its output.

Capturing Output

To capture the output of a command, you can pass the stdout argument to subprocess.run() or subprocess.Popen(). The stdout argument specifies where the standard output of the command should be sent. You can set it to subprocess.PIPE to capture the output.

Here’s an example using subprocess.Popen():

import subprocess

command = ['echo', 'hello']
proc = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = proc.communicate()
print(out.decode(), err.decode())

In this example, the communicate() method is used to wait for the command to complete and capture its output.

Error Handling

When running external commands, it’s essential to handle errors properly. You can use the try-except block to catch any exceptions raised by the subprocess module.

Here’s an example:

import subprocess

command = ['non-existent-command']
try:
    result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
    print(result.returncode, result.stdout, result.stderr)
except FileNotFoundError:
    print("Command not found")

In this example, the FileNotFoundError exception is caught and handled when the command does not exist.

Best Practices

When working with external commands in Python, keep the following best practices in mind:

• Always validate user input to prevent shell injection attacks.
• Use absolute paths for commands to avoid ambiguity.
• Handle errors properly using try-except blocks.
• Consider using subprocess.run() instead of subprocess.Popen() for simpler use cases.

By following these guidelines and examples, you can effectively capture output from external commands in Python using the subprocess module.