Executing Shell Commands in Python: Capturing Output Without Displaying It on Screen

When working with shell commands within a Python script, it’s often useful to execute these commands and capture their output for further processing or logging. However, displaying the command’s output directly on the screen may not always be desirable. In this tutorial, we’ll explore various methods in Python to run shell commands, capture their outputs into variables, and prevent them from being displayed to the console.

Understanding os.system

The os.system() function is one of the simplest ways to execute a command in Python. It runs the command specified as a string in a subshell and returns an integer status code that indicates whether the command was successful (0) or not. However, it does not capture the output; rather, it directly displays it on the screen.

import os

var = os.system("cat /etc/services")
print(var)  # This will print 0 if the command is successful.

Using os.popen for Output Capture

An alternative to os.system() is using os.popen(), which allows capturing the output of a shell command. The function returns an object that can be read from, just like a file handle.

import os

output = os.popen('cat /etc/services').read()
print(output)  # Displays the contents without running in a subshell.

This method works well for capturing output but is less flexible than newer approaches available in Python.

The subprocess Module: A Robust Solution

Introduced to provide more powerful facilities, the subprocess module has become the preferred way to spawn new processes and interact with their input/output/error pipes. It offers a range of functions, like Popen(), run(), and check_output(), that can be tailored for specific use cases.

Using subprocess.Popen

This approach provides extensive control over how commands are executed:

import subprocess

proc = subprocess.Popen(
    ["cat", "/etc/services"], 
    stdout=subprocess.PIPE, 
    shell=True
)
(out, err) = proc.communicate()
print("Program output:", out.decode())

Here, Popen() starts the process and returns a Popen object. The communicate() method reads the output (and error if needed) of the command.

Using subprocess.run

For simpler use cases where only the result is required, subprocess.run() can be used to execute a shell command:

import subprocess

result = subprocess.run(
    "cat /etc/services", 
    capture_output=True, 
    text=True, 
    shell=True
)
print("Program output:", result.stdout)

The capture_output parameter captures the standard output and error, while text=True ensures that the output is returned as a string.

Using subprocess.check_output

This function runs a command with arguments and returns its output:

import subprocess

output = subprocess.check_output("cat /etc/services", shell=True)
print(output.decode())

It’s a convenient way to capture output when you expect the command to succeed, as it raises an exception if the command returns a non-zero exit status.

Summary

In Python, while os.system() can run commands, capturing their output requires more sophisticated solutions like os.popen(), or the powerful and flexible subprocess module. The latter is highly recommended for its robustness and versatility in handling subprocesses, providing you control over command execution and input/output streams.

By understanding these methods and using them appropriately, you can execute shell commands effectively within your Python scripts while managing their outputs as needed without displaying them on the screen.