Reading a File Line by Line in Bash

In this tutorial, we’ll explore how to read a file line by line in Bash and assign each line’s value to a variable. This technique is essential for processing text files, configuration files, or any other type of file that contains data organized into lines.

Introduction to Reading Files in Bash

Bash provides several ways to read files, but the most common approach is using a while loop with the read command. The read command reads a line from standard input and assigns it to one or more variables.

Basic Syntax

The basic syntax for reading a file line by line in Bash is:

while IFS= read -r line; do
    # Process the line here
done < filename.txt

Let’s break down this syntax:

IFS=: This sets the internal field separator to an empty string, which prevents leading or trailing whitespace from being trimmed.
read -r: The -r option tells read not to interpret backslash escapes. This ensures that backslashes are treated as literal characters.
line: This is the variable that will hold the value of each line read from the file.
< filename.txt: This redirects the contents of the file filename.txt to standard input, which is then read by the while loop.

Processing Each Line

Inside the while loop, you can process each line as needed. For example, you might want to assign the value of each line to a variable and perform some tasks with it:

while IFS= read -r line; do
    name="$line"
    echo "Name read from file: $name"
    # Do something with $name here
done < filename.txt

Handling Non-POSIX Text Files

If the file you’re reading is not a standard POSIX text file (i.e., it doesn’t end with a newline character), you might need to modify the while loop to handle trailing partial lines:

while IFS= read -r line || [[ -n "$line" ]]; do
    echo "Text read from file: $line"
done < filename.txt

This modification uses the || operator to check if the last line is not empty, and if so, processes it accordingly.

Using a Bash Script

You can also put the file-reading code in a Bash script for easier reuse. Here’s an example:

#!/bin/bash

filename="$1"
while IFS= read -r line; do
    echo "Text read from file: $line"
done < "$filename"

Save this script to a file (e.g., readfile.sh), make it executable with chmod +x readfile.sh, and then run it with ./readfile.sh filename.txt.

Best Practices

When reading files in Bash, keep the following best practices in mind:

Always use the -r option with read to prevent backslash escapes from being interpreted.
Set IFS= to prevent leading or trailing whitespace from being trimmed.
Use a while loop to read files line by line, as it’s more efficient and flexible than other approaches.

By following these guidelines and examples, you should be able to read files line by line in Bash with ease.