Splitting Delimited Strings into Arrays with Awk

Awk is a powerful command-line tool for processing and manipulating text files. One of its most useful features is the ability to split delimited strings into arrays, allowing for easy access and manipulation of individual fields. In this tutorial, we’ll explore how to use awk’s split() function to achieve this.

Introduction to Awk

Before diving into string splitting, let’s cover some basics about awk. Awk is a programming language designed specifically for text processing. It reads input line by line, splits each line into fields based on a delimiter (by default, whitespace), and allows you to perform various operations on these fields.

The `split()` Function

The split() function in awk is used to divide a string into an array of substrings based on a specified delimiter. Its general syntax is as follows:

split(string, array, fieldsep)

string is the input string you want to split.
array is the name of the array where the split strings will be stored.
fieldsep is the delimiter used to split the string. If omitted, awk uses the current value of the FS (Field Separator) variable.

Basic Example

Let’s start with a simple example where we have a string "12|23|11" and we want to split it into an array using the "|" character as the delimiter.

echo "12|23|11" | awk '{split($0, a, "|"); print a[1], a[2], a[3]}'

This command will output:

12 23 11

Here’s what happens in this example:

echo "12|23|11" outputs the string to be split.
awk processes this string line by line (in this case, just one line).
{split($0, a, "|")} splits the entire input line ($0) into an array a, using "|" as the delimiter. Note that $0 refers to the whole line, and a is the name of the array where the split substrings are stored.
print a[1], a[2], a[3]} then prints out each element of the a array.

Using Regular Expressions as Delimiters

One powerful feature of awk’s split() function is its ability to use regular expressions (regex) as delimiters. For example, if you want to split a string on one or more "|" characters, you could use:

echo "12||23|11" | awk '{split($0, a, /\|+/); print a[1], a[2], a[3]}'

The /\|+/ is a regex that matches one or more "|" characters.

Setting FS for Default Splitting

Alternatively, you can set the FS (Field Separator) variable to split lines without explicitly using split(). For instance:

echo "12|23|11" | awk -F\| '{print $1, $2, $3}'

Here, -F\| sets the field separator to "|", so $1, $2, and $3 refer to the first, second, and third fields separated by "|".

Conclusion

Splitting delimited strings into arrays is a fundamental operation in text processing with awk. By mastering the split() function and understanding how to manipulate the FS variable, you can efficiently parse and analyze complex data sets. Whether you’re working with simple pipe-delimited files or more complex formats requiring regular expressions, awk provides the flexibility and power needed for a wide range of tasks.