Filtering JSON Objects with JQ: Selecting Data Based on Key-Value Pairs

Filtering JSON Objects with JQ: Selecting Data Based on Key-Value Pairs

JQ is a powerful and flexible command-line JSON processor. It’s incredibly useful for transforming, filtering, and extracting data from JSON files or streams. A common task is to select specific objects within a JSON array based on the value of a key within those objects. This tutorial will guide you through how to accomplish this effectively.

Understanding the Problem

Imagine you have a JSON file containing an array of objects, each representing a record with various attributes. You need to extract only the objects that meet a certain condition – for example, objects where a specific key has a specific value.

Let’s consider a sample JSON structure:

{
  "FOO": {
    "name": "Donald",
    "location": "Stockholm"
  },
  "BAR": {
    "name": "Walt",
    "location": "Stockholm"
  },
  "BAZ": {
    "name": "Jack",
    "location": "Whereever"
  }
}

Our goal is to select only the objects where the "location" key has the value "Stockholm".

Using select() to Filter Objects

JQ provides the select() function to filter elements based on a boolean expression. The general syntax is:

.[] | select(.key == "value")

• .[]: This iterates through the array of objects in the JSON input.
• select(.key == "value"): This filters the objects. The .key == "value" part is the boolean expression. It checks if the value of the "key" attribute in the current object is equal to "value". Only objects that satisfy this condition are passed along the pipeline.

Applying this to our example, we can select the objects with "location" equal to "Stockholm":

.[] | select(.location == "Stockholm")

This will output:

{
  "location": "Stockholm",
  "name": "Walt"
}
{
  "location": "Stockholm",
  "name": "Donald"
}

Extracting Specific Values After Filtering

Often, you don’t need the entire object, only a specific value from the filtered objects. You can chain another operation after select() to extract the desired value. For example, to extract just the "name" attribute from the filtered objects:

.[] | select(.location == "Stockholm") | .name

This will output:

"Walt"
"Donald"

Working with Objects and Key-Value Pairs

Sometimes, your JSON data might be structured differently, or you might need to transform it before filtering. The to_entries and from_entries functions are useful for converting between objects and arrays of key-value pairs.

• to_entries: Converts an object into an array of key-value pair objects. Each object in the array has "key" and "value" attributes.
• from_entries: Converts an array of key-value pair objects back into an object.

Let’s say you want to get an array of [key, name] pairs for objects where the location is Stockholm. You can use:

to_entries[] | select(.value.location == "Stockholm") | [.key, .value.name]

This outputs:

["FOO", "Donald"]
["BAR", "Walt"]

Using Variables in JQ

You can also pass variables into your JQ expressions using the --arg option. This is useful for making your expressions more dynamic. For example:

location_value="Stockholm"
cat data.json | jq --arg loc "$location_value" '.[] | select(.location == $loc)'

This allows you to change the filtering criteria without modifying the JQ expression itself.

Combining Techniques for Complex Filtering

You can combine these techniques to create complex filtering expressions. For example, you might need to filter based on multiple conditions, extract multiple values, or transform the data before filtering. JQ’s powerful pipeline allows you to chain multiple operations together seamlessly.