Deleting Keys by Pattern in Redis

Redis is an in-memory data structure store, often used as a database, cache, and message broker. A common task when managing a Redis database is deleting multiple keys that match a specific pattern. This tutorial will explore several ways to accomplish this, focusing on efficiency and atomicity.

Understanding the Challenge

Deleting keys by pattern can be tricky. The KEYS command in Redis, while straightforward, can be problematic in production environments. As the Redis documentation warns, KEYS can block the server for a significant amount of time if the database contains a large number of keys, potentially leading to performance degradation or even a denial-of-service situation. Therefore, it’s crucial to choose an appropriate approach, especially when dealing with large datasets.

Approaches to Key Deletion

Here are several methods for deleting keys based on a pattern, with increasing complexity and benefits:

1. The KEYS Command (Use with Caution)

The simplest approach is to use the KEYS command in conjunction with the DEL command. This works by first retrieving all keys matching the pattern using KEYS, then iterating through the results and deleting each key using DEL.

redis-cli KEYS "prefix:*" | xargs redis-cli DEL

Warning: As mentioned previously, this approach is generally discouraged in production due to the potential for blocking the Redis server.

2. Using SCAN for Iterative Deletion

The SCAN command provides an iterative way to retrieve keys, avoiding the blocking behavior of KEYS. SCAN returns a cursor and a batch of keys, and you can repeatedly call it, providing the returned cursor, until the cursor is 0, indicating that all keys have been processed.

Here’s a Bash script demonstrating how to use SCAN for deletion:

#!/bin/bash

if [ $# -ne 3 ]
then
  echo "Delete keys from Redis matching a pattern using SCAN & DEL"
  echo "Usage: $0 <host> <port> <pattern>"
  exit 1
fi

cursor=-1
keys=""

while [ $cursor -ne 0 ]; do
  if [ $cursor -eq -1 ]
  then
    cursor=0
  fi

  reply=`redis-cli -h $1 -p $2 SCAN $cursor MATCH $3`
  cursor=`expr "$reply" : '\([0-9]*[0-9 ]\)'`
  keys=${reply##[0-9]*[0-9 ]}
  redis-cli -h $1 -p $2 DEL $keys
done

This script is a safer alternative to using KEYS directly, as it retrieves keys in batches and avoids blocking the server.

3. Leveraging Lua Scripting for Atomicity

Redis provides the ability to execute Lua scripts atomically. This is the most robust approach, as it guarantees that the deletion process will not be interrupted by other commands.

Here’s an example Lua script:

return redis.call('DEL', unpack(redis.call('KEYS', ARGV[1] .. '*')))

This script retrieves all keys matching the pattern (e.g., prefix:*) and then deletes them atomically. You can execute this script using the EVAL command:

redis-cli EVAL "return redis.call('DEL', unpack(redis.call('KEYS', ARGV[1] .. '*')))" 0 prefix:

Handling Large Numbers of Keys in Lua

If you have a very large number of keys that might exceed the available memory for unpacking in Lua, you can iterate through the keys within the Lua script:

for _,k in ipairs(redis.call('keys', ARGV[1])) do
  redis.call('del', k)
end

This alternative approach avoids the memory limitations of unpack and provides a more scalable solution. Execute this version with:

redis-cli EVAL "for _,k in ipairs(redis.call('keys', ARGV[1])) do redis.call('del', k) end" 0 prefix:

Choosing the Right Approach

For debugging or small datasets, the KEYS and DEL combination might be sufficient. However, always be aware of the potential performance implications.
For larger datasets, SCAN provides a safer and more scalable solution.
For guaranteed atomicity and optimal performance, especially with very large datasets, Lua scripting is the recommended approach.

By understanding the trade-offs between these different methods, you can choose the most appropriate solution for your specific needs.