Leveraging Environment Variables in Docker Compose

Introduction

Managing configurations for containerized applications can be challenging, especially when you want to keep sensitive information secure and maintain different settings across various environments. This is where environment variables come into play. In this tutorial, we’ll explore how to effectively use environment variables with Docker Compose, allowing dynamic configuration without modifying your docker-compose.yml file.

Understanding Environment Variables

Environment variables are key-value pairs that influence the behavior of processes running in an operating system’s environment. They’re used for various purposes, such as configuring applications, storing sensitive data like API keys or database credentials, and providing flexibility across development, testing, and production environments.

Docker Compose and Environment Variables

Docker Compose allows you to define and run multi-container Docker applications. By using a docker-compose.yml file, you can configure services, networks, and volumes. With version 1.5 onwards, Docker Compose supports environment variable substitution directly within the configuration files.

Declaring Environment Variables

Using `.env` Files

Docker Compose natively supports reading variables from a .env file located in the same directory as your docker-compose.yml. This file contains key-value pairs that can be used throughout your compose file. For example:

# .env
MY_SECRET_KEY=SOME_SECRET
IMAGE_NAME=docker_image

You can then reference these variables in your docker-compose.yml like so:

my-service:
  image: ${IMAGE_NAME}
  environment:
    MY_SECRET_KEY: ${MY_SECRET_KEY}

To use a different file than .env, specify it with the env_file option:

my-service:
  env_file: path_to_env.env

Inline Substitution

If you prefer not to rely on an external file, Docker Compose allows inline substitution directly from your shell environment. You can run a command like this:

POSTGRES_VERSION=14 docker-compose up -d

And in your docker-compose.yml file, use the ${VARIABLE} syntax:

db:
  image: "postgres:${POSTGRES_VERSION}"

Important Considerations

Precedence: If an environment variable is defined both in a .env file and as an inline argument or shell variable, the latter takes precedence.
Syntax: Both $VARIABLE and ${VARIABLE} are supported. For literal dollar signs within values, use $$.

Alternative Approaches

Using `--env-file`

To bypass sourcing environment variables manually in a script, you can use:

docker-compose --env-file .my-env up -d

This method allows Docker Compose to read the environment variables directly from a specified file. Again, ensure that there’s no conflict with shell or command-line-provided variables.

Bash Scripting

For more advanced scenarios, you might write a Bash script to manage your environment setup and deployment process:

#!/bin/bash

docker rm -f $(docker ps -aq -f name=myproject_*)
set -a
source .env
cat ${COMPOSE_CONFIG} | envsubst | docker-compose -f - -p "myproject" up -d

This script demonstrates a more flexible deployment process, allowing dynamic configuration adjustments.

Conclusion

Leveraging environment variables with Docker Compose enhances your application’s configurability and security by keeping sensitive data out of version control. Whether you use .env files or inline variable substitution, Docker Compose provides the tools needed to manage complex configurations effectively. By understanding these techniques, you can streamline your development workflow and ensure consistent environments across different stages of deployment.