Troubleshooting Invalid Token Errors with the AWS CLI

Understanding AWS CLI Authentication Errors

When working with the AWS Command Line Interface (CLI), you might encounter "The security token included in the request is invalid" errors. This commonly occurs when the AWS CLI is unable to properly authenticate your requests. This tutorial explains the common causes of this issue and how to resolve them, ensuring you can successfully interact with AWS services.

How AWS CLI Authentication Works

The AWS CLI relies on credentials to verify your identity and authorize access to AWS resources. These credentials typically consist of an Access Key ID and a Secret Access Key. More advanced setups may also include a session token, particularly when Multi-Factor Authentication (MFA) is enabled. The AWS CLI stores these credentials in a configuration file, by default located at ~/.aws/credentials. It also looks for environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN) and IAM roles associated with your EC2 instance or other AWS resources.

Common Causes of "Invalid Security Token" Errors

Several factors can lead to this error:

1. Incorrect Credentials: The most common cause is simply using incorrect or outdated Access Key ID and Secret Access Key combinations.
2. Expired Credentials: AWS credentials can expire. Regularly rotate your credentials to maintain security.
3. Conflicting Credentials: Multiple sets of credentials defined in your environment (e.g., both in the credentials file and as environment variables) can cause conflicts. The AWS CLI might be picking up the wrong set.
4. Session Token Issues: If you’re using MFA, a valid session token is required alongside the access and secret keys. An expired or incorrect token will lead to authentication failures.
5. Credential File Corruption: The ~/.aws/credentials file might be corrupted or contain unexpected entries.

Troubleshooting Steps

Here’s a step-by-step guide to resolving the "Invalid Security Token" error:

1. Verify Your Credentials:

• Double-check that the Access Key ID and Secret Access Key you are using are correct.
• Log in to the AWS Management Console and navigate to your IAM user. Verify that the keys you are using are still active. If not, create new access keys.

2. Configure the AWS CLI:

Use the aws configure command to set up your credentials:

aws configure

This command will prompt you for:

• AWS Access Key ID: Enter your Access Key ID.
• AWS Secret Access Key: Enter your Secret Access Key.
• Default region name: Enter your desired AWS region (e.g., us-east-1).
• Default output format: (Optional) Choose an output format like json or text.

This ensures that the credentials are correctly stored in the ~/.aws/credentials file.

3. Handle Conflicting Credentials:

• Environment Variables: If you’ve set environment variables like AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, ensure they are correct and consistent with the credentials stored in ~/.aws/credentials. It’s often best to rely on the configuration file and avoid setting environment variables unless necessary.
• Multiple Profiles: If you’re using multiple AWS profiles, make sure you’re specifying the correct profile when running AWS CLI commands using the --profile option.

4. Address Session Token Issues (MFA):

If you’ve enabled MFA for your AWS account, you need to obtain a temporary session token after providing your MFA code. The process usually involves using the aws sts get-session-token command. Then, configure the CLI with the token:

aws sts get-session-token --serial-number <your MFA serial number>

This command requires your MFA serial number and will prompt you for your MFA code. It outputs temporary credentials, including a session token. You can then set the token using:

aws configure set aws_session_token "<your session token>"

5. Clean Up the Credentials File:

Sometimes, the ~/.aws/credentials file can become corrupted or contain stale entries. Deleting the file and re-running aws configure can resolve these issues. Be cautious when deleting the file, as it will remove all stored credentials.

rm ~/.aws/credentials
aws configure

6. Verify Configuration with aws sts get-caller-identity

After making changes to your credentials, it is recommended to verify that the AWS CLI is using the correct credentials by running the following command:

aws sts get-caller-identity

This command will return information about the AWS account and IAM user associated with the current credentials. If the command executes successfully, it indicates that the AWS CLI is properly configured and authenticated.

By following these steps, you should be able to effectively troubleshoot and resolve "Invalid Security Token" errors, ensuring smooth and secure interactions with AWS services.