Managed Authentication for AWS RDS and Aurora PostgreSQL

Managed authentication lets the Atatus Infrastructure Agent connect to your RDS or Aurora PostgreSQL database using a short-lived IAM token instead of a password. Nothing long-lived is written to the agent host, and you revoke access by editing an IAM policy rather than by rotating a password that every host holds a copy of.

This page covers PostgreSQL on Amazon RDS and Amazon Aurora. If you are using a password today, you can move to managed authentication without changing anything else about your setup.

Before you begin

Check these requirements first, because a mismatch here fails later in a way that is hard to read:

How it works

The agent signs a connection token locally using the IAM credentials it already holds, then presents that token to PostgreSQL as the password. The token is valid for 15 minutes and the agent mints a new one for each connection.

Two consequences are worth knowing up front:

• Token minting is not logged in CloudTrail. The token is signed on the client side, so no AWS API call is made and no event is recorded. You can audit the connection landing in your PostgreSQL logs instead.
• The IAM policy controls who, not where. IAM database authentication ignores aws:SourceIp, aws:SourceVpc and aws:SourceVpce in a policy condition. Use security groups to restrict the network path.

Step 1: Turn on IAM database authentication

Your database has to accept IAM tokens before any of the following matters. Without this, the token is signed correctly and then rejected at login.

In the AWS console, go to Modify, then Database authentication, and choose Password and IAM database authentication.

For a DB instance, you can also use the CLI:

$ aws rds modify-db-instance --db-instance-identifier <DB_IDENTIFIER> \
    --enable-iam-database-authentication --apply-immediately

For an Aurora cluster, the setting lives on the cluster:

$ aws rds modify-db-cluster --db-cluster-identifier <CLUSTER_IDENTIFIER> \
    --enable-iam-database-authentication --apply-immediately

aws rds describe-db-instances --db-instance-identifier <DB_IDENTIFIER> \
    --query 'DBInstances[0].PendingModifiedValues'

Step 2: Create the database role

Connect as your master user and create the atatus role. Run this once per cluster:

CREATE USER atatus WITH LOGIN;
GRANT rds_iam TO atatus;
GRANT pg_monitor TO atatus;
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

pg_monitor is the built-in read-only monitoring role. Without it the agent sees only its own session in pg_stat_activity, and the query text of every other session is redacted to <insufficient privilege>.

The agent opens a connection per database it collects from. Roles inherit CONNECT from PUBLIC by default. If you have revoked that, grant it back for each database you want monitored:

GRANT CONNECT ON DATABASE <DB_NAME> TO atatus;

Step 3: Allow the agent to connect

The identity the agent runs as needs permission to open an IAM-authenticated connection as the atatus role. Attach this policy to the agent's EC2 instance role, ECS task role, or EKS pod role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AtatusDBMConnect",
      "Effect": "Allow",
      "Action": "rds-db:connect",
      "Resource": "arn:aws:rds-db:<REGION>:<ACCOUNT_ID>:dbuser:<RESOURCE_ID>/atatus"
    }
  ]
}

<RESOURCE_ID> is the database resource id, not the database name. Find it under Configuration in the RDS console:

Step 4: Configure the agent

Edit /etc/atatus-infra-agent/conf.d/postgresql.d/postgresql.yml and add an aws: block:

metrics:
  - hosts: ["postgres://prod-pg.abc123.us-east-2.rds.amazonaws.com:5432"]
    username: atatus
    dbm: true
    db_name: postgres
    aws:
      region: us-east-2
      instance_endpoint: "prod-pg.abc123.us-east-2.rds.amazonaws.com:5432"
      managed_authentication:
        enabled: true

These are the fields that matter:

Set instance_endpoint explicitly whenever hosts: is not the real RDS endpoint, such as a CNAME, an IP address, or a connection pooler. AWS rejects a token presented to a host it was not signed for.

Restart the agent to pick up the change:

$ sudo service atatus-infra-agent restart

TLS

TLS is mandatory under managed authentication and defaults to verify-full. The token is a bearer credential, so a connection that can fall back to plaintext would hand it to anyone on the network path. The agent does not let you turn TLS off here.

You do not need to install a certificate. The agent ships the Amazon RDS certificate bundle and trusts it alongside the system store, which matters because the RDS authorities are a private PKI that is in no operating system trust store.

verify-ca and require are also accepted. Point ssl_root_cert at a PEM file only if you terminate TLS with your own certificate authority:

    aws:
      region: us-east-2
      managed_authentication:
        enabled: true
      ssl_mode: "verify-full"
      # ssl_root_cert: "/etc/ssl/certs/my-ca.pem"

Monitor several databases with one agent

One agent can collect from any number of PostgreSQL databases. You do not need an agent per database, and the databases do not have to share a region, an account, or a set of credentials.

There are three ways to arrange it. They can be mixed in the same setup.

One entry per database

metrics is a list, so add an entry for each database. This is the clearest option and the one to reach for by default, because every setting is scoped to its own database:

metrics:
  - hosts: ["postgres://orders-db.abc123.us-east-2.rds.amazonaws.com:5432"]
    username: atatus
    dbm: true
    db_name: orders
    reported_hostname: orders-production
    labels:
      team: payments
      env: production
    aws:
      region: us-east-2
      instance_endpoint: "orders-db.abc123.us-east-2.rds.amazonaws.com:5432"
      managed_authentication:
        enabled: true

  - hosts: ["postgres://analytics-db.xyz789.eu-west-1.rds.amazonaws.com:5432"]
    username: atatus
    dbm: true
    db_name: analytics
    reported_hostname: analytics-production
    labels:
      team: data
      env: production
    aws:
      region: eu-west-1
      instance_endpoint: "analytics-db.xyz789.eu-west-1.rds.amazonaws.com:5432"
      managed_authentication:
        enabled: true

Because each entry carries its own aws block, the databases can sit in different regions, and with role_arn they can sit in different AWS accounts. Each entry can also use a different authentication method, so one database can use a password while another uses IAM tokens.

Several hosts in one entry

When databases share the same credentials and region, list them under a single hosts entry:

metrics:
  - hosts:
      - "postgres://orders-db.abc123.us-east-2.rds.amazonaws.com:5432"
      - "postgres://billing-db.abc123.us-east-2.rds.amazonaws.com:5432"
      - "postgres://reporting-db.abc123.us-east-2.rds.amazonaws.com:5432"
    username: atatus
    dbm: true
    db_name: postgres
    aws:
      region: us-east-2
      managed_authentication:
        enabled: true

Everything outside hosts is shared, so this works only when the username, the region and the authentication method are the same for all of them.

Each host still needs its own rds-db:connect permission, because the resource id differs per database. Grant them in one policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AtatusDBMConnect",
      "Effect": "Allow",
      "Action": "rds-db:connect",
      "Resource": [
        "arn:aws:rds-db:us-east-2:<ACCOUNT_ID>:dbuser:db-ORDERSRESOURCEID/atatus",
        "arn:aws:rds-db:us-east-2:<ACCOUNT_ID>:dbuser:db-BILLINGRESOURCEID/atatus",
        "arn:aws:rds-db:us-east-2:<ACCOUNT_ID>:dbuser:cluster-REPORTINGRESOURCEID/atatus"
      ]
    }
  ]
}

One file per database

The agent reads every .yml file under conf.d/postgresql.d/, not only postgresql.yml. Give each database its own file when you want them managed separately, for example by different teams or by a configuration management tool:

$ ls /etc/atatus-infra-agent/conf.d/postgresql.d/
orders.yml
analytics.yml
billing.yml
postgresql.yml.template

Each file holds its own metrics list and is loaded independently, so a mistake in one file is easier to find than a mistake in a long combined file.

Things to watch

Keep these in mind as the number of databases grows:

• Every database role has to be set up separately. Steps 2 and 3 above are per database, so create the atatus role and grant rds_iam on each one.
• The agent opens a connection per database it collects from. Check max_connections on your smaller instances before you add many databases to one agent.
• Security groups are per database. The agent host needs a path to each one on port 5432.
• Restart once, after all the edits. The agent reads the whole conf.d tree at startup.

$ sudo service atatus-infra-agent restart

Monitoring a database in another AWS account

RDS has no resource-based policy, so the only workable shape is a role in the database's account that the agent assumes. The agent's own credentials are used only to call sts:AssumeRole. The token is then signed with the temporary credentials that role returns.

Add role_arn to the managed_authentication block:

    aws:
      region: us-east-2
      instance_endpoint: "prod-pg.abc123.us-east-2.rds.amazonaws.com:5432"
      managed_authentication:
        enabled: true
        role_arn: "arn:aws:iam::<DB_ACCOUNT_ID>:role/AtatusDBMAccess"
        external_id: "<EXTERNAL_ID>"

Set external_id only if the trust policy requires one. In the database owner's CloudTrail, the agent appears with the role session name atatus-infra-agent.

Confirm the assume works from the agent host before you restart the agent:

$ aws sts assume-role \
    --role-arn arn:aws:iam::<DB_ACCOUNT_ID>:role/AtatusDBMAccess \
    --role-session-name atatus-test \
    --external-id <EXTERNAL_ID>

Running the agent outside AWS

With no credential fields set, the agent uses the standard AWS credential chain: environment variables, then the shared config file, then the EC2, ECS or EKS role. That is the best setup inside AWS, because no key is written to disk.

On a host outside AWS, supply credentials explicitly. They are used only to sign the token, or to assume the role above:

    aws:
      region: us-east-2
      access_key_id: "<ACCESS_KEY_ID>"
      secret_access_key: "<SECRET_ACCESS_KEY>"
      # credential_profile_name: "atatus"
      # shared_credential_file: "/etc/atatus-infra-agent/aws-credentials"
      managed_authentication:
        enabled: true

Set it up with Terraform

Everything on this page can be provisioned instead of configured by hand: an EC2 host running the agent, the rds-db:connect policy with the resource id resolved at apply time, the security group rule, and optionally the database role itself created at first boot.

That has its own page, including the DocumentDB half and how to run both engines from one host:

• Set Up the Agent with Terraform for PostgreSQL
• Set Up the Agent with Terraform for DocumentDB

The working Terraform is at atatus-database-monitoring-sample.

Troubleshooting

Work through these in order. Each step rules out one layer, so you do not have to guess which one is at fault.

Check the agent version

Managed authentication needs 4.3.0 or higher. An older agent ignores the aws: block without complaining:

$ atatus-infra-agent version

Check the agent is running and loaded your config

$ sudo systemctl status atatus-infra-agent --no-pager
$ sudo cat /etc/atatus-infra-agent/conf.d/postgresql.d/postgresql.yml

If more than one agent process is running, the second one exits with a lock error and the first keeps running with whatever config it started with. Stop everything and start once:

$ sudo systemctl stop atatus-infra-agent
$ sudo pkill -f /usr/bin/atatus-infra-agent
$ sudo rm -f /var/lib/atatus-infra-agent/atatus-infra-agent.lock
$ sudo systemctl restart atatus-infra-agent

Turn up the log level

The default log level is warning, which hides the messages that show the integration starting up. Set log_level: info in /etc/atatus-infra-agent/atatus.yml, restart, and read the log:

$ sudo tail -f /var/log/atatus-infra-agent/agent.log

To filter for the PostgreSQL integration:

$ sudo grep -iE 'postgres|rds|token|auth|dbm' /var/log/atatus-infra-agent/agent.log | tail -40

Prove the token works on its own

This is the single most useful check, because it splits the problem cleanly in half. Run it on the agent host, so it uses the same identity the agent uses:

$ curl -sSo /tmp/rds-ca.pem https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem
$ TOKEN=$(aws rds generate-db-auth-token --region <REGION> \
    --hostname <ENDPOINT> --port 5432 --username atatus)
$ PGPASSWORD=$TOKEN psql \
    "host=<ENDPOINT> port=5432 dbname=postgres user=atatus sslmode=verify-full sslrootcert=/tmp/rds-ca.pem" \
    -c 'select current_user, pg_is_in_recovery()'

Read the result like this:

Query metrics are empty

If the agent connects but you see no queries, pg_stat_statements is most likely not loaded. It is a shared library that has to be loaded when the server starts, so CREATE EXTENSION on its own is not enough:

SHOW shared_preload_libraries;

The output has to list pg_stat_statements. If it does not, set shared_preload_libraries = pg_stat_statements in your parameter group and reboot the instance. This is a static parameter, so applying the parameter group without a reboot does nothing, CREATE EXTENSION still succeeds, and query metrics stay empty with no error.

aws rds reboot-db-instance --db-instance-identifier <WRITER_INSTANCE>

Common error messages

Related pages

• Database Monitoring for AWS RDS PostgreSQL
• Database Monitoring for AWS Aurora PostgreSQL
• Install the Atatus Infrastructure Agent