Articles

Using AWS CloudFormation Templates to Define RDS Read Replicas

Dan White | August 14, 2019

In this post we will discuss how you can use AWS CloudFormation templates to define Amazon Relational Database Service (RDS) read replicas.

Intro

AWS CloudFormation provides a simple way to define and manage cloud infrastructure resources as code (IaC). It allows you to deploy and manage related groups of cloud infrastructure resources as “Stacks.”
Using CloudFormation templates enforces consistency and secure default configuration settings across regions as well as consistency across your test and production environments.
In a disaster recovery scenario, a well-tested template not only expedites the recovery process but also instills confidence in the results.

The quickest and easiest way to get started with AWS CloudFormation is via AWS Management Console, which has a built-in management user interface for managing Stacks.

AWS Console -> Services -> Management & Governance -> CloudFormation

Template Creation

For this example, we will use the AWS CloudFormation JSON syntax, although note that CloudFormation also supports YAML syntax.
The key design goal with this template is to maintain consistency of certain configuration options such as DB parameter groups, encryption configuration, logging, and storage types while allowing for quick customization of other settings via parameters.

Using parameterization within a CloudFormation template will allow for maximum flexibility from a single template.

Sample Template

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "CloudFormation template for establishing regional read replicas",
  "Parameters": {
    "DBInstanceClass": {
      "Type": "String",
      "Description": "The database subnet group name to use",
      "Default": "db.t2.large",
      "AllowedValues": [
      "db.r5.large", "db.r5.xlarge", "db.r5.2xlarge",
      "db.t2.small", "db.t2.medium", "db.t2.large"
      ]
    },
    "SourceRegion": {
      "Type": "String",
      "Description": "The region to replicate from.",
      "Default": "us-east-2"
    },
    "TargetRegion": {
      "Type": "String",
      "Description": "The region to replicate to.",
      "Default": "us-west-2"
    },
    "DBSubnetGroupName": {
      "Type": "String",
      "Description": "The database subnet group name to use"
    },
    "SourceDBInstanceIdentifier": {
      "Type": "String",
      "Description": "The replication source"
    },
    "TargetDBInstanceIdentifier": {
      "Type": "String",
      "Description": "The new database instance identifier"
    },
    "VPCSecurityGroups": {
      "Type": "List<AWS::EC2::SecurityGroup::Id>",
      "Description": "The list of security groups to associate with this instance"
    },
    "KmsKeyId": {
      "Type": "String",
      "Description": "The fully qualified KMS key ARN used for data encryption."
    }
  },
  "Resources": {
    "RstrengthProdReadReplica": {
      "Type": "AWS::RDS::DBInstance",
      "Properties": {
        "StorageEncrypted": true,
        "KmsKeyId": {
          "Ref": "KmsKeyId"
        },
        "StorageType": "gp2",
        "AllocatedStorage": 400,
        "AutoMinorVersionUpgrade": true,
        "AllowMajorVersionUpgrade": false,
        "CopyTagsToSnapshot": true,
        "DeletionProtection": true,
        "DBInstanceClass": {
          "Ref":  "DBInstanceClass"
        },
        "DBInstanceIdentifier": {
          "Ref": "TargetDBInstanceIdentifier"
        },
        "DBParameterGroupName": "case-insensitive",
        "DBSubnetGroupName": {
          "Ref": "DBSubnetGroupName"
        },
        "EnableCloudwatchLogsExports": [
          "error",
          "general",
          "audit"
        ],
        "EnableIAMDatabaseAuthentication": false,
        "Engine": "mysql",
        "EngineVersion": "5.7.21",
        "MultiAZ": false,
        "OptionGroupName": "default:mysql-5-7",
        "PubliclyAccessible": false,
        "SourceDBInstanceIdentifier": {
          "Ref": "SourceDBInstanceIdentifier"
        },
        "SourceRegion": {
          "Ref": "SourceRegion"
        },
        "Tags": [
        ],
        "VPCSecurityGroups": {
          "Ref": "VPCSecurityGroups"
        }
      }
    }
  }
}

Note the following elements in the template above:

  • Ability to create a read replica of from any source RDS instance into another region.
  • The RDS instance type, source and destination regions, as well as the source and target RDS instance identifiers are all parameterized.
  • A key requirement is the need to specify a list of VPC security groups. This is performed by creating a parameter that is a list of AWS intrinsic types:
"Type": "List<AWS::EC2::SecurityGroup::Id>"
  • Referencing a parameter within the CloudFormation template is accomplished using the { “Ref”: “ParameterName” } syntax.
  • The key difference between creating a read replica and simply creating a new RDS instance is presence of the SourceDBInstanceIdentifier attribute.
  • Reasonable default parameters may be specified for the most common scenarios.

Template Testing and Usage

AWS CloudFormation gives you several options for consuming the template. You have the ability to create objects from CloudFormation templates directly from with AWS Console. It’s also possible to run them from the AWS cli using the aws cloudformation deploy command.

  • Note that when dealing with multiple regions, you will be running the template in the destination region.

AWS Console -> Services -> Management & Governance -> CloudFormation -> Create Stack

For production usage and disaster recovery scenarios, creating a simple shell script that will re-establish well known read replicas is a more practical option.  A script similar to the following might be used to quickly re-establish your read replica topology:

#! /usr/bin/env bash

SOURCE_REGION=us-west-2
# use the ARN for your source database here
SOURCE_ID=arn:aws:rds:us-west-2:<account number>:db:develop

TARGET_REGION=us-east-1
TARGET_ID=development-replica

SUBNET_GROUP=default-vpc
# use the KMS KEY ARN for your target environment, if the source is encrypted.
KEY_ARN=arn:aws:kms:us-east-2:<account number>:key/<id>

# security group(s) here
SECURITY_GROUPS=sg-1232456

aws cloudformation deploy "$@" --template-file $(pwd)/rds-read-replica.template.json \
    --stack-name mys-read-replica \
    --parameter-overrides SourceRegion=${SOURCE_REGION} \
    TargetRegion=${TARGET_REGION} \
    DBSubnetGroupName=${SUBNET_GROUP} \
    SourceDBInstanceIdentifier=${SOURCE_ID} \
    TargetDBInstanceIdentifier=${TARGET_ID} \
    KmsKeyId=${KEY_ARN} \
    VPCSecurityGroups=${SECURITY_GROUPS}
  • Note that the inclusion of  “$@” will allow you to pass in additional optional parameters to the AWS CLI, such as a profile name.
  • Any parameters not specified in the –parameter-overrides argument will attempt to use the default value in the template.

    Gathering configuration parameters from an existing RDS instance

    You may want to inspect the configuration of an existing RDS instance as JSON so that you can capture the appropriate configuration parameters. This is simple to do using the AWS cli, by using the following command:

    aws rds describe-db-instances --region=<aws region code> --db-instance-identifier=<instance name>

Sample Output (edited for privacy):

$ aws rds describe-db-instances --region=us-west-2 --db-instance-identifier=develop
{
    "DBInstances": [
        {
            "DBInstanceIdentifier": "develop",
            "DBInstanceClass": "db.t2.large",
            "Engine": "mysql",
            "DBInstanceStatus": "available",
            "MasterUsername": "REDACTED",
            "Endpoint": {
                "Port": 3306,
            },
            "AllocatedStorage": 400,
            "InstanceCreateTime": "2018-11-08T16:12:05.365Z",
            "PreferredBackupWindow": "09:16-09:46",
            "BackupRetentionPeriod": 1,
            "DBSecurityGroups": [],
            "VpcSecurityGroups": [
            ],
            "DBParameterGroups": [
                {
                    "DBParameterGroupName": "caseinsensitive",
                    "ParameterApplyStatus": "in-sync"
                }
            ],
            "AvailabilityZone": "us-west-2c",
            "DBSubnetGroup": {
                "DBSubnetGroupName": "default-vpc",
                "DBSubnetGroupDescription": "Created from the RDS Management Console",
            },
            "PreferredMaintenanceWindow": "tue:09:50-tue:10:20",
            "PendingModifiedValues": {},
            "LatestRestorableTime": "2019-08-02T19:25:00Z",
            "MultiAZ": false,
            "EngineVersion": "5.7.21",
            "AutoMinorVersionUpgrade": true,
            "ReadReplicaDBInstanceIdentifiers": [
                "replicatest"
            ],
            "LicenseModel": "general-public-license",
            "OptionGroupMemberships": [
                {
                    "OptionGroupName": "default:mysql-5-7",
                    "Status": "in-sync"
                }
            ],
            "PubliclyAccessible": false,
            "StorageType": "gp2",
            "DbInstancePort": 0,
            "StorageEncrypted": true,
            "KmsKeyId": "",
            "CACertificateIdentifier": "rds-ca-2015",
            "DomainMemberships": [],
            "CopyTagsToSnapshot": true,
            "MonitoringInterval": 0,
            "DBInstanceArn": "arn:aws:rds:us-west-2:<account number>:db:develop",
            "IAMDatabaseAuthenticationEnabled": false,
            "PerformanceInsightsEnabled": false,
            "DeletionProtection": true
        }
    ]
}

Conclusion

AWS CloudFormation and other IaC solutions are an excellent supplement to your existing cloud infrastructure. Taking the time to implement such solutions will lower costs, increase the speed and quality of your infrastructure deployments, and ensure that security and auditing requirements are met.

Back to list