Skip to content

Collector Configuration Manual for "AWS-RDS"

Before reading this article, please read the following first:

Before using this collector, you must install the 'Integration Core Package' and its associated third-party dependency packages

1. Configuration Structure

The configuration structure of this collector is as follows:

Field Type Required Description
regions list Required List of regions to collect data from
regions[#] str Required Region ID. For example: 'cn-north-1'
Refer to the appendix for the full list
engines list No List of engine types (if not configured, all RDS instances of all engine types will be collected)
engines[#] str No Engine type, currently supported: aurora-mysql, aurora-postgresql, mariadb, mysql, oracle-ee, oracle-ee-cdb, oracle-se2, oracle-se2-cdb, postgres, sqlserver-ee, sqlserver-se, sqlserver-ex, sqlserver-web

2. Configuration Example

Specified Regions

Collect instance data for Ningxia, Beijing regions with engine types aurora-mysql, aurora-postgresql, and mysql

Python
1
2
3
4
collector_configs = {
    'engines': ['aurora-mysql', 'aurora-postgresql', 'mysql'],
    'regions': [ 'cn-northwest-1', 'cn-north-1' ]
}

Configure Filters (Optional)

This collector script supports user-defined filters that allow users to filter target resources based on object properties. The filter function returns True or False.

  • True: Target resource needs to be collected.
  • False: Target resource does not need to be collected.

Supported object properties for filtering:

Property Description
name Instance ID
Engine Instance Engine
DBInstanceClass Instance Class
DBInstanceIdentifier Instance Identifier
AvailabilityZone Instance Availability Zone
SecondaryAvailabilityZone Backup Availability Zone
Endpoint_Address Endpoint Address
DBInstanceStatus Instance Status
Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# Example: Enable filters, filter by DBInstanceIdentifier and Engine properties of objects, configuration format as follows:
def filter_instance(instance):
    '''
    Collect instances where instance_id is xxx and engine is mysql
    '''
    # return True
    instance_id = instance['DBInstanceIdentifier']
    instance_engine = instance['Engine']
    if instance_id in ['xxx'] and instance_engine in ['mysql']:
        return True
    return False

@DFF.API('AWS-RDS Collection', timeout=3600, fixed_crontab='*/15 * * * *')
def run():
    Runner(main.DataCollector(account, collector_configs, filters=[filter_instance])).run()

3. Data Reporting Format

After data synchronization, you can view the data in the "Infrastructure - Resource Catalog" section of {{( brand_name }}}.

An example of reported data is as follows:

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
{
  "measurement": "aws_rds",
  "tags": {
    "name"                     : "xxxxx",
    "RegionId"                 : "cn-northwest-1",
    "Engine"                   : "mysql",
    "DBInstanceClass"          : "db.t3.medium",
    "DBInstanceIdentifier"     : "xxxxxx",
    "AvailabilityZone"         : "cn-northwest-1c",
    "SecondaryAvailabilityZone": "cn-northwest-1d"
  },
  "fields": {
    "InstanceCreateTime"  : "2018-03-28T19:54:07.871Z",
    "LatestRestorableTime": "2018-03-28T19:54:07.871Z",
    "Endpoint"            : "{JSON connection address data}",
    "AllocatedStorage"    : 100,
    "message"             : "{JSON serialized instance data}",
  }
}

Fields in tags and fields may change with subsequent updates

The value of tags.name is the instance ID, used as a unique identifier

fields.message and fields.Endpoint are both JSON serialized strings

4. IAM Policy Permissions

If the user collects resources using an IAM role, certain operation permissions must be enabled

This collector requires the following permissions:

rds:DescribeDBInstances

X. Appendix

Please refer to the AWS official documentation: