Skip to content

Collector Configuration Manual for "AWS-KinesisAnalytics"

Before reading this, please first read:

Before using this collector, you must install the 'Integration Core package' and its associated third-party dependency packages

1. Configuration Structure

The configuration structure of this collector is as follows:

Field Type Required Description
regions list Required List of regions to collect data from
regions[#] str Required Region ID. For example: 'cn-north-1'
See the full table in the appendix

2. Configuration Example

Collecting instance data from the Beijing region

Python
1
2
3
collector_configs = {
    'regions': ['cn-north-1' ]
}

Configuring Filters (Optional)

This collector script supports user-defined filters that allow users to filter target resources by object attributes. The filter function returns True or False.

  • True: The target resource needs to be collected.
  • False: The target resource does not need to be collected.
Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# Example: Enable filters, filtering by the ApplicationName property of the object, with the following configuration format:
def filter_instance(instance):

    application_name = instance['tags'].get('ApplicationName')

    if application_name in ['xxx'] :
        return True
    return False


@DFF.API('AWS-KinesisAnalytics Collection', timeout=3600, fixed_crontab='* * * * *')
def run():
    Runner(main.DataCollector(account, collector_configs, filters=[filter_instance])).run()

3. Data Reporting Format

After the data is synchronized successfully, you can view it in the "Infrastructure - Resource Catalog" section of {{( brand_name }}}.

The reported data example is as follows:

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
{
  "measurement": "aws_kinesis_analytics",
  "tags": {
    "ApplicationARN": "arn:aws-cn:xxxx:xxxx",
    "ApplicationMode": "STREAMING",
    "ApplicationName": "zsh_test",
    "ApplicationStatus": "RUNNING",
    "ApplicationVersionId": "3",
    "RegionId": "cn-northwest-1",
    "RuntimeEnvironment": "FLINK-1_15",
    "name": "zsh_test"
  },
  "fields": {
    "message"     : "{Instance JSON Data}"
  }
}

Fields in tags and fields may change with subsequent updates.

4. IAM Policy Permissions

If the user collects resources using an assumed IAM role, certain operation permissions need to be enabled.

This collector requires the following permissions:

kinesisanalytics:ListApplications

X. Appendix

Please refer to the AWS official documentation: