Skip to content

Collector "AWS-Kafka" Configuration Manual

Before reading this article, please read the following first:

Before using this collector, you must install the «Integration Core Package» and its associated third-party dependency packages

1. Configuration Structure

The configuration structure of this collector is as follows:

Field Type Required? Description
regions list Required List of regions to collect data from
regions[#] str Required Region ID. Example: 'cn-north-1'
See full table in appendix

2. Configuration Example

Collecting instance data from the Beijing region

Python
1
2
3
collector_configs = {
    'regions': ['cn-north-1' ]
}

3. Data Reporting Format

After data synchronization is successful, the data can be viewed in the «Infrastructure - Resource Catalog» section of {{( brand_name }}}.

An example of the reported data is as follows:

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
{
  "measurement": "aws_kafka",
  "tags": {
    "ClusterArn"    : "arn:aws:kafka:ap-southeast-1:",
    "ClusterName"   : "kafka",
    "ClusterType"   : "PROVISIONED",
    "CurrentVersion": "K1JYRMDPD0WRC1",
    "State"         : "ACTIVE",
    "name"          : "kafka"
  },
  "fields": {
    "BrokerNodeGroupInfo" : "{}",
    "ClientAuthentication": "{}",
    "CreationTime"        : "2022-12-02T07:36:10.439000Z",
    "StateInfo"           : "{}",
    "Tags"                : "{\"GCR-Migration-2021-463\": \"d-serv79n\", \"map-migrated-project-id\": \"GCR-3\"}",
    "message"             : "{JSON data for instance}"
  }
}

Fields in tags and fields may change with subsequent updates

4. IAM Policy Permissions

If the user collects resources using an assumed IAM role, certain operation permissions need to be enabled

This collector requires the following permissions:

kafka:ListClustersV2

X. Appendix

Please refer to the official AWS documentation: