Skip to content

Configuration Manual for the "Huawei Cloud - RDS Error Log Statistics Collector"

Before reading this article, please read the following first:

Before using this collector, you must install the 'Integration Core Package' and its corresponding third-party dependency packages

The code execution of this script depends on RDS instance object collection. If custom object collection for RDS is not configured, the error log script will be unable to collect error log data.

1. Configuration Structure

The configuration structure for this collector is as follows:

Configuration One (Recommended)

Field Type Required Description
regions list Required List of regions to collect data from
regions[#] str Required Region ID. For example: 'cn-north-4'
Refer to the appendix for a complete list

The collector will automatically retrieve all IAM projects under the region and then fetch resources based on these IAM projects.

Configuration Two (Backward Compatibility)

Field Type Required Description
region_projects dict Required List of 'Region - Project ID' pairs to collect data from
region_projects[#] str:list Required In the key-value pairs:
Key represents the region (e.g., 'cn-north-4')
Value represents the list of project IDs under that region to collect data from
Refer to the appendix for a complete list
log_level str Optional The log level of the error logs to collect. Available options: ALL, INFO, LOG, NOTE, WARNING, ERROR, FATAL, PANIC

The severity of log levels from lowest to highest are ALL, INFO, LOG, NOTE, WARNING, ERROR, FATAL, PANIC. If not configured, it collects all log levels. When configured for a specific level, it collects logs of that level and higher. Default configuration collects logs at ERROR level and higher. For example, if log_level is set to WARNING, it collects logs at WARNING, ERROR, FATAL, and PANIC levels.

2. Configuration Example

Specifying Region

Collecting RDS error log data for projects in the cn-north-4 region.

Python
1
2
3
collector_configs = {
    'regions': ['cn-north-4']
}

Specifying Log Level

Collecting error logs at ERROR level and higher.

Python
1
2
3
collector_configs = {
    'log_level': 'ERROR'
}

3. Data Reporting Format

After the data is synchronized successfully, it can be viewed in the 'Logs' section of TrueWatch.

An example of the reported data is as follows:

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
{
  "measurement": "huaweicloud_rds_errorlog",
  "tags": {
    "RegionId"               : "cn-north-4",
    "project_id"             : "c631f046252dxxxxxxxf253c62d48585",
    "enterprise_project_id"  : "0",
    "instance_id"            : "1d0c91561f4644dxxxxxxd68304b0520din01",
    "instance_name"          : "rds-df54-xxxx",
    "status"                 : "ACTIVE",
    "type"                   : "Single",
    "engine"                 : "MySQL",
    "level"                  : "NOTE"
  },
  "fields": {
    "content"                : "Shutting down plugin '",
    "time"                   : "2023-04-25T06:27:10Z",
    "message"                : "{JSON serialized log data}"
  }
}

4. Notes

Fields in tags and fields may change with subsequent updates.

fields.message is a JSON serialized string.

5. Troubleshooting

When running the program, you might encounter the following error messages:

Text Only
1
!!!!!! HuaweiCloud client.do_api Error Message: ClientRequestException - {status_code:404,request_id:a6ade677c6a4659ede84e24a2f3b25a3,error_code:DBS.280110,error_msg:Selected DB instance does not exist. }

The error message will print relevant information about the request, such as:

Text Only
1
!!!!!! HuaweiCloud client.do_api resource:rds, resource_path:/v3/c631ff253c62d48585/instances/c37ab0506e9a183394in01/error-logs, r_p_params:{'project_id': 'c631f04653c62d48585'}, region: cn-north-4, project_id: c631f04625da452d48585, method:POST, root_domain: myhuaweicloud.com, params: {'instance_id': 'c37ab0506e9a4298b39075983394in01', 'body': {'start_time': '2023-04-26T05:01:22+0000', 'end_time': '2023-04-26T07:01:22+0000', 'limit': 60, 'level': 'PANIC'}}

Reason: When trying to access error log data for an RDS instance, the instance has already been released, causing the API to throw an error.

Solution: Ignore the error.

X. Appendix

HuaweiCloud-RDS "Regions"

Please refer to the HuaweiCloud official documentation:

HuaweiCloud-RDS "Error Log Information Documentation"

Please refer to the HuaweiCloud official documentation: