Skip to content

Configuration Manual for the "AWS-Billing (Instance Level)" Collector

Before reading this, please read:

Before using this collector, you must install the 'Integration Core Package' and its corresponding third-party dependency packages

To ensure data completeness, the billing data collected is from the day before the collection date

The collector requires setting account_name and account_id in extra_tags

Before the first execution of the collector, you need to enable Hourly and Resource Level Data in Amazon Web Services Console / Amazon Cost Explorer / Cost Management / Preferences, and wait 24 hours to obtain the data

1. Configuration Structure

This collector does not require additional configuration.

2. Data Reporting Format

After the data is synchronized normally, it can be viewed in the 'Cloud Billing Explorer' of TrueWatch.

An example of the reported data is as follows:

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
{
  "measurement": "cloud_billing",
  "tags": {
    "billing_id"    : "anvxfafakjflxxxxxxx",
    "product_code": "Amazon Elastic Compute Cloud - Compute",
    "product_name": "Amazon Elastic Compute Cloud - Compute",
    "intance_id"  : "i-xxxxxxxx",
    "billing_date"        : "2022-09-09",
    "cloud_provider"      : "aws",
    "account_id"  : "xxxxxx",
    "account_name": "xxxxxx"
  },
  "fields": {
    "amount"        : 9419.23
  },
  "timestamp": 1657507021
}

Fields in tags and fields may change with subsequent updates.

3. Field Description

Reported Field API Returned Field Meaning of API Field
Billing ID billing_id Unique billing ID
Amount Paid amount BlendedCost Price paid by user
Product Code product_code Product code
Product Name product_name Product name
Instance ID intance_id Billing instance ID
Billing Period billing_date Date of resource consumption record. Format: YYYY-MM-DD
Account Name account_name Account name
Account ID account_id Account ID

4. Coordination with Custom Object Collectors

When other custom object collectors (such as EC2, RDS) are running in the same DataFlux Func, this collector will automatically attempt to match the name field of the custom objects' tags.name. When a successful match occurs, all fields except name from the matched custom object's tags will be added to the tags of the billing data.

Assume the original billing data collected is as follows:

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
{
  "measurement": "cloud_billing",
  "tags": {
    "billing_id"    : "anvxfafakjflxxxxxxx",
    "product_code": "ec2",
    "product_name": "Amazon Elastic Compute Cloud - Compute",
    "name"                : "i-xxxxxxxx",
    "billing_date"        : "2022-09-09",
    "cloud_provider"      : "aws",
    "account_id"  : "xxxxx",
    "account_name": "xxxxx"
  },
  "fields": {
    "amount": 9419.23
  },
  "timestamp": 1657507021
}

At the same time, AWS EC2 collector collects the following custom object data:

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
{
  "measurement": "aws_ec2",
  "tags": {
    "name"       : "i-xxxxxxxx",
    "InstanceId" : "i-xxxxxxxx",
    "RegionId"   : "cn-hangzhou",
    "{other_fields}" : "{略}"
  },
  "fields": {
    "{other_fields}": "{略}"
  }
}

Thus, the final reported billing data would look like this:

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
{
"measurement": "cloud_billing",
  "tags": {
    "billing_id"    : "anvxfafakjflxxxxxxx",
    "instance_id"          : "i-xxxxxxxx",
    "region_id"            : "cn-hangzhou",
    "product_code": "ec2",
    "product_name": "Amazon Elastic Compute Cloud - Compute",
    "billing_date"        : "2022-09-09",
    "cloud_provider"      : "aws",
    "account_id"  : "xxxxx",
    "account_name": "xxxxx",
    "{other_fields}"          : "{略}"
  },
  "fields": {
    "amount": 9419.23,
  },
  "timestamp": 1657507021
}

5. Explanation of Cloud Billing API Call Counts

Each page call to the AWS Cost Explorer API costs $0.01. Below is a detailed explanation of the number of calls made by the script set:

. Find the actual number of calls through task execution logs:

The collector keeps track of the number of API calls per task execution, which can be seen in the logs. For example:

Bash
1
2
3
4
[2023-07-04 11:35:18.497] [+0ms] The 【1st】 account has finished collection, total execution time 【759 milliseconds】, during which 【2】 API calls were made.
[2023-07-04 11:35:18.498] [+0ms] Detailed calls are as follows:
[2023-07-04 11:35:18.498] [+0ms] -> ce.cn-northwest-1.amazonaws.com.cn/?Action=get_dimension_values: 1 time
[2023-07-04 11:35:18.498] [+0ms] -> ce.cn-northwest-1.amazonaws.com.cn/?Action=get_cost_and_usage_with_resources: 1 time

Appendix

AWS-Billing Management "Query Instance Billing Service"