Skip to content

Collector Configuration Manual for "Azure-Monitor Metric Collection"

Before reading this, please first read:

Before using this collector, you must install the 'Integration Core Package' and its corresponding third-party dependency packages.

This collector supports enabling multiple threads by default (five threads are enabled by default). If you need to change the thread pool size, you can set the environment variable COLLECTOR_THREAD_POOL_SIZE.

1. Configuration Structure

The configuration structure of this collector is as follows:

Field Type Required Description
targets list Required Cloud monitoring collection object configuration list
The logical relationship between multiple configurations with the same namespace is "AND".
targets[#].namespace str Required Namespace of cloud monitoring to be collected (Azure resource type).
Example: 'Microsoft.Compute/virtualMachines'
Refer to the appendix for values.
targets[#].metrics list Required List of cloud monitoring metrics names to be collected
Refer to the appendix for values.
targets[#].metrics[#] str Required Metric name pattern, supporting "NOT" and wildcard matching.
Normally, the logical relationship between multiple entries is "OR".
When including "NOT", the logical relationship between multiple entries is "AND".
See below for details.
subscriptions list Required List of subscription IDs to be collected.
subscriptions[#] str Subscription ID
locations list Optional Region list
locations[#] str Region, example: westus2

2. Configuration Examples

Specifying Specific Metrics

Collect 2 metrics named CPU Credits Consumed and CPU Credits Remaining from VMs.

Python
1
2
3
4
5
6
7
8
collector_configs = {
    'targets': [
        {
            'namespace': 'Microsoft.Compute/virtualMachines',
            'metrics'  : ['CPU Credits Consumed', 'CPU Credits Remaining'],
        },
    ],
}

Wildcard Matching Metrics

Metric names can use * wildcards for matching.

In this example, the following metrics will be collected:

  • Metric named CPU Credits Consumed
  • Metrics starting with CPU
  • Metrics ending with Remaining
  • Metrics containing IOPS
Python
1
2
3
4
5
6
7
8
collector_configs = {
    'targets': [
        {
            'namespace': 'Microsoft.Compute/virtualMachines',
            'metrics'  : ['CPU Credits Consumed', 'CPU*', '*Remaining', '*IOPS*'],
        },
    ],
}

Excluding Some Metrics

Adding a "NOT" prefix indicates that the following metrics should be excluded.

In this example, the following metrics will not be collected:

  • Metric named CPU Credits Consumed
  • Metrics starting with CPU
  • Metrics ending with Remaining
  • Metrics containing IOPS
Python
1
2
3
4
5
6
7
8
collector_configs = {
    'targets': [
        {
            'namespace': 'Microsoft.Compute/virtualMachines',
            'metrics'  : ['NOT', 'CPU Credits Consumed', 'CPU*', '*Remaining', '*IOPS*'],
        },
    ],
}

Multiple Filtering to Specify Desired Metrics

The same namespace can be specified multiple times, filtering metrics sequentially from top to bottom.

In this example, it is equivalent to performing the following filtering steps on metric names:

  1. Select all metrics whose names contain CPU.
  2. From the results of the previous step, exclude the metric named CPU Credits Consumed.
Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
collector_configs = {
    'targets': [
        {
            'namespace': 'Microsoft.Compute/virtualMachines',
            'metrics'  : ['*CPU*'],
        },
        {
            'namespace': 'Microsoft.Compute/virtualMachines',
            'metrics'  : ['NOT', 'CPU Credits Consumed'],
        },
    ],
}

Configuring Filters (Optional)

This collector script supports user-defined filters, allowing users to filter target resources based on object properties. Filter functions return True or False.

  • True: Target resources should be collected.
  • False: Target resources should not be collected.

When custom object collection is enabled, more object properties can be filtered. For details, refer to the documentation for the custom object collector of the corresponding product.

Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# Example: Enable filters, filtering by resourceId attribute, with the following configuration format:

def filter_metrics(instance, namespace='Microsoft.Compute/virtualMachines'):
    '''
    Collect metrics for resources with IDs /subscriptions/xxxx/xxx1, /subscriptions/xxxx/xxx2
    '''
    resource_id = instance['tags'].get('resourceId')
    if resource_id in ['/subscriptions/xxxx/xxx1', '/subscriptions/xxxx/xxx2']:
        return True
    return False

from guance_integration__runner import Runner
import guance_azure_monitor__main as main

@DFF.API('Azure-Monitor Metric Collection', timeout=3600, fixed_crontab="*/5 * * * *")
def run():
    Runner(main.DataCollector(account, collector_configs, filters=[filter_metrics])).run()

When configuring multiple filters under the same namespace, all filters must be satisfied simultaneously for data to be reported.

3. Data Collection Explanation

Cloud Product Configuration Information

Cloud product configuration includes two necessary parameters: namespace and metrics. Below is an example using Microsoft.Compute/virtualMachines: 1. namespace: Namespace, which represents the Microsoft cloud product resource type (case-insensitive), value: "Microsoft.Compute/virtualMachines".

  1. metrics: List of metrics to collect. Refer to Supported Metrics for Microsoft.Compute/virtualMachines under the "Name in REST API" column. Value: ["Available Memory Bytes", "CPU Credits Consumed"]

4. Data Reporting Format

After data is synchronized normally, it can be viewed in the 'Metrics' section of TrueWatch.

For example, with the following collector configuration:

Python
1
2
3
4
5
6
7
8
collector_configs = {
    'targets': [
        {
            'namespace': 'Microsoft.Compute/virtualMachines',
            'metrics'  : ['CPU*'],
        },
    ],
}

An example of the reported data is as follows:

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
{
  "measurement": "azure_compute_virtualmachines",
  "tags": {
    "location"       : "eastus",
    "resource_group" : "func-resource",
    "resource_id"    : "/subscriptions/xxxx/xxx1",
    "subscription_id": "63a4a998-bd43-4cca-bd86-c4e7ed33a643",
    "tenant_id"      : "ce9fe5b4-ba02-4c9a-b54e-f0bbff18579c",
    "unit"           : "Count"
  },
  "fields": {
    "cpu_credits_consumed_average" : 0.0,
    "cpu_credits_remaining_average": 98.97
  }
}

All metric values are reported as float types.

Since Azure raw data field keys do not have a unified naming convention, the collector standardizes them into lowercase underscores when reporting.

5. Coordination with Custom Object Collectors

When other custom object collectors (such as VirtualMachines) are running within the same DataFlux Func, this collector will automatically attempt to match the tags.resourceId fields with the tags.name fields in custom objects (tags.name in custom objects is derived from resourceId).

Since custom object information must be known first to enable coordination in cloud monitoring collectors, it is generally recommended to place the cloud monitoring collector at the end of the list, such as:

Python
1
2
3
4
5
    # Create collector
    collectors = [
        azure_vm.DataCollector(account, common_azure_configs),
        azure_monitor.DataCollector(account, monitor_collector_configs), # Cloud monitoring collectors are usually placed at the end
    ]

Upon successful matching, fields from the matched custom object tags will be added to the tags of the monitoring data. The specific effect is as follows:

Assuming the original data collected by cloud monitoring is as follows:

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
{
  "measurement"  : "azure_compute_virtualmachines",
  "tags": {
    "resource_id": "/subscriptions/xxxx/xxx1",
    "unit"       : "Count",
    "{other fields}"   : "{omitted}"
  },
  "fields": {
    "{metric}"      : "{metric value}"
  }
}

At the same time, the custom object data collected by the Microsoft cloud VM collector is as follows:

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
{
  "measurement": "azure_compute_virtualmachines",
  "tags": {
    "name"          : "/subscriptions/xxxx/xxx1",
    "resource_id"   : "/subscriptions/xxxx/xxx1",
    "power_state"   : "running",
    "computer_name" : "test_vm",
    "zone"          : "1",
    "{other fields}"     : "{omitted}"
  },
  "fields": {
    "{other fields}": "{omitted}"
  }
}

Thus, the final cloud monitoring data reported is as follows:

JSON
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
{
"measurement": "azure_compute_virtualmachines",
  "tags": {
    "name"          : "/subscriptions/xxxx/xxx1",
    "resource_id"    : "/subscriptions/xxxx/xxx1",
    "power_state"    : "running",
    "computer_name"  : "test_vm",
    "zone"          : "1",
    "unit"          : "Count",
    "{other fields}"   : "{omitted}"
  },
  "fields": {
    "{metric}": "{metric value}"
  }
}

6. Explanation of Cloud Monitoring API Call Limits

Azure Monitor imposes free quota limits on some API calls (currently: Query API free quota 1 million times/month, exceeding costs $0.01 USD per thousand calls), and the / {resourceUri} / providers / Microsoft.Insights / metrics used by this collector is also within the limit. Below is a detailed explanation of the number of calls made by the script set:

1. Users have multiple resources and need to collect various monitoring items, determining whether they exceed the free quota:

This collector uses / {resourceUri} / providers / Microsoft.Insights / metrics (querying resource metric values) where one request can obtain multiple monitoring items for a resource under a certain dimension (maximum 20 monitoring items with the same timeGrain, exceeding requires another request).

2. Find the actual number of calls by checking task execution logs:

The collector counts the number of API calls for each task execution result, which can be viewed in the logs, for example:

Bash
1
2
3
4
[2023-10-24 14:31:53.203] [+6569ms] Completed collection for account 【1】, total execution time 【16784 milliseconds】, called API 【2 times】
[2023-10-24 14:31:53.203] [+6569ms] Detailed calls as follows:
[2023-10-24 14:31:53.203] [+6569ms] -> management.azure.com/{resourceUri}/providers/microsoft.insights/metricdefinitions: 1 time
[2023-10-24 14:31:53.203] [+6569ms] -> management.azure.com/{resourceUri}/providers/microsoft.insights/metrics: 1 time

Given that cloud monitoring call counts have free quotas, users are advised to configure monitoring items as needed to avoid additional costs due to wildcard usage.

Notes

Common Errors and Solutions

X. Appendix

Azure related documents: