Skip to content

Script Development / DataKit, DataWay

The DataKit and DataWay Connector primarily provides methods for data writing.

DFF.CONN(...) parameters are as follows:

Parameter Type Required / Default Description
connector_id str Required Connector ID
source str None Specify Source
Note: Avoid using "mysql" to prevent conflicts with other collectors
Parameter Type Required / Default Description
connector_id str Required Connector ID
token str None Specify Token
  • For general data reporting, use .write_by_category(...), .write_by_category_many(...) methods
  • For general DQL execution, use .query(...) method
  • For direct GET requests, use .get(...) method
  • For direct POST requests, use .post_json(...) method
  • For direct line protocol data sending, use .post_line_protocol(...) method

This Connector is essentially an encapsulation of HTTP requests

Most interfaces between DataKit and DataWay are identical"

Since DataKit and DataWay interfaces frequently change, this Connector does not encapsulate all interfaces one-to-one

Since different versions of DataKit and DataWay may have different requirements or constraints for data reporting, please use this Connector after reading the relevant documentation

Detailed documentation can be found at:

.write_by_category(...)

Write specific types of data to DataKit or DataWay, parameters are as follows:

Parameter Type Required / Default Description
category str Required Data type, see TrueWatch Guide / DataKit API
measurement str Required Measurement name
tags dict Required Tags. Both key names and key values must be strings
fields dict Required Metrics. Key names must be strings, key values can be strings/integers/floats/booleans
timestamp int/long/float {Current time} Timestamp, supports seconds/milliseconds/microseconds/nanoseconds.
headers dict None Request Header parameters

Parameter headers added in version 3.3.0

Example
1
2
3
tags   = { 'host': 'web-01' }
fields = { 'cpu' : 10 }
status_code, result = datakit.write_by_category(category='metric', measurement='Host Monitoring', tags=tags, fields=fields)

.write_by_category_many(...)

Batch version of write_by_category(...), parameters are as follows:

Parameter Type Required / Default Description
category str Required Data type, see TrueWatch Guide / DataKit API
data list Required List of data points
data[#].measurement str Required Measurement name
data[#].tags dict Required Tags. Both key names and key values must be strings
data[#].fields dict Required Metrics. Key names must be strings, key values can be strings/integers/floats/booleans
data[#].timestamp int/long/float {Current time} Timestamp, supports seconds/milliseconds/microseconds/nanoseconds.
headers dict None Request Header parameters

Parameter headers added in version 3.3.0

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
data = [
    {
        'measurement': 'Host Monitoring',
        'tags'       : { 'host' : 'web-01' },
        'fields'     : { 'value': 10 }
    },
    {
        'measurement': 'Host Monitoring',
        'tags'       : {'host' : 'web-02'},
        'fields'     : {'value': 20}
    }
]
status_code, result = datakit.write_by_category_many(category='metric', data=data)

.write_metric(...) / .write_point(...)

Legacy methods, equivalent to .write_by_category(category='metric', ...)

Example
1
status_code, result = datakit.write_metric(measurement='Host Monitoring', tags={'host': 'web-01'}, fields={'cpu': 10})

.write_metrics(...) / .write_metric_many(...) / .write_points(...)

Legacy methods, equivalent to .write_by_category_many(category='metric', ...)

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
data = [
    {
        'measurement': 'Host Monitoring',
        'tags'       : {'host' : 'web-01'},
        'fields'     : {'value': 10}
    },
    {
        'measurement': 'Host Monitoring',
        'tags'       : {'host' : 'web-02'},
        'fields'     : {'value': 20}
    }
]
status_code, result = datakit.write_metrics(data=data)

.write_logging(...) / .write_logging_many(...)

Legacy methods, equivalent to .write_by_category_many(category='logging', ...)

.query(...)

This method supports parameters from the DataKit API DQL query interface

Detailed documentation can be found at TrueWatch Guide / DataKit API Guide

Execute DQL statements via DataKit or DataWay, parameters are as follows:

Parameter Type Required / Default Description
dql str Required DQL statement
dict_output bool False Whether to automatically convert data to dict.
raw bool False Whether to return raw response. When enabled, dict_output parameter is ignored.
all_series bool False Whether to automatically paginate using slimit and soffset to get all time series.
{DataKit, DataWay native parameters} - - Pass through to queries[0].{DataKit, DataWay native parameters}
Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
import time
import json

@DFF.API('Run DQL via DataKit')
def run_dql_via_datakit():
    datakit = DFF.CONN('datakit')

    # Use DataKit native parameter `time_range` to limit data to the last 1 hour
    time_range = [
        int(time.time() - 3600) * 1000,
        int(time.time()) * 1000,
    ]

    # Query and return data in Dict format
    status_code, result = datakit.query(dql='O::HOST:(host,load,create_time)', dict_output=True, time_range=time_range)
    print(json.dumps(result))
Output Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
  "series": [
    [
      {
        "time": 1622463105293,
        "host": "iZbp152ke14timzud0du15Z",
        "load": 2.18,
        "create_time": 1622429576363,
        "tags": {}
      },
      {
        "time": 1622462905921,
        "host": "ubuntu18-base",
        "load": 0.08,
        "create_time": 1622268259114,
        "tags": {}
      },
      {
        "time": 1622461264175,
        "host": "shenrongMacBook.local",
        "load": 2.395508,
        "create_time": 1622427320834,
        "tags": {}
      }
    ]
  ]
}
Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
import time
import json

@DFF.API('Run DQL via DataKit')
def run_dql_via_datakit():
    datakit = DFF.CONN('datakit')

    # Add raw parameter to get raw DQL query results
    time_range = [
        int(time.time() - 3600) * 1000,
        int(time.time()) * 1000,
    ]

    # Query and return data in DataKit raw response format
    status_code, result = datakit.query(dql='O::HOST:(host,load,create_time)', raw=True, time_range=time_range)
    print(json.dumps(result, indent=2))
Output Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
{
  "content": [
    {
      "series": [
        {
          "name": "HOST",
          "columns": [
            "time",
            "host",
            "load",
            "create_time"
          ],
          "values": [
            [
              1622463165152,
              "iZbp152ke14timzud0du15Z",
              1.92,
              1622429576363
            ],
            [
              1622462905921,
              "ubuntu18-base",
              0.08,
              1622268259114
            ],
            [
              1622461264175,
              "shenrongMacBook.local",
              2.395508,
              1622427320834
            ]
          ]
        }
      ],
      "cost": "1ms",
      "total_hits": 3
    }
  ]
}

.get(...)

This is a general-purpose method

For specific parameter formats and content, refer to TrueWatch Guide / DataKit API

Send a GET request to DataKit or DataWay, parameters are as follows:

Parameter Type Required / Default Description
path str Required Request path
query dict None Request URL parameters
headers dict None Request Header parameters

.post_json(...)

This is a general-purpose method

For specific parameter formats and content, refer to TrueWatch Guide / DataKit API

Send a POST request in JSON format to DataKit or DataWay, parameters are as follows:

Parameter Type Required / Default Description
path str Required Request path
json_obj dict/list Required JSON object to send
query dict None Request URL parameters
headers dict None Request Header parameters

Parameter path adjusted to be the first parameter in version 1.6.8

.post_line_protocol(...)

This is a general-purpose method

For specific parameter formats and content, refer to TrueWatch Guide / DataKit API

Send a POST request in line protocol format to DataKit or DataWay, parameters are as follows:

Parameter Type Required / Default Description
path str Required Request path
points list Required List of data points in line protocol format
points[#].measurement str Required Measurement name
points[#].tags dict Required Tags. Both key names and key values must be strings
points[#].fields dict Required Metrics. Key names must be strings, key values can be strings/integers/floats/booleans
points[#].timestamp int/long/float {Current time} Timestamp, supports seconds/milliseconds/microseconds/nanoseconds.
query dict None Request URL parameters
headers dict None Request Header parameters

Parameter path adjusted to be the first parameter in version 1.6.8