Skip to content

Script Development / DataKit, DataWay

The operation objects of the DataKit and DataWay connectors mainly provide data writing methods.

DFF.CONN(...) parameters are as follows:

Parameter Type Required / Default Value Description
connector_id str Required Connector ID
source str None Specifies Source
Note: Do not fill in "mysql" etc., to avoid conflicts with other collectors
Parameter Type Required / Default Value Description
connector_id str Required Connector ID
token str None Specifies Token
  • For general data reporting, please use .write_by_category(...) and .write_by_category_many(...) methods.
  • For executing general DQL statements, please use the .query(...) method.
  • To directly send a GET request, use the .get(...) method.
  • To directly send a POST request, use the .post_json(...) method.
  • To directly send line protocol data, use the .post_line_protocol(...) method.

This connector is essentially an encapsulation of HTTP requests

The vast majority of interfaces between DataKit and DataWay are completely identical.

Due to frequent changes in the interfaces of DataKit and DataWay, this connector does not fully encapsulate all the interfaces one-to-one.

Since different versions of DataKit and DataWay may have different requirements or constraints for data reporting, please use this connector based on reading the relevant documentation.

Detailed documentation can be found at:

.write_by_category(...)

Write specific types of data to DataKit or DataWay. Parameters are as follows:

Parameter Type Required / Default Value Description
category str Required Data type, see:
　Guance Documentation / DataKit API:
　TrueWatch Documentation / DataKit API
measurement str Required Measurement name
tags dict Required Tags. Both key names and values must be strings
fields dict Required Metrics. Key names must be strings, values can be strings/integers/floating-point numbers/booleans
timestamp int/long/float {Current Time} Timestamp, supports seconds/milliseconds/microseconds/nanoseconds
headers dict None Request Header parameters

The parameter headers was added in version 3.3.0

Example
1
2
3
tags   = { 'host': 'web-01' }
fields = { 'cpu' : 10 }
status_code, result = datakit.write_by_category(category='metric', measurement='HOST Monitoring', tags=tags, fields=fields)

.write_by_category_many(...)

Batch version of write_by_category(...). Parameters are as follows:

Parameter Type Required / Default Value Description
category str Required Data type, see:
　Guance Documentation / DataKit API:
　TrueWatch Documentation / DataKit API
data list Required List of data points
data[#].measurement str Required Measurement name
data[#].tags dict Required Tags. Both key names and values must be strings
data[#].fields dict Required Metrics. Key names must be strings, values can be strings/integers/floating-point numbers/booleans
data[#].timestamp int/long/float {Current Time} Timestamp, supports seconds/milliseconds/microseconds/nanoseconds
headers dict None Request Header parameters

The parameter headers was added in version 3.3.0

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
data = [
    {
        'measurement': 'HOST Monitoring',
        'tags'       : { 'host' : 'web-01' },
        'fields'     : { 'value': 10 }
    },
    {
        'measurement': 'HOST Monitoring',
        'tags'       : {'host' : 'web-02'},
        'fields'     : {'value': 20}
    }
]
status_code, result = datakit.write_by_category_many(category='metric', data=data)

.write_metric(...) / .write_point(...)

Old method, equivalent to .write_by_category(category='metric', ...).

Example
1
status_code, result = datakit.write_metric(measurement='HOST Monitoring', tags={'host': 'web-01'}, fields={'cpu': 10})

.write_metrics(...) / .write_metric_many(...) / .write_points(...)

Old method, equivalent to .write_by_category_many(category='metric', ...).

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
data = [
    {
        'measurement': 'HOST Monitoring',
        'tags'       : {'host' : 'web-01'},
        'fields'     : {'value': 10}
    },
    {
        'measurement': 'HOST Monitoring',
        'tags'       : {'host' : 'web-02'},
        'fields'     : {'value': 20}
    }
]
status_code, result = datakit.write_metrics(data=data)

.write_logging(...) / .write_logging_many(...)

Old method, equivalent to .write_by_category_many(category='logging', ...).

.query(...)

This method supports the parameters from the DataKit API DQL query interface

Detailed documentation can be found at:

Execute DQL statements via DataKit or DataWay. Parameters are as follows:

Parameter Type Required / Default Value Description
dql str Required DQL statement
dict_output bool False Whether to automatically convert data to dict.
raw bool False Whether to return the raw response. If enabled, the dict_output parameter becomes invalid.
all_series bool False Whether to automatically paginate through slimit and soffset to retrieve all time series.
{DataKit, DataWay native parameters} - - Passed through to queries[0].{DataKit, DataWay native parameters}
Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
import time
import json

@DFF.API('Run DQL via DataKit')
def run_dql_via_datakit():
    datakit = DFF.CONN('datakit')

    # Use the native DataKit parameter `time_range` to limit data to the last hour
    time_range = [
        int(time.time() - 3600) * 1000,
        int(time.time()) * 1000,
    ]

    # Query and return data in Dict format
    status_code, result = datakit.query(dql='O::HOST:(host,load,create_time)', dict_output=True, time_range=time_range)
    print(json.dumps(result))
Output Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
  "series": [
    [
      {
        "time": 1622463105293,
        "host": "iZbp152ke14timzud0du15Z",
        "load": 2.18,
        "create_time": 1622429576363,
        "tags": {}
      },
      {
        "time": 1622462905921,
        "host": "ubuntu18-base",
        "load": 0.08,
        "create_time": 1622268259114,
        "tags": {}
      },
      {
        "time": 1622461264175,
        "host": "shenrongMacBook.local",
        "load": 2.395508,
        "create_time": 1622427320834,
        "tags": {}
      }
    ]
  ]
}
Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
import time
import json

@DFF.API('Run DQL via DataKit')
def run_dql_via_datakit():
    datakit = DFF.CONN('datakit')

    # Add the `raw` parameter to get the original value of the DQL query
    time_range = [
        int(time.time() - 3600) * 1000,
        int(time.time()) * 1000,
    ]

    # Query and return data in the original DataKit return value format
    status_code, result = datakit.query(dql='O::HOST:(host,load,create_time)', raw=True, time_range=time_range)
    print(json.dumps(result, indent=2))
Output Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
{
  "content": [
    {
      "series": [
        {
          "name": "HOST",
          "columns": [
            "time",
            "host",
            "load",
            "create_time"
          ],
          "values": [
            [
              1622463165152,
              "iZbp152ke14timzud0du15Z",
              1.92,
              1622429576363
            ],
            [
              1622462905921,
              "ubuntu18-base",
              0.08,
              1622268259114
            ],
            [
              1622461264175,
              "shenrongMacBook.local",
              2.395508,
              1622427320834
            ]
          ]
        }
      ],
      "cost": "1ms",
      "total_hits": 3
    }
  ]
}

.get(...)

This method is a general processing method

Specific parameter formats and content should refer to:

Send a GET request to DataKit or DataWay. Parameters are as follows:

Parameter Type Required / Default Value Description
path str Required Request path
query dict None Request URL parameters
headers dict None Request Header parameters

.post_json(...)

This method is a general processing method

Specific parameter formats and content should refer to:

Send a POST request with JSON format to DataKit or DataWay. Parameters are as follows:

Parameter Type Required / Default Value Description
path str Required Request path
json_obj dict/list Required JSON object to be sent
query dict None Request URL parameters
headers dict None Request Header parameters

Parameter path was adjusted to the first parameter in version 1.6.8

.post_line_protocol(...)

This method is a general processing method

Specific parameter formats and content should refer to:

Send a POST request with line protocol format to DataKit or DataWay. Parameters are as follows:

Parameter Type Required / Default Value Description
path str Required Request path
points list Required List of data points in data point format
points[#].measurement str Required Measurement name
points[#].tags dict Required Tags. Both key names and values must be strings
points[#].fields dict Required Metrics. Key names must be strings, values can be strings/integers/floats/booleans
points[#].timestamp int/long/float {Current Time} Timestamp, supports seconds/milliseconds/microseconds/nanoseconds
query dict None Request URL parameters
headers dict None Request Header parameters

Parameter path was adjusted to the first parameter in version 1.6.8