Skip to content

Script Development / DataKit, DataWay

The DataKit and DataWay connector objects primarily provide data writing methods.

The parameters for DFF.CONN(...) are as follows:

Parameter Type Required / Default Description
connector_id str Required Connector ID
source str None Specify Source
Note: Do not fill in values like "mysql" to prevent conflicts and confusion with other collectors
Parameter Type Required / Default Description
connector_id str Required Connector ID
token str None Specify Token
  • For general data reporting, use the .write_by_category(...) and .write_by_category_many(...) methods.
  • For general execution of DQL statements, use the .query(...) method.
  • To send a GET request directly, use the .get(...) method.
  • To send a POST request directly, use the .post_json(...) method.
  • To send line protocol data directly, use the .post_line_protocol(...) method.

This connector is essentially a wrapper for HTTP requests

Most interfaces between DataKit and DataWay are identical.

Since DataKit and DataWay interfaces change frequently, this connector does not provide one-to-one encapsulation for all interfaces.

As different versions of DataKit and DataWay may have different requirements or constraints for data reporting, please use this connector after reading the relevant documentation.

Detailed documentation can be found at:

.write_by_category(...)

Write data of a specific category to DataKit or DataWay. Parameters are as follows:

Parameter Type Required / Default Description
category str Required Data category. For details, see TrueWatch Documentation / DataKit API
measurement str Required Measurement name
tags dict Required Tags. Both keys and values must be strings.
fields dict Required Fields. Keys must be strings. Values can be strings, integers, floats, or booleans.
timestamp int/long/float {Current time} Timestamp, supports seconds, milliseconds, microseconds, or nanoseconds.
headers dict None Request Header parameters

Parameter headers added in version 3.3.0

Example
1
2
3
tags   = { 'host': 'web-01' }
fields = { 'cpu' : 10 }
status_code, result = datakit.write_by_category(category='metric', measurement='Host Monitoring', tags=tags, fields=fields)

.write_by_category_many(...)

The batch version of write_by_category(...). Parameters are as follows:

Parameter Type Required / Default Description
category str Required Data category. For details, see TrueWatch Documentation / DataKit API
data list Required List of data points
data[#].measurement str Required Measurement name
data[#].tags dict Required Tags. Both keys and values must be strings.
data[#].fields dict Required Fields. Keys must be strings. Values can be strings, integers, floats, or booleans.
data[#].timestamp int/long/float {Current time} Timestamp, supports seconds, milliseconds, microseconds, or nanoseconds.
headers dict None Request Header parameters

Parameter headers added in version 3.3.0

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
data = [
    {
        'measurement': 'Host Monitoring',
        'tags'       : { 'host' : 'web-01' },
        'fields'     : { 'value': 10 }
    },
    {
        'measurement': 'Host Monitoring',
        'tags'       : {'host' : 'web-02'},
        'fields'     : {'value': 20}
    }
]
status_code, result = datakit.write_by_category_many(category='metric', data=data)

.write_metric(...) / .write_point(...)

Legacy methods, equivalent to .write_by_category(category='metric', ...)

Example
1
status_code, result = datakit.write_metric(measurement='Host Monitoring', tags={'host': 'web-01'}, fields={'cpu': 10})

.write_metrics(...) / .write_metric_many(...) / .write_points(...)

Legacy methods, equivalent to .write_by_category_many(category='metric', ...)

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
data = [
    {
        'measurement': 'Host Monitoring',
        'tags'       : {'host' : 'web-01'},
        'fields'     : {'value': 10}
    },
    {
        'measurement': 'Host Monitoring',
        'tags'       : {'host' : 'web-02'},
        'fields'     : {'value': 20}
    }
]
status_code, result = datakit.write_metrics(data=data)

.write_logging(...) / .write_logging_many(...)

Legacy methods, equivalent to .write_by_category_many(category='logging', ...)

.query(...)

This method supports parameters from the DataKit and DataWay API DQL query interface.

Detailed documentation can be found at TrueWatch Documentation / DataKit API Documentation

This method is merely a wrapper for HTTP requests.

This method essentially just sends an HTTP request to DataKit or DataWay. The returned content depends on DataKit, DataWay, or the database.

If you have questions about the returned results, you can try using requests to send a request directly to DataKit or DataWay:

Using requests to call the interface
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
def query():
    domain = '<Domain>'
    token  = '<Token>'

    url = f'https://{domain}/v1/query/raw?token={token}'
    body = {
        'queries': [
            {
                # DQL statement
                'query': 'M::`cpu`:(`load5s`) BY `host`',

                # Last 1 hour
                'time_range': [
                    _DFF_TRIGGER_TIME_MS - 3600 * 1000,
                    _DFF_TRIGGER_TIME_MS,
                ],
            }
        ],
        'token': token
    }

    resp = requests.post(url, json=body)
    print(resp.status_code)
    print(resp.text)

Execute DQL statements via DataKit or DataWay. Parameters are as follows:

Parameter Type Required / Default Description
dql str Required DQL statement
dict_output bool False Whether to automatically convert data to dict.
raw bool False Whether to return the raw response. When enabled, the dict_output parameter is ignored.
all_series bool False Whether to automatically paginate using slimit and soffset to retrieve all time series.
{DataKit, DataWay native parameters} - - Passed through to queries[0].{DataKit, DataWay native parameters}
Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
import time
import json

@DFF.API('Run DQL via DataKit')
def run_dql_via_datakit():
    datakit = DFF.CONN('datakit')

    # Use the DataKit native parameter `time_range` to limit data to the last 1 hour.
    time_range = [
        int(time.time() - 3600) * 1000,
        int(time.time()) * 1000,
    ]

    # Query and return data in Dict format.
    status_code, result = datakit.query(dql='O::HOST:(host,load,create_time)', dict_output=True, time_range=time_range)
    print(json.dumps(result))
Output Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
  "series": [
    [
      {
        "time": 1622463105293,
        "host": "iZbp152ke14timzud0du15Z",
        "load": 2.18,
        "create_time": 1622429576363,
        "tags": {}
      },
      {
        "time": 1622462905921,
        "host": "ubuntu18-base",
        "load": 0.08,
        "create_time": 1622268259114,
        "tags": {}
      },
      {
        "time": 1622461264175,
        "host": "shenrongMacBook.local",
        "load": 2.395508,
        "create_time": 1622427320834,
        "tags": {}
      }
    ]
  ]
}
Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
import time
import json

@DFF.API('Run DQL via DataKit')
def run_dql_via_datakit():
    datakit = DFF.CONN('datakit')

    # Add the `raw` parameter to get the raw DQL query result.
    time_range = [
        int(time.time() - 3600) * 1000,
        int(time.time()) * 1000,
    ]

    # Query and return data in the original DataKit response format.
    status_code, result = datakit.query(dql='O::HOST:(host,load,create_time)', raw=True, time_range=time_range)
    print(json.dumps(result, indent=2))
Output Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
{
  "content": [
    {
      "series": [
        {
          "name": "HOST",
          "columns": [
            "time",
            "host",
            "load",
            "create_time"
          ],
          "values": [
            [
              1622463165152,
              "iZbp152ke14timzud0du15Z",
              1.92,
              1622429576363
            ],
            [
              1622462905921,
              "ubuntu18-base",
              0.08,
              1622268259114
            ],
            [
              1622461264175,
              "shenrongMacBook.local",
              2.395508,
              1622427320834
            ]
          ]
        }
      ],
      "cost": "1ms",
      "total_hits": 3
    }
  ]
}

.get(...)

This is a general-purpose processing method.

For specific parameter formats, content, etc., please refer to TrueWatch Documentation / DataKit API

Send a GET request to DataKit or DataWay. Parameters are as follows:

Parameter Type Required / Default Description
path str Required Request path
query dict None Request URL parameters
headers dict None Request Header parameters

.post_json(...)

This is a general-purpose processing method.

For specific parameter formats, content, etc., please refer to TrueWatch Documentation / DataKit API

Send a POST request in JSON format to DataKit or DataWay. Parameters are as follows:

Parameter Type Required / Default Description
path str Required Request path
json_obj dict/list Required JSON object to send
query dict None Request URL parameters
headers dict None Request Header parameters

Parameter path was adjusted to be the first parameter in version 1.6.8.

.post_line_protocol(...)

This is a general-purpose processing method.

For specific parameter formats, content, etc., please refer to TrueWatch Documentation / DataKit API

Send a POST request in line protocol format to DataKit or DataWay. Parameters are as follows:

Parameter Type Required / Default Description
path str Required Request path
points list Required List of data in point format
points[#].measurement str Required Measurement name
points[#].tags dict Required Tags. Both keys and values must be strings.
points[#].fields dict Required Fields. Keys must be strings. Values can be strings, integers, floats, or booleans.
points[#].timestamp int/long/float {Current time} Timestamp, supports seconds, milliseconds, microseconds, or nanoseconds.
query dict None Request URL parameters
headers dict None Request Header parameters

Parameter path was adjusted to be the first parameter in version 1.6.8.