Skip to content

Script Development / DataKit, DataWay

DataKit, DataWay connector operation objects mainly provide data writing methods.

DFF.CONN(...) parameters are as follows:

Parameter Type Required / Default Description
connector_id str Required Connector ID
source str None Specify Source
Note: Do not fill in "mysql", etc., to avoid conflicts
Parameter Type Required / Default Description
connector_id str Required Connector ID
token str None Specify Token
  • For general data reporting, use .write_by_category(...), .write_by_category_many(...) methods
  • For general DQL statement execution, use .query(...) method
  • For direct GET requests, use .get(...) method
  • For direct POST requests, use .post_json(...) method
  • For direct line protocol data, use .post_line_protocol(...) method

This connector is essentially a wrapper for HTTP requests

The majority of interfaces between DataKit and DataWay are identical"

Since DataKit and DataWay interfaces frequently change, this connector does not encapsulate all interfaces one-to-one

Since different versions of DataKit and DataWay may have different requirements or constraints for reported data, please use this connector after reading the relevant documentation

Detailed documentation can be found at:

.write_by_category(...)

Write specific types of data to DataKit, DataWay, parameters are as follows:

Parameter Type Required / Default Description
category str Required Data type, see TrueWatch Guide / DataKit API
measurement str Required Measurement name
tags dict Required Tags. Both key and value must be strings
fields dict Required Fields. Key must be a string, value can be string/integer/float/boolean
timestamp int/long/float {Current time} Timestamp, supports seconds/milliseconds/microseconds/nanoseconds.
headers dict None Request Header parameters

Parameter headers added in 3.3.0

Example
1
2
3
tags   = { 'host': 'web-01' }
fields = { 'cpu' : 10 }
status_code, result = datakit.write_by_category(category='metric', measurement='Host Monitoring', tags=tags, fields=fields)

.write_by_category_many(...)

Bulk version of write_by_category(...), parameters are as follows:

Parameter Type Required / Default Description
category str Required Data type, see TrueWatch Guide / DataKit API
data list Required List of data points
data[#].measurement str Required Measurement name
data[#].tags dict Required Tags. Both key and value must be strings
data[#].fields dict Required Fields. Key must be a string, value can be string/integer/float/boolean
data[#].timestamp int/long/float {Current time} Timestamp, supports seconds/milliseconds/microseconds/nanoseconds.
headers dict None Request Header parameters

Parameter headers added in 3.3.0

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
data = [
    {
        'measurement': 'Host Monitoring',
        'tags'       : { 'host' : 'web-01' },
        'fields'     : { 'value': 10 }
    },
    {
        'measurement': 'Host Monitoring',
        'tags'       : {'host' : 'web-02'},
        'fields'     : {'value': 20}
    }
]
status_code, result = datakit.write_by_category_many(category='metric', data=data)

.write_metric(...) / .write_point(...)

Legacy method, equivalent to .write_by_category(category='metric', ...)

Example
1
status_code, result = datakit.write_metric(measurement='Host Monitoring', tags={'host': 'web-01'}, fields={'cpu': 10})

.write_metrics(...) / .write_metric_many(...) / .write_points(...)

Legacy method, equivalent to .write_by_category_many(category='metric', ...)

Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
data = [
    {
        'measurement': 'Host Monitoring',
        'tags'       : {'host' : 'web-01'},
        'fields'     : {'value': 10}
    },
    {
        'measurement': 'Host Monitoring',
        'tags'       : {'host' : 'web-02'},
        'fields'     : {'value': 20}
    }
]
status_code, result = datakit.write_metrics(data=data)

.write_logging(...) / .write_logging_many(...)

Legacy method, equivalent to .write_by_category_many(category='logging', ...)

.query(...)

This method supports parameters from the DataKit, DataWay API DQL query interface

Detailed documentation can be found at TrueWatch Guide / DataKit API Guide

This method is just a wrapper for HTTP requests

This method essentially sends an HTTP request to DataKit, DataWay, and the returned content depends on DataKit, DataWay / database

If you have questions about the returned results, you can try using requests to send a request directly to DataKit, DataWay:

Using requests to call the interface
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
def query():
    domain = '<Domain>'
    token  = '<Token>'

    url = f'https://{domain}/v1/query/raw?token={token}'
    body = {
        'queries': [
            {
                # DQL statement
                'query': 'M::`cpu`:(`load5s`) BY `host`',

                # Last 1 hour
                'time_range': [
                    _DFF_TRIGGER_TIME_MS - 3600 * 1000,
                    _DFF_TRIGGER_TIME_MS,
                ],
            }
        ],
        'token': token
    }

    resp = requests.post(url, json=body)
    print(resp.status_code)
    print(resp.text)

Execute DQL statements through DataKit, DataWay, parameters are as follows:

Parameter Type Required / Default Description
dql str Required DQL statement
dict_output bool False Whether to automatically convert data to dict.
raw bool False Whether to return the raw response. If enabled, dict_output is invalid.
all_series bool False Whether to automatically paginate through slimit and soffset.
{DataKit, DataWay native params} - - Pass through to queries[0].{DataKit, DataWay native params}
Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
import time
import json

@DFF.API('Run DQL via DataKit')
def run_dql_via_datakit():
    datakit = DFF.CONN('datakit')

    # Use DataKit native parameter `time_range` to limit data to the last 1 hour
    time_range = [
        int(time.time() - 3600) * 1000,
        int(time.time()) * 1000,
    ]

    # Query and return data in Dict format
    status_code, result = datakit.query(dql='O::HOST:(host,load,create_time)', dict_output=True, time_range=time_range)
    print(json.dumps(result))
Output Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
  "series": [
    [
      {
        "time": 1622463105293,
        "host": "iZbp152ke14timzud0du15Z",
        "load": 2.18,
        "create_time": 1622429576363,
        "tags": {}
      },
      {
        "time": 1622462905921,
        "host": "ubuntu18-base",
        "load": 0.08,
        "create_time": 1622268259114,
        "tags": {}
      },
      {
        "time": 1622461264175,
        "host": "shenrongMacBook.local",
        "load": 2.395508,
        "create_time": 1622427320834,
        "tags": {}
      }
    ]
  ]
}
Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
import time
import json

@DFF.API('Run DQL via DataKit')
def run_dql_via_datakit():
    datakit = DFF.CONN('datakit')

    # Add raw parameter to get raw DQL query value
    time_range = [
        int(time.time() - 3600) * 1000,
        int(time.time()) * 1000,
    ]

    # Query and return data in DataKit raw response format
    status_code, result = datakit.query(dql='O::HOST:(host,load,create_time)', raw=True, time_range=time_range)
    print(json.dumps(result, indent=2))
Output Example
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
{
  "content": [
    {
      "series": [
        {
          "name": "HOST",
          "columns": [
            "time",
            "host",
            "load",
            "create_time"
          ],
          "values": [
            [
              1622463165152,
              "iZbp152ke14timzud0du15Z",
              1.92,
              1622429576363
            ],
            [
              1622462905921,
              "ubuntu18-base",
              0.08,
              1622268259114
            ],
            [
              1622461264175,
              "shenrongMacBook.local",
              2.395508,
              1622427320834
            ]
          ]
        }
      ],
      "cost": "1ms",
      "total_hits": 3
    }
  ]
}

.get(...)

This method is a general-purpose processing method

Specific parameter formats, content, etc., please refer to TrueWatch Guide / DataKit API

Send a GET request to DataKit, DataWay, parameters are as follows:

Parameter Type Required / Default Description
path str Required Request path
query dict None Request URL params
headers dict None Request Header params

.post_json(...)

This method is a general-purpose processing method

Specific parameter formats, content, etc., please refer to TrueWatch Guide / DataKit API

Send a POST request in JSON format to DataKit, DataWay, parameters are as follows:

Parameter Type Required / Default Description
path str Required Request path
json_obj dict/list Required JSON object to send
query dict None Request URL params
headers dict None Request Header params

Parameter path adjusted to the first parameter in version 1.6.8

.post_line_protocol(...)

This method is a general-purpose processing method

Specific parameter formats, content, etc., please refer to TrueWatch Guide / DataKit API

Send a POST request in line protocol format to DataKit, DataWay, parameters are as follows:

Parameter Type Required / Default Description
path str Required Request path
points list Required List of data points in point format
points[#].measurement str Required Measurement name
points[#].tags dict Required Tags. Both key and value must be strings
points[#].fields dict Required Fields. Key must be a string, value can be string/integer/float/boolean
points[#].timestamp int/long/float {Current time} Timestamp, supports seconds/milliseconds/microseconds/nanoseconds.
query dict None Request URL params
headers dict None Request Header params

Parameter path adjusted to the first parameter in version 1.6.8