Skip to content

Connecting and Operating Guance DataKit

This document primarily introduces how to use this system to connect with Datakit.

Always use the latest version of DataFlux Func for operations

1. Background

In some cases, independently installed DataFlux Func also needs to interact with Guance, including but not limited to:

  • DataFlux Func acts as a collector, reporting data to Guance
  • Business processing within DataFlux Func requires data from Guance

For the installation of DataFlux Func, please refer to:

For the usage and maintenance of DataFlux Func, please refer to:

2. Ensuring DataKit Connector and Connectivity

Before using DataFlux Func to write data into DataKit, connectivity must be ensured first. Therefore, after the DataKit installation is completed, the configuration needs to be adjusted to allow DataFlux Func to connect.

DataFlux Func and DataKit can be installed on the same server, and DataFlux Func will automatically search for locally installed DataKit upon startup.

However, since DataFlux Func runs via Docker Stack and bridges to the host's docker0, it does not directly connect to the host's local network. Therefore, even if DataFlux Func and DataKit are installed on the same server, it is not simple to bind the DataKit listening port to the local network (127.0.0.1).

At this point, the configuration should be modified to bind the listening port to docker0 (172.17.0.1) or 0.0.0.0. Summary as follows:

Bound IP Host DataFlux Func External Systems
127.0.0.1 Accessible via 127.0.0.1 Cannot access Cannot access
172.17.0.1 Accessible via 172.17.0.1 Accessible via 172.17.0.1 Cannot access
0.0.0.0 Accessible via 127.0.0.1, 172.17.0.1, etc. Accessible via 172.17.0.1 Accessible via host IP

Reference operations when DataKit is located on the host where DataFlux Func resides:

  1. Open DataKit configuration: sudo vim /usr/local/datakit/conf.d/datakit.conf
  2. Change http_listen = "localhost:9529" to http_listen = "0.0.0.0:9529"
  3. Restart DataKit: sudo datakit service -R
  4. Confirm that DataKit can be accessed via docker0: curl -i http://172.17.0.1:9529/v1/ping
  5. Return HTTP/1.1 200 OK indicates success

DataKit installation, deployment, and other documents can be found at: - Guance Documentation / DataKit - TrueWatch Documentation / DataKit

2.1 Using Automatically Generated DataKit Connectors

When DataFlux Func starts, it will automatically attempt to find possible DataKits in the host network.

If DataKit is successfully discovered, the system will automatically create its connector in the "Connectors" section, eliminating the need for manual addition by the user.

2.2 Manually Adding a DataKit Connector

In some cases, DataKit Func cannot automatically detect DataKit, such as:

  • DataFlux Func started before DataKit was installed
  • DataKit was not correctly configured, causing DataFlux Func to fail connecting to the local DataKit
  • DataKit configuration has been changed, such as using a non-default port
  • The DataKit to be connected resides on another server

In these situations, users need to manually add DataKit in the "Connectors":

Option Example Value Description
ID datakit Used for subsequent operations with DFF.CONN('{Connector ID}')
Host 172.17.0.1 Since DataFlux Func runs in Docker, enter the address of docker0 here to access DataKit on the host.
If DataKit is on a network location, enter the actual domain or IP
Port 9529 If you have changed the default listening port of DataKit, then enter the actual listening port here
Protocol HTTP Normal HTTP is sufficient.
If your DataKit is deployed in a network location with a domain and certificate, fill in accordingly
Source dataflux_func This is the input for DataKit.
Do not change it to mysql or similar inputs that already exist to avoid confusion

3. Writing Code to Operate DataKit

After creating the DataKit connector, you can operate DataKit in the code:

Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
@DFF.API('DataKit Demo')
def datakit_demo():
    # Get the DataKit operation object
    datakit = DFF.CONN('datakit')

    # Write data as metrics into DataKit
    datakit.write_metric(measurement='Metrics', tags={'Tag Name':'Tag Value'}, fields={'Field Name':'Field Value'})

    # Write data as logs into DataKit
    datakit.write_logging(measurement='Metrics', tags={'Tag Name':'Tag Value'}, fields={'Field Name':'Field Value'})

    # Query data using DQL statements
    data = datakit.query(dql='M:: Metrics')

X. More Documentation

For complete API documentation of the DataKit operation object, please refer to: