Deployment and Maintenance / Benchmark Performance Testing

This article mainly describes how to conduct benchmark performance testing on DataFlux Func.

1. Preface

All example results in this article were obtained from tests in the following hardware environment:

	Description
Computer	HP ProBook laptop
CPU	AMD Ryzen 5 7530U with Radeon Graphics / 4.5 GHz
Memory	32 GB / 3200 MHz (0.3 ns)
OS	Ubuntu 22.04.5 LTS

2. Benchmark Test Package

Download Benchmark.zip and import it into DataFlux Func

The Benchmark test package contains the following:

Content	Description
Script Set benchmark
Function benchmark__main.hello_world	An empty function that directly returns "ok", used for concurrency testing
Function benchmark__main.json_dump	Performs JSON serialization / deserialization, used for performance testing
Function benchmark__main.calc_pi	Calculates pi, used for performance testing
Sync API benchmark-hello-world	Used for concurrency testing
Cron Job benchmark-hello-world	Used for testing task scheduling performance
Cron Job benchmark-compute-pi	Used for testing computation performance (calculating pi)
Cron Job benchmark-json-dump-and-load	Used for testing computation performance (JSON serialization / deserialization)

3. Executing Tests

After importing the Benchmark test package, since the Cron Jobs run automatically, you can view the results from the task records after a few minutes.

However, concurrency performance testing of DataFlux Func requires the use of other tools.

3.1 Testing Task Scheduling Performance

After importing the Benchmark test package, wait a few minutes, and you can view the results in the "Hello, World" task record under "Cron Job".

Generally speaking, the time consumed by each task should be within 10 milliseconds.

3.2 Testing Computation Performance

After importing the Benchmark test package, wait a few minutes, and you can view the results in the "Compute pi" and "JSON dump and load" task records under "Cron Job".

For the test environment described in the "Preface", each task takes about 1 second:

• Calculate pi "Compute pi"

• JSON serialization / deserialization "JSON dump and load"

3.3 Testing Func API Concurrency Performance

When testing concurrency performance, you can use the ab (ApacheBench) tool.

If ab is not installed yet, you can install it with the following commands:

Install via aptInstall via yum

apt-get install apache2-utils

yum install httpd-tools

Use ab to call and test "Hello, World" in the test package. The command is as follows:

Func on the local machineFunc on another server

ab -c 10 -n 5000 -k http://localhost:8088/api/v1/sync/benchmark-hello-world

ab -c 10 -n 5000 -k http://{DataFlux Func IP or domain}:8088/api/v1/sync/benchmark-hello-world

For the test environment described in the "Preface", the concurrency of the empty function is around 1000 (Requests per second):

# ab -c 10 -n 5000 -k http://localhost:8088/api/v1/sync/benchmark-hello-world
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 500 requests
Completed 1000 requests
...
Completed 5000 requests
Finished 5000 requests


Server Software:
Server Hostname:        localhost
Server Port:            8088

Document Path:          /api/v1/sync/benchmark-hello-world
Document Length:        13 bytes

Concurrency Level:      10
Time taken for tests:   4.694 seconds
Complete requests:      5000
Failed requests:        0
Keep-Alive requests:    5000
Total transferred:      3045525 bytes
HTML transferred:       65000 bytes
Requests per second:    1065.17 [#/sec] (mean)
Time per request:       9.388 [ms] (mean)
Time per request:       0.939 [ms] (mean, across all concurrent requests)
Transfer rate:          633.60 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       1
Processing:     5    9   1.8      9      29
Waiting:        4    9   1.8      9      29
Total:          5    9   1.8      9      29

Percentage of the requests served within a certain time (ms)
  50%      9
  66%     10
  75%     10
  80%     10
  90%     12
  95%     13
  98%     14
  99%     15
  100%     29 (longest request)

4. Factors Affecting Performance

Since the actual execution content of a task is determined by the Script actually called, if the task execution takes a long time, you can check whether the code has the following issues:

4.1 Excessive print(...)

For debugging convenience, someone may directly output all the data in the DB and the data returned by the API through print(...) in the Script.

Since DataFlux Func automatically records the print(...) content of task execution, such operations can cause significant performance loss.

It is recommended to keep only the necessary print(...) output after the code is put into production, and avoid outputting large blocks of text.

4.2 Slow Response from External Systems

When querying external databases or calling API interfaces (including executing DQL queries, etc.) in the Script, if network congestion occurs, or simply because the other system responds slowly, it will also cause long task execution time.

This is basically unrelated to DataFlux Func. You should contact the owners of these external systems to improve response speed.

The following is a simple example code for testing processing time:

import time
import requests

def test():
    t1 = time.time()
    requests.get('http://github.com')
    print(f'Cost: {time.time() -  t1} s')