Skip to content

Deployment and Maintenance / Standalone Deployment

This article mainly introduces how to directly install and deploy the standalone version of DataFlux Func on a server.

For installing DataFlux Func using Helm in k8s, please refer to Deployment and Maintenance / Installation and Deployment / Helm Deployment

1. Download the Installation Package

DataFlux Func supports downloading the required installation files and then installing them in an environment without public network access using a USB drive or other mobile devices.

The downloaded "installation package" comes with an automatic installation script, Docker, etc., and can be executed to install (details below).

1.1 One-Command Download

For Linux, macOS, and other systems, it is recommended to use the official shell command to download the installation package.

Run the following command to automatically download the required installation files for DataFlux Func. The download script will automatically select the x86_64 or aarch64 architecture version based on the current environment:

Please confirm system requirements and server configuration before installation

Bash
1
/bin/bash -c "$(curl -fsSL docs.dataflux-func.com/download)"

If the architecture of the download is different from the server architecture where it will be installed, you can specify the architecture by adding the architecture option:

Bash
1
/bin/bash -c "$(curl -fsSL docs.dataflux-func.com/download)" -- --arch=x86_64
Bash
1
/bin/bash -c "$(curl -fsSL docs.dataflux-func.com/download)" -- --arch=aarch64

After the command is executed, all installation files will be saved in the automatically created dataflux-func-portable-{architecture}-{version} directory.

  • If you need to install DataFlux Func on a server without public network access, you can download it on your local machine first, then copy the entire directory to the target machine using a USB drive or other mobile storage devices, or use the scp tool.
  • If you need to install DataFlux Func on a server with public network access, you can download it directly on the server.

1.2 Manual Download

For systems where it is inconvenient to use shell commands, you can manually download the required installation files.

If you need to manually download, here is the complete file list:

# Content File Name x86_64 Architecture (Original) aarch64 Architecture (Original)
1 Docker Binary docker-24.0.9.tgz Download Download
2 DataFlux Func Image dataflux-func.tar.gz Download Download
3 MySQL/MariaDB Image mysql.tar.gz Download Download
4 Redis Image redis.tar.gz Download Download
5 Docker Service Config File docker.service Download Download
6 DataFluxFunc Installation Script run-portable.sh Download Download
7 Docker Stack Config Template docker-stack.example.yaml Download Download
8 Image List image-list Download Download
9 Version Info version Download Download

After manually downloading all installation files, place them in the same directory.

1.3 Download Options

When executing the download script, you can specify download options to meet personalized needs.

Example:

Note: There is a -- (two hyphens) before the specified download options

Bash
1
/bin/bash -c "$(curl -fsSL docs.dataflux-func.com/download)" -- --download-dir=func-download

Supported download options are as follows:

--arch={architecture} Specify Architecture

The download script defaults to downloading the installation package files that match the architecture of the local machine. If you need to download installation files for a different architecture, you can specify this parameter.

Available architectures are as follows:

Architecture Description Parameter Example
Intel Intel / AMD 64-bit processors --arch=x86_64
ARM ARM64v8 processors --arch=aarch64

--download-dir={download directory} Specify Download Directory

Added in version 2.6.1

This parameter is suitable for automated script deployment of DataFlux Func

The download script defaults to creating/clearing the dataflux-func-portable-{architecture}-{version} directory in the current directory and downloading the installation files there.

If you need to download to a specified directory, you can specify this parameter, such as:

Bash
1
/bin/bash -c "$(curl -fsSL docs.dataflux-func.com/download)" -- --download-dir=func-download

2. Execute Installation

Enter the installation package directory downloaded in the previous section and run the one-click installation script run-portable.sh:

Please confirm system requirements and server configuration before installation

DataFlux Func does not support macOS, Windows, please copy it to a Linux system before running the installation

Bash
1
sudo /bin/bash {installation file directory}/run-portable.sh

Using the automatic installation script can achieve a quick installation and setup within minutes. The automatic configuration includes:

  • Running MySQL, Redis, DataFlux Func (including Server, Worker, Beat)
  • Automatically creating and saving all data in the /usr/local/dataflux-func/ directory (including MySQL data, Redis data, DataFlux Func configuration, log files, etc.)
  • Randomly generating MySQL root user password and system Secret, and saving them in the DataFlux Func configuration file
  • Redis has no password
  • MySQL, Redis are not accessible externally

After execution, you can use a browser to access http://{server IP address/domain}:8088 for the initialization operation interface.

If the running environment has poor performance, you should use the docker ps command to confirm that all components have successfully started before accessing (see the list below)

  1. dataflux-func_server
  2. dataflux-func_worker-0
  3. dataflux-func_worker-1
  4. dataflux-func_worker-2
  5. dataflux-func_worker-3
  6. dataflux-func_worker-5
  7. dataflux-func_worker-6
  8. dataflux-func_beat
  9. dataflux-func_mysql
  10. dataflux-func_redis
  1. dataflux-func_server
  2. dataflux-func_worker-0
  3. dataflux-func_worker-1-6
  4. dataflux-func_worker-7
  5. dataflux-func_worker-8-9
  6. dataflux-func_beat
  7. dataflux-func_mysql
  8. dataflux-func_redis

2.1 Installation Options

When executing the installation script, you can specify installation options to meet personalized needs, such as:

Bash
1
sudo /bin/bash run-portable.sh --port=80

--mini Install Mini Version

For low-configuration environments, this installation mode saves resources.

When enabled:

  • Only a single Worker is started to listen to all queues
  • Heavy load tasks are more likely to cause queue blocking and lag
  • System tasks and function tasks share the processing queue and may affect each other
  • System requirements are reduced to:
    • CPU cores >= 1
    • Memory >= 2GB
  • If built-in MySQL, Redis are not used, system requirements can be further reduced

--port={port number} Specify Listening Port

DataFlux Func defaults to using port 8088 for access. If this port is occupied by another program, you can choose another port, such as 9000.

--install-dir={installation directory} Specify Installation Directory

When you need to install to a different path than the default /usr/local/dataflux-func, you can specify this parameter

--no-mysql Disable Built-in MySQL

When you need to use an existing MySQL database, you can specify this parameter to disable MySQL startup on the local machine.

After enabling this option, you need to specify the correct MySQL connection information on the configuration page after installation

--no-redis Disable Built-in Redis

When you need to use an existing Redis database, you can specify this parameter to disable Redis startup on the local machine.

After enabling this option, you need to specify the correct Redis connection information on the configuration page after installation

--auto-setup Automatically Execute Configuration

Added in version 2.6.0

This parameter is suitable for automated script deployment of DataFlux Func

When enabled, configuration and database initialization are automatically performed, and there will be no configuration page, such as:

Bash
1
sudo /bin/bash run-portable.sh --auto-setup

Additionally, when the --auto-setup option is enabled, you can add other --auto-setup-* options to adjust the automatic configuration. The extra options for automatic configuration are as follows:

Automatic Configuration Extra Options Default Value Description
--auto-setup-admin-username={username} admin Specify the administrator username
--auto-setup-admin-password={password} admin Specify the administrator password
--auto-setup-ak-secret={AK Secret} Automatically generated Automatically create AK
and use the specified value as the AK Secret
--auto-setup-ak-id={AK ID} ak-auto-setup AK ID when automatically creating AK
(requires the --auto-setup-ak-secret option)

If you need to automatically configure and specify the administrator password as AdminPass, the complete command is as follows:

Bash
1
sudo /bin/bash run-portable.sh --auto-setup --auto-setup-admin-password='AdminPass'

3. Verify Installation

By default, DataFlux Func comes with some example scripts after installation.

Perform the following steps to verify the installation:

  1. Click the "Script Editor" in the top navigation bar, then select "Script Lib" - "Examples" - "Basic Demo" in the left sidebar
  2. In the right script editor, click "Edit" to enter edit mode, select the "hello_world" function and click the "Execute" button to run the function
  3. At this point, if you can see the function's return value in the "Script Output" at the bottom

At this point, the installation verification is complete

3.1 Service Descriptions

By default, after DataFlux Func is successfully started, the following services are running:

Queues #7, #8, #9 are reserved for the Data Platform attached version of Func, and are reserved queues in the standalone version of Func

Name Description
dataflux-func_server The front-end service of DataFlux Func.
Mainly provides the web interface, API interface, etc.
dataflux-func_worker-0 Python worker unit listening to queues #0, #4, #7, #8, #9.
Mainly handles internal tasks of DataFlux Func (#0 queue) and tasks mistakenly assigned to reserved queues
dataflux-func_worker-1 Python worker unit listening to queue #1.
Mainly handles synchronously executed function APIs
dataflux-func_worker-2 Python worker unit listening to queue #2.
Mainly handles cron jobs
dataflux-func_worker-3 Python worker unit listening to queue #3.
Mainly handles asynchronously executed function APIs
dataflux-func_worker-5 Python worker unit listening to queue #5.
Mainly handles debugging tasks for scripts run in the web interface
dataflux-func_worker-6 Python worker unit listening to queue #6.
Mainly handles tasks executed by connector subscription messages
dataflux-func_beat The trigger for cron jobs, only one can be enabled globally
dataflux-func_mysql Built-in MySQL for DataFlux Func
dataflux-func_redis Built-in Redis for DataFlux Func
Name Description
dataflux-func_server The front-end service of DataFlux Func.
Mainly provides the web interface, API interface, etc.
dataflux-func_worker-0 Python worker unit listening to queue #0.
Mainly handles internal tasks of DataFlux Func
dataflux-func_worker-1-6 Python worker unit listening to queues #1, #2, #3, #4, #5, #6.
Mainly handles synchronously executed function APIs
dataflux-func_worker-7 Python worker unit listening to queue #7.
Mainly handles debugging tasks for scripts run in the web interface
dataflux-func_worker-8-9 Python worker unit listening to queues #8, #9.
Mainly handles asynchronous tasks (cron jobs, etc.)
dataflux-func_beat The trigger for cron jobs, only one can be enabled globally
dataflux-func_mysql Built-in MySQL for DataFlux Func
dataflux-func_redis Built-in Redis for DataFlux Func

3.2 Data Storage Locations

DataFlux Func requires storage of various data during operation. The main content and storage locations are as follows:

Storage Storage Path Storage Content
MySQL {installation directory}/mysql/ Most of the data generated during UI operations, including but not limited to:
1. Scripts, connector configurations, environment variables
2. User information, function APIs, cron jobs
3. Operation records, script execution records, import/export records, etc.
Redis {installation directory}/redis/ Mainly used for caching and queues, including but not limited to:
1. User login information
2. Various caches established during script execution
3. Script execution task queues
4. Func self-monitoring data, etc.
Directory {installation directory}/data/ Mainly used for data that must exist in file form, see below for details
Directory {installation directory}/data/resources/ Resource folder (i.e., the root directory of "File Service")
Directory {installation directory}/data/resources/extra-python-packages/ Third-party packages installed via PIP
Directory {installation directory}/data/resources/script-market/ Script Market download data
Directory {installation directory}/data/sqldump/ Database automatic backups
Directory {installation directory}/data/logs/ System logs
File {installation directory}/data/user-config.yaml DataFlux Func system configuration
File /etc/dataflux-func DataFlux Func installation directory record
Used to correctly obtain the current installation location of DataFlux Func during upgrades

4. Using External Databases

When installing DataFlux Func for the first time, if you need to use external databases (MySQL, Redis), you can specify to disable the built-in MySQL and Redis when executing the installation script, such as:

Bash
1
sudo /bin/bash {installation file directory}/run-portable.sh --no-mysql --no-redis

After installation, on the first run interface, click "Show More Configuration" and modify the MySQL and Redis configurations.

For MySQL and Redis configuration requirements, please refer to Deployment and Maintenance / System Requirements

If you need to migrate the existing DataFlux Func database to an external one, please refer to Deployment and Maintenance / Daily Operations / Migrating Databases