Deployment and Maintenance / Data Storage Location

This article mainly describes where the configuration and data files required by DataFlux Func are stored.

By default, all configuration and data files required for DataFlux Func operation are stored on the host machine, where they can be viewed directly.

By default, the installation directory is /usr/local/dataflux-func

1. Installation Directory Configuration

After the first installation of DataFlux Func, the installation script will automatically record the installation directory so that subsequent upgrades are installed to the same path. The configuration file is stored in the following location:

Environment	Location
Host machine	/etc/dataflux-func

2. Docker Stack Configuration

By default, the Docker Stack configuration file is stored in the following location:

Environment	Location
Host machine	{installation-directory}/docker-stack.yaml

3. DataFlux Func Configuration

By default, the configuration file is stored in the following location:

Environment	Location
Container	/data/user-config.yaml
Host machine	{installation-directory}/data/user-config.yaml

4. DataFlux Func Logs

By default, log files are stored in the following location:

Environment	Location
Container	/data/logs/
Host machine	{installation-directory}/data/logs/

By default, log files are automatically rotated and compressed according to the logrotate configuration. The logrotate configuration file is stored in the following location:

Environment	Location
Host machine	/etc/logrotate.d/dataflux-func

5. DataFlux Func Database Backup

By default, the DataFlux Func database is automatically backed up using mysqldump. The backup is stored in the following location:

Environment	Location
Container	/data/sqldump/
Host machine	{installation-directory}/data/sqldump/

6. DataFlux Func Resource Directory

By default, files and data generated during DataFlux Func operation are saved in the resource directory. The location is as follows:

Environment	Location
Container	/data/resources/
Host machine	{installation-directory}/data/resources/

The resource directory may contain the following items:

Host location	Description
{installation-directory}/data/resources/extra-python-packages/	Directory for Python packages installed via the PIP tool
{installation-directory}/data/resources/user-python-packages/	Directory for Python packages used locally by users Users can upload Python packages themselves and import them in their Script
{installation-directory}/data/resources/pre-run-scripts/	Pre-run Scripts directory Users can upload Bash scripts themselves, and they will be run automatically when DataFlux Func restarts. For details, see Script Development / Pre-run Scripts
{installation-directory}/data/resources/script-market/	Local temporary files for the Script Market
{installation-directory}/data/resources/.downloads/	Temporary directory for downloads
{installation-directory}/data/resources/.uploads/	Temporary directory for uploads

Developers / users can also store other required resource files in the resource directory by themselves, so that they can be read and used in Scripts.

7. Built-in Redis Data Directory

If your DataFlux Func uses the built-in Redis, Redis data is stored as follows:

Environment	Location
Host machine	{installation-directory}/redis/

8. Built-in MySQL Data Directory

If your DataFlux Func uses the built-in MySQL, MySQL data is stored as follows:

Environment	Location
Host machine	{installation-directory}/mysql/

8.1 Database Table Data

DataFlux Func uses the following database tables; the table names and their purposes are as follows:

Table name	Data	Remarks
biz_main_api_auth	API Authentication
biz_main_blueprint	Blueprint
biz_main_connector	Connector
biz_main_cron_job	Cron Job
biz_main_env_variable	Environment Variable
biz_main_file_service	File Service
biz_main_func_store	Function Storage
biz_main_func	Function
biz_main_func_aou	Func API
biz_main_operation_record	Operation Record
biz_main_script_market	Script Market
biz_main_script_publish_history	Script Publish History
biz_main_script_recover_point	Script Restore Point
biz_main_script_set_export_history	Export History
biz_main_script_set_import_history	Import History
biz_main_script_set	Script Set
biz_main_script	Script
biz_main_task_record_func	Task Record (Function)
biz_main_task_record	Task Record
wat_main_access_key	OpenAPI Access Key
wat_main_system_setting	System Settings
wat_main_user	User
biz_main_sync_api	Sync API	Legacy table
biz_main_async_api	Async API	Legacy table
biz_main_auth_link	Auth Link	Legacy table; the latest table now uses biz_main_sync_api
biz_main_crontab_config	Auto Trigger Configuration	Legacy table; the latest table now uses biz_main_cron_job
biz_main_batch	Batch	Legacy table; the latest table now uses biz_main_async_api
biz_main_batch_task_info	Batch Task Information	Legacy table; deprecated in the latest version
biz_main_crontab_task_info	Auto Trigger Task Information	Legacy table; deprecated in the latest version
biz_main_script_failure	Script Failure Information	Legacy table; deprecated in the latest version
biz_main_script_log	Script Log Information	Legacy table; deprecated in the latest version
biz_main_task_info	Task Information	Legacy table; deprecated in the latest version
biz_main_task_result_dataflux_func	DataFlux Func Task Result	Legacy table; deprecated in the latest version
biz_rel_func_running_info	Function Execution Information	Legacy table; deprecated in the latest version
wat_main_task_result_example	Example Task Record	Legacy table; deprecated in the latest version

Some table data may be extremely large

The task record (function) table biz_main_task_record_func may leave tens of GB or more of task record data after heavy use

You can refer to Deployment and Maintenance / System Metrics and Task Records / Disable Local Function Task Record to disable "Local Function Task Record"