Deployment and Maintenance / Configuration and Data Files
This article mainly introduces the storage locations of configuration and data files required by DataFlux Func.
Various configuration and data files required for DataFlux Func operation are saved on the host machine by default and can be viewed directly on the host.
By default, the installation directory is /usr/local/dataflux-func
1. Installation Directory Configuration
After the initial installation of DataFlux Func, the installation script automatically records the installation directory for subsequent upgrades to be automatically installed to the same path. The configuration file is saved at the following location:
| Environment | Location |
|---|---|
| Host | /etc/dataflux-func |
2. Docker Stack Configuration
By default, the Docker Stack configuration file is saved at the following location:
| Environment | Location |
|---|---|
| Host | {installation directory}/docker-stack.yaml |
3. DataFlux Func Configuration
By default, the configuration file is saved at the following location:
| Environment | Location |
|---|---|
| Container | /data/user-config.yaml |
| Host | {installation directory}/data/user-config.yaml |
4. DataFlux Func Logs
By default, log files are saved at the following location:
| Environment | Location |
|---|---|
| Container | /data/logs/ |
| Host | {installation directory}/data/logs/ |
By default, log files are automatically rotated and compressed according to the logrotate configuration. The logrotate configuration file is located at:
| Environment | Location |
|---|---|
| Host | /etc/logrotate.d/dataflux-func |
5. DataFlux Func Database Backup
By default, the DataFlux Func database is automatically backed up using mysqldump and saved at the following location:
| Environment | Location |
|---|---|
| Container | /data/sqldump/ |
| Host | {installation directory}/data/sqldump/ |
6. DataFlux Func Resource Directory
By default, files and data generated during DataFlux Func operation are saved in the resource directory at the following location:
| Environment | Location |
|---|---|
| Container | /data/resources/ |
| Host | {installation directory}/data/resources/ |
The resource file directory may contain the following:
| Host Location | Description |
|---|---|
{installation directory}/data/resources/extra-python-packages/ |
Directory for Python packages installed via PIP tool |
{installation directory}/data/resources/user-python-packages/ |
Directory for Python packages used locally by users Users can upload their own Python packages and import them in scripts |
{installation directory}/data/resources/pre-run-scripts/ |
Pre-execution script directory Users can upload their own Bash scripts to run automatically when DataFlux Func restarts. For details, refer to Script Development / Pre-run Scripts |
{installation directory}/data/resources/script-market/ |
Local temporary files for the Script Market |
{installation directory}/data/resources/.downloads/ |
Temporary directory for downloads |
{installation directory}/data/resources/.uploads/ |
Temporary directory for uploads |
Developers / users can also store other required resource files in the resource directory for reading and use in scripts.
7. Built-in Redis Data Directory
If your DataFlux Func uses the built-in Redis, the Redis data is saved at the following location:
| Environment | Location |
|---|---|
| Host | {installation directory}/redis/ |
8. Built-in MySQL Data Directory
If your DataFlux Func uses the built-in MySQL, the MySQL data is saved at the following location:
| Environment | Location |
|---|---|
| Host | {installation directory}/mysql/ |
8.1 Database Table Data
DataFlux Func uses the following database tables. The table names and their purposes are as follows:
| Table Name | Data | Remarks |
|---|---|---|
biz_main_api_auth |
API Authentication | |
biz_main_blueprint |
Blueprints | |
biz_main_connector |
Connectors | |
biz_main_cron_job |
Scheduled Tasks | |
biz_main_env_variable |
Environment Variables | |
biz_main_file_service |
File Service | |
biz_main_func_store |
Function Store | |
biz_main_func |
Functions | |
biz_main_func_aou |
Function APIs | |
biz_main_operation_record |
Operation Records | |
biz_main_script_market |
Script Market | |
biz_main_script_publish_history |
Script Publish History | |
biz_main_script_recover_point |
Script Recovery Points | |
biz_main_script_set_export_history |
Export History | |
biz_main_script_set_import_history |
Import History | |
biz_main_script_set |
Script Sets | |
biz_main_script |
Scripts | |
biz_main_task_record_func |
Task Records (Functions) | |
biz_main_task_record |
Task Records | |
wat_main_access_key |
OpenAPI Access Keys | |
wat_main_system_setting |
System Settings | |
wat_main_user |
Users | |
biz_main_sync_api |
Synchronous APIs | Legacy table |
biz_main_async_api |
Asynchronous APIs | Legacy table |
biz_main_auth_link |
Authorization Links | Legacy table, the latest table uses biz_main_sync_api |
biz_main_crontab_config |
Automatic Trigger Configurations | Legacy table, the latest table uses biz_main_cron_job |
biz_main_batch |
Batch Processing | Legacy table, the latest table uses biz_main_async_api |
biz_main_batch_task_info |
Batch Task Information | Legacy table, deprecated in the latest version |
biz_main_crontab_task_info |
Automatic Trigger Task Information | Legacy table, deprecated in the latest version |
biz_main_script_failure |
Script Failure Information | Legacy table, deprecated in the latest version |
biz_main_script_log |
Script Log Information | Legacy table, deprecated in the latest version |
biz_main_task_info |
Task Information | Legacy table, deprecated in the latest version |
biz_main_task_result_dataflux_func |
DataFlux Func Task Results | Legacy table, deprecated in the latest version |
biz_rel_func_running_info |
Function Execution Information | Legacy table, deprecated in the latest version |
wat_main_task_result_example |
Example Task Records | Legacy table, deprecated in the latest version |
Some table data can be very large
The task record (functions) table biz_main_task_record_func may accumulate tens of GB or more of task record data after heavy use.
You can refer to Deployment and Maintenance / System Metrics and Task Records / Disable Local Function Task Records to disable "Local Function Task Records".