Troubleshooting / Excessive MySQL Storage Usage
After heavy usage of DataFlux Func for a period of time, you may encounter a situation where MySQL storage space becomes excessively large.
1. Causes
By default, DataFlux saves the execution records of each function task (including all print(...) logs) in MySQL. The retention policy is as follows:
| Function Task Source | Number of Function Task Records Saved |
|---|---|
| Sync API (Legacy: Auth Link) | Last 100 tasks for each Sync API (Legacy: Auth Link) |
| Cron Job (Legacy: Auto Trigger Config) | Last 500 tasks for each Cron Job (Legacy: Auto Trigger Config) |
| Async API (Legacy: Batch) | Last 100 tasks for each Async API (Legacy: Batch) |
| Connector Subscription Message Handling | Last 1,000 tasks for each Connector |
Each function task record is subject to the following reduction limits:
| Limit | |
|---|---|
Length of each print(...) log |
3,000 characters |
| Total log length of each function task record | 50,000 characters, retaining: First 20,000 characters Last 30,000 characters |
Since the system needs to retain a certain number of the latest task records for each task source, when there are a large number of configured Sync APIs (Legacy: Auth Links), Cron Jobs (Legacy: Auto Trigger Configs), etc., it may occupy a significant amount of MySQL storage space.
2. Solutions
Disabling local function task records can significantly reduce MySQL storage pressure.
You can refer to the Deployment and Maintenance Guide / System Metrics and Task Records / Disable Local Function Task Records to disable "Local Function Task Records" and alleviate MySQL storage pressure.