Troubleshooting / Excessive MySQL Storage Usage
After heavy usage of DataFlux Func for a period of time, you may encounter the issue of excessive MySQL storage usage.
1. Cause
By default, DataFlux saves the execution records of each Func task (including all print(...) logs) in MySQL. The saving strategy is as follows:
| Func Task Source | Number of Func Task Records Saved |
|---|---|
| Func API | Last 100 tasks for each Func API |
| Cron Job | Last 500 tasks for each Cron Job |
| Connector Subscription Message Processing | Last 1,000 tasks for each Connector |
Each Func task record is reduced according to the following limits:
| Limit | |
|---|---|
Length of each print(...) log |
3,000 characters |
| Total log length for each Func task record | 50,000 characters, retaining: First 20,000 characters Last 30,000 characters |
Since the system needs to retain a certain number of the latest task records for each task source, when there are a large number of configured Func APIs, Cron Jobs, etc., it may occupy a significant amount of MySQL storage space.
2. Solution
Disabling local Func task records can greatly reduce MySQL storage pressure.
You can refer to the Deployment and Maintenance Guide / System Metrics and Task Records / Disable Local Func Task Record to disable "Local Func Task Records" and alleviate MySQL storage pressure.