Troubleshooting / Excessive MySQL Data Storage
After heavy usage of DataFlux Func for a period of time, the situation of excessive MySQL storage space may occur.
1. Cause
By default, DataFlux saves execution records of each function task in MySQL (including all print(...) logs). The saving strategy is as follows:
| Source of Function Tasks | Number of Function Task Records Saved |
|---|---|
| Sync API (Old Version: Authorized Links) | The last 100 tasks for each sync API (Old Version: Authorized Links) |
| Scheduled Tasks (Old Version: Auto-trigger Configurations) | The last 500 tasks for each scheduled task (Old Version: Auto-trigger Configurations) |
| Async API (Old Version: Batch Processing) | The last 100 tasks for each async API (Old Version: Batch Processing) |
| Connector Subscription Message Handling | The last 1,000 tasks for each connector |
Each function task record undergoes corresponding reduction with the following limitation strategy:
| Limitation | |
|---|---|
Length of Each print(...) Log |
3,000 characters |
| Total Length of Each Function Task Record Log | 50,000 characters, retained: First 20,000 characters Last 30,000 characters |
Since the system needs to retain a certain number of recent task records for each task source, when there are many configured sync APIs (Old Version: Authorized Links), scheduled tasks (Old Version: Auto-trigger Configurations), etc., they may occupy a large amount of MySQL storage space.
2. Solution
Disabling local function task records can significantly reduce MySQL storage pressure.
You can refer to Deployment and Maintenance / System Metrics and Task Records / Disable Local Function Task Records to disable "Local Function Task Records" and alleviate MySQL storage pressure.