Troubleshooting / Too Much Data Stored in MySQL
After heavy use of DataFlux Func for a period of time, the MySQL storage space may become too large.
1. Cause
By default, DataFlux saves the execution records of every function task in MySQL (including all print(...) logs), with the following retention policy:
| Function Task Source | Number of Saved Function Task Records |
|---|---|
| Func API | Last 100 tasks per Func API |
| Cron Job | Last 500 tasks per Cron Job |
| Connector Subscription Message Processing | Last 1,000 tasks per Connector |
Each function task record is correspondingly reduced, with the following restriction policy:
| Limit | |
|---|---|
Length of each print(...) log |
3,000 characters |
| Total log length per function task record | 50,000 characters, retained: First 20,000 characters Last 30,000 characters |
Because the system must retain a certain number of latest task records for each task source, when there are many configured Func APIs, Cron Jobs, and so on, it may occupy a large amount of MySQL storage space.
2. Solution
Disabling local function task records can greatly reduce MySQL storage pressure
You can refer to Deployment and Maintenance / System Metrics and Task Records / Disable Local Function Task Records to disable "Local Function Task Records", reducing MySQL storage pressure.