Skip to content

Deployment and Maintenance / High Availability Deployment

DataFlux Func supports multiple deployments to meet high availability requirements.

This article mainly introduces how to install and deploy high availability DataFlux Func directly on servers.

When selecting a high availability solution for Redis, do not use 'Cluster Redis', you can use 'Master-Slave Redis'

If you have previously installed DataFlux Func in a single-node setup, when switching to high availability deployment, please refer to Deployment and Maintenance / Daily Operations / Database Migration for migration

1. Multi-Replica Deployment

Both the Server and Worker services of DataFlux Func support multiple deployments to achieve high availability and scaling requirements.

Generally, the bottleneck in function execution efficiency lies in the Worker service (i.e., Python code), so the Server service only needs to avoid a single point of failure, while the Worker service needs to increase the number of replicas based on actual business volume.

When deploying multiple replicas, ensure that all services have identical user-config.yaml files, connect to the same MySQL and Redis, and mount the same storage for the resource catalog.

Additionally, the Beat service, as the trigger for Cron Jobs, can and should only run one replica, otherwise it may generate duplicate scheduled tasks.

flowchart TB
    USER[User]
    SERVER_1[Server 1]
    SERVER_2[Server 2]
    WORKER_1[Worker 1]
    WORKER_2[Worker 2]
    WORKER_3[Worker 3]
    BEAT[Beat]
    REDIS[Redis]

    USER --HTTP Request--> SLB
    SLB --HTTP Forward--> SERVER_1
    SLB --HTTP Forward--> SERVER_2
    SERVER_1 --Function Execution Task Enqueue--> REDIS
    SERVER_2 --> REDIS
    REDIS --Function Execution Task Dequeue--> WORKER_1
    REDIS --Function Execution Task Dequeue--> WORKER_2
    REDIS --Function Execution Task Dequeue--> WORKER_3

    BEAT --"Function Execution Task Enqueue\n(Scheduled)"--> REDIS

2. Fully Independent Primary-Backup Deployment

Let's not consider for now whether this deployment method counts as 'high availability', assuming there is indeed such a deployment requirement

A fully independent primary-backup deployment essentially means deploying two completely independent sets of DataFlux Func (with identical configurations in the user-config.yaml file regarding Secret, MySQL, and Redis).

Since the primary and backup DataFlux Func run independently, the Beat services on both the primary and backup servers will trigger scheduled tasks in their respective environments, which will result in duplicate scheduled tasks.

To avoid this issue, you can shut down the backup node's DataFlux Func during normal operations, or write scripts to handle the avoidance of duplicate task execution.

flowchart TB
    USER[User]
    MAIN_NODE_SERVER[Primary Node Server]
    MAIN_NODE_WORKER[Primary Node Worker]
    MAIN_NODE_BEAT[Primary Node Beat]
    MAIN_NODE_REDIS_QUEUE[Primary Node Redis Queue]
    BACKUP_NODE_SERVER[Backup Node Server]
    BACKUP_NODE_WORKER[Backup Node Worker]
    BACKUP_NODE_BEAT[Backup Node Beat]
    BACKUP_NODE_REDIS_QUEUE[Backup Node Redis Queue]

    USER --HTTP Request--> SLB
    SLB --HTTP Forward--> MAIN_NODE_SERVER
    SLB -.-> BACKUP_NODE_SERVER

    subgraph "Backup Node - Shutdown"
        direction TB
        BACKUP_NODE_SERVER --Function Execution Task Enqueue--> BACKUP_NODE_REDIS_QUEUE
        BACKUP_NODE_REDIS_QUEUE --Function Execution Task Dequeue--> BACKUP_NODE_WORKER

        BACKUP_NODE_BEAT --"Function Execution Task Enqueue\n(Scheduled)"--> BACKUP_NODE_REDIS_QUEUE
    end

    subgraph Primary Node
        direction TB
        MAIN_NODE_SERVER --Function Execution Task Enqueue--> MAIN_NODE_REDIS_QUEUE
        MAIN_NODE_REDIS_QUEUE --Function Execution Task Dequeue--> MAIN_NODE_WORKER

        MAIN_NODE_BEAT --"Function Execution Task Enqueue\n(Scheduled)"--> MAIN_NODE_REDIS_QUEUE
    end