Skip to content

Script Development / Pre-run Script

Added in version 2.6.18

Important Note

The pre-run script directory was changed from init-scripts to pre-run-scripts starting from version 2.6.22.

At the same time, the new version will automatically copy all files from init-scripts to pre-run-scripts, and users do not need to perform additional actions.

Although DataFlux Func already provides PIP tool for installing third-party Python packages, they may not function properly due to missing dependency libraries.

For example, when using OpenCV in DataFlux Func, apart from installing opencv-python, it is also necessary to install dependency libraries through apt or other means.

Otherwise, the following issues might occur:

dependency-error.png

1. Solving Dependencies with Pre-run Scripts

To resolve this issue, you can provide a pre-run script to DataFlux Func. A pre-run script will execute before DataFlux Func starts, allowing installation of necessary dependencies.

The specific steps are as follows:

1.1 Prepare the Script

The DataFlux Func image is based on Ubuntu:22.04, and the APT source uses Tsinghua University TUNA mirror (http://mirrors.tuna.tsinghua.edu.cn/ubuntu).

Taking the previously mentioned OpenCV dependency issue as an example, prepare the following Bash script.

Bash
1
2
3
4
5
6
7
8
9
# Script only needs to be executed on the Worker side
# Distinguished by the value of $1
if [ $1 != 'worker' ]; then
    exit 0
fi

# Install OpenCV dependencies
apt-get update
apt-get install -y libgl1-mesa-glx libglib2.0-0 libsm6 libxext6 libxrender-dev

Save it as prepare-for-opencv.sh.

The filename can be arbitrary but must end with .sh. To avoid unnecessary issues, avoid using Chinese characters or other unconventional symbols in the filename.

Distinguishing Execution Environments

DataFlux Func consists of Server and Worker sides.

  • The Server side primarily functions as an HTTP server, providing Web pages and HTTP APIs, and does not participate in Python code execution.
  • The Worker side handles actual Python code execution.

Therefore, in most cases, pre-run scripts only need to be executed on the Worker side.

When needing to distinguish the environment, read $1 to determine whether its value is server or worker.

Beat service, MySQL service, Redis service will not execute pre-run scripts

Sample Bash code for distinguishing environments is as follows:

Bash
1
2
3
4
# Execute only on the Worker side
if [ $1 != 'worker' ]; then
    exit 0
fi
Bash
1
2
3
4
# Execute only on the Server side
if [ $1 != 'server' ]; then
    exit 0
fi

1.2 Upload the Script

The location for storing pre-run scripts is as follows:

Environment Location
Inside container /data/resources/pre-run-scripts/
On host machine {Installation Directory}/data/resources/pre-run-scripts/

Users can place pre-run scripts into this directory on the host machine.

Alternatively, within DataFlux Func's File Management, navigate inside and upload your custom-prepared pre-run script:

enable-file-manager.png

upload-script.png

script-ready.png

1.3 Restart DataFlux Func and Verify

After completing all preparations, restart DataFlux Func

Return to the previous script and execute it again; you should now see that the opencv-python library can be imported correctly:

import-ok.png

2. Details of Pre-run Script Execution

Each time DataFlux Func starts, it first checks if there are any pre-run scripts present.

When pre-run scripts exist, DataFlux Func executes them sequentially according to their filenames sorted in alphabetical order. DataFlux Func will only start normally after all scripts have executed successfully.

If you want to observe the execution process of the pre-run scripts, you can track it using the following command:

Bash
1
docker logs {DataFlux Func Container ID} -f

Taking the aforementioned pre-run script as an example, you would see output similar to the following:

Text Only
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
[PRERUN SCRIPT] prepare-for-opencv.sh
Hit:1 http://ports.ubuntu.com/ubuntu-ports jammy InRelease
Get:2 http://ports.ubuntu.com/ubuntu-ports jammy-updates InRelease [128 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports jammy-backports InRelease [127 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports jammy-security InRelease [129 kB]
Get:5 http://ports.ubuntu.com/ubuntu-ports jammy-updates/universe arm64 Packages [1528 kB]
Get:6 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 Packages [2960 kB]
Get:7 http://ports.ubuntu.com/ubuntu-ports jammy-updates/restricted arm64 Packages [3920 kB]
Get:8 http://ports.ubuntu.com/ubuntu-ports jammy-backports/main arm64 Packages [82.8 kB]
Get:9 http://ports.ubuntu.com/ubuntu-ports jammy-backports/universe arm64 Packages [33.3 kB]
Get:10 http://ports.ubuntu.com/ubuntu-ports jammy-security/universe arm64 Packages [1230 kB]
Get:11 http://ports.ubuntu.com/ubuntu-ports jammy-security/main arm64 Packages [2656 kB]
Get:12 http://ports.ubuntu.com/ubuntu-ports jammy-security/restricted arm64 Packages [3777 kB]
Fetched 16.6 MB in 18s (923 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
libxext6 is already the newest version (2:1.3.4-1build1).
libxext6 set to manually installed.
The following additional packages will be installed:
  libdrm-amdgpu1 libdrm-common libdrm-nouveau2 libdrm-radeon1 libdrm2 libgl1
  libgl1-amber-dri libgl1-mesa-dri libglapi-mesa libglib2.0-data libglvnd0
  libglx-mesa0 libglx0 libice6 libllvm15 libpthread-stubs0-dev libx11-dev
  libx11-xcb1 libxau-dev libxcb-dri2-0 libxcb-dri3-0 libxcb-glx0
  libxcb-present0 libxcb-randr0 libxcb-shm0 libxcb-sync1 libxcb-xfixes0
  libxcb1-dev libxdmcp-dev libxfixes3 libxrender1 libxshmfence1 libxxf86vm1
  shared-mime-info x11-common x11proto-dev xdg-user-dirs xorg-sgml-doctools
  xtrans-dev
Suggested packages:
  libx11-doc libxcb-doc
The following NEW packages will be installed:
  libdrm-amdgpu1 libdrm-common libdrm-nouveau2 libdrm-radeon1 libdrm2 libgl1
  libgl1-amber-dri libgl1-mesa-dri libgl1-mesa-glx libglapi-mesa libglib2.0-0
  libglib2.0-data libglvnd0 libglx-mesa0 libglx0 libice6 libllvm15
  libpthread-stubs0-dev libsm6 libx11-dev libx11-xcb1 libxau-dev libxcb-dri2-0
  libxcb-dri3-0 libxcb-glx0 libxcb-present0 libxcb-randr0 libxcb-shm0
  libxcb-sync1 libxcb-xfixes0 libxcb1-dev libxdmcp-dev libxfixes3
  libxrender-dev libxrender1 libxshmfence1 libxxf86vm1 shared-mime-info
  x11-common x11proto-dev xdg-user-dirs xorg-sgml-doctools xtrans-dev
0 upgraded, 43 newly installed, 0 to remove and 27 not upgraded.
Need to get 38.4 MB of archives.
After this operation, 163 MB of additional disk space will be used.
Get:1 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 libglib2.0-0 arm64 2.72.4-0ubuntu2.5 [1435 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 libglib2.0-data all 2.72.4-0ubuntu2.5 [4656 B]
Get:3 http://ports.ubuntu.com/ubuntu-ports jammy/main arm64 shared-mime-info arm64 2.1-2 [453 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports jammy/main arm64 xdg-user-dirs arm64 0.17-2ubuntu4 [53.5 kB]
Get:5 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 libdrm-common all 2.4.113-2~ubuntu0.22.04.1 [5450 B]

Omitted below

You can see that the pre-run script executed correctly and installed the required dependency packages.