Skip to content

Script Development / Code Planning and Orchestration

Script collections, scripts, and functions should be organized according to a certain logic rather than arbitrarily piling code together. Properly organizing script collections and scripts is beneficial for code maintenance and system runtime efficiency.

1. Reasonably divide script collections and scripts by purpose and type

Generally, the following few methods of code organization are recommended:

  • Create separate script collections for business processing scripts based on industry, project, or organization, such as: eShop, IoT, monitor, sales, marketing
  • When there's too much code, divide scripts based on usage frequency into low-frequency and high-frequency scripts, such as prediction, advanced_prediction

2. Call a function from another script

Scripts can be divided into different scripts or collections based on functionality and purpose, and code located in different scripts can call each other. To call a function from another script, simply import the corresponding script.

When importing another script, it must follow a fixed format:

Python
1
2
3
4
5
6
# import <script collection ID>__<script ID>
import demo__script

# Or use an alias to shorten the length
# import <script collection ID>__<script ID> as alias
import demo__script as script

In the script editor, you can hover over the question mark icon in the left panel to directly copy relevant statements

If you need to export a script collection, any other scripts this collection depends on must also be exported. Otherwise, the exported script collection will not run properly due to missing functions!

Scripts or script collections are not Python modules and cannot originally be imported using import

However, DataFlux Func implements a dynamic loading mechanism internally, allowing the use of import statements to load dynamic code. Therefore, the following formats are incorrect.

Python
1
2
3
4
5
6
# Incorrect syntax 1: Importing a script collection as a module
import demo

# Incorrect syntax 2: Importing a script as a module
import demo.script
from demo import script

Additionally, when importing scripts, avoid creating circular references, such as:

Python
1
2
3
4
5
6
7
8
# Script `script2` under the `demo` collection
import demo__script2

# Script `script3` under the `demo` collection
import demo__script3

# Script `script1` under the `demo` collection
import demo__script1

Note that when editing multiple scripts simultaneously, if one script imports another, the referenced script will execute in its published version, and the system will never import draft versions!

3. Code volume and dependency chains in a single script

Since DataFlux Func dynamically loads required scripts during execution, if one script imports another, the imported script will also be dynamically loaded.

Therefore, if certain functions within a script are called very frequently, consider extracting them into a separate script to reduce loading overhead. It is recommended to keep the size of a single script under 1000 lines.

Additionally, try to avoid overly long dependency chains to prevent unnecessary performance loss, such as:

  • Script 1 depends on Script 2
  • Script 2 depends on Script 3
  • Script 3 depends on Script 4
  • Script 4 depends on Script 5
  • ...

Built-in Python modules and third-party modules are not affected by this restriction.

4. Cases where scripts are not divided

Although the above mentions reasonable planning of scripts and methods for calling between scripts, in some specific cases (such as when only a few simple common functions are actually used), you may consider not dividing scripts and placing all the code in a single script.

This approach, while introducing a little redundancy in the code, also brings additional benefits:

  • A single script can run, reducing loading overhead
  • It won't be affected by changes to common functions
  • No need to consider dependencies when exporting scripts

Choose the most appropriate way to plan your scripts based on actual circumstances.

5. Abbreviation for mutual references between scripts in the same collection

Added in version 1.1.0rc51

Within the same script collection, scripts can reference each other without writing the script collection ID part (i.e., just write the part starting with __).

This makes it convenient to clone a script collection and change the script collection ID, while still allowing internal functions to correctly reference each other.

For example:

Python
1
2
3
4
# Script: demo__utils

def echo(msg):
    return msg
Python
1
2
3
4
5
6
7
# Script: demo__test

# Equivalent to import demo__utils as utils
import __utils as utils

def my_func(msg):
    return utils.echo(msg)

6. Avoid Over-Encapsulation

Python is a multi-paradigm programming language that can use both simple procedural programming and object-oriented programming.

However, in DataFlux Func, to make debugging easier, it is recommended to write code using a more procedural style, avoiding excessive encapsulation.

For example, the same functionality can be implemented in two different ways:

Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
import requests

# Procedural
def query_some_api(url, params):
    r = requests.get(url, params)
    return r.json()

# Object-Oriented
class APIQuery(object):
    def __init__(self, url):
        self.url = url

    def do(self, params):
        r = requests.get(self.url, params)
        return r.json()

def test_api_query(url, params):
    api_query = APIQuery(url)
    api_query.do(params)

In the above example, although both can achieve the same functionality, since query_some_api(...) is a function that can be called directly, it allows selecting this function in the editor and running it directly after filling in parameters.

To debug and run the do(...) method of the APIQuery class, you must first instantiate the object before calling it, meaning you would have to write an additional test function test_api_query(...) to invoke it.

Which approach to use should be determined based on the actual situation.