Script Development / Upload User Python Modules

In some cases, the Python modules required by users are not published on the PyPi platform but exist directly as files.

In such scenarios, these files can be uploaded to the "user-python-packages/" directory in the Resource Catalog of DataFlux Func and directly referenced in scripts.

1. Upload Python Modules

Users can choose the upload method based on their actual situation, and there is no difference in the result.

Direct Upload to the Host

If users have access to the host, they can directly upload Python files/directories to the {installation directory}/data/resources/user-python-packages/ directory using their preferred SSH tools.

The default installation directory is /usr/local/dataflux-func. If unsure, use cat /etc/dataflux-func to check.

Upload Using "File Service"

If users do not have access to the host, they can enable "Manage / Experimental Features / Enable File Service" and upload via "Manage / File Service."

To prevent the directory structure from being disrupted after unzipping the uploaded zip file, unzipping in "File Service" will forcibly create a directory.

Therefore, if the uploaded zip file already has a directory layer, an additional directory structure will be created after unzipping.

Move the Python module directory to user-python-packages/xxxxx as needed.

Go to the user-python-packages/ directory and confirm the Python package directory to be uploaded.

2. Use Uploaded Python Modules

Assume the uploaded Python module is as follows:

from my_pkg.hello_world import hello_world

def hello_world():
    return 'Hello, world!!!'

Then, you can directly import and use this module in a script, such as:

import my_pkg

def run():
    data = my_pkg.hello_world()
    print(data)

3. Notes

Ensure that the self-uploaded Python modules do not have the same name as other Python modules, and do not delete or rename directories that originally exist in extra-python-packages.