Setting up before-job and after-job subroutines in DataStage
You use before-job and after-job subroutines to run built-in subroutines.
Before-job and after-job subroutines include running a script before the job runs or generating a report after the job successfully completes. A return code of 0 from the subroutine indicates success. Any other code indicates failure and causes an unrecoverable error when the job is run.
To set up a before-job or after-job subroutine, complete the following steps.
- Open a DataStage® flow, then click the Settings icon.
- On the Settings page, click Before/after-job subroutines.
- Specify a before-job subroutine, an after-job subroutine, or both. Then, click Save.
Using custom Python code in subroutines
You can install Python packages to run scripts in before-job and after-job subroutines.
- Open a DataStage flow, navigate to Settings , and click Before/after-job subroutines.
- Under Built-in subroutine to execute, choose Execute shell command.
- In the Shell command text box, enter a command to create a directory for
your modules under /px-storage and a command to install the desired modules.
This example command installs modules in the directory
pip_modules.
mkdir -p /px-storage/pip_modules && pip3 install <modulename1> <modulename2> --target /px-storage/pip_modules –user
- Save and execute the flow.
- To enable non-root users to execute your script, append the file path of the module directory at
the top of your Python script. Following the previous example:
import sys sys.path.append("/px-storage/pip_modules")
- Replace the command in the Shell command text box with a command to call
the Python script with its filepath. This example calls the script
test_data.py
from /ds-storage/ing_py_env.python3 /ds-storage/ing_py_env/test_data.py
- Save and execute the flow.