0 / 0
Setting up before-job and after-job subroutines in DataStage

Setting up before-job and after-job subroutines in DataStage

You use before-job and after-job subroutines to run built-in subroutines.

Before-job and after-job subroutines include running a script before the job runs or generating a report after the job successfully completes. A return code of 0 from the subroutine indicates success. Any other code indicates failure and causes an unrecoverable error when the job is run.

To set up a before-job or after-job subroutine, complete the following steps.
  1. Open a DataStage® flow, then click the Settings Settings icon icon.
  2. On the Settings page, click Before/after-job subroutines.
  3. Specify a before-job subroutine, an after-job subroutine, or both. Then, click Save.

Using custom Python code in subroutines

You can install Python packages to run scripts in before-job and after-job subroutines.
  1. Open a DataStage flow, navigate to Settings Settings icon, and click Before/after-job subroutines.
  2. Under Built-in subroutine to execute, choose Execute shell command.
  3. In the Shell command text box, enter a command to create a directory for your modules under /px-storage and a command to install the desired modules. This example command installs modules in the directory pip_modules.
    mkdir -p /px-storage/pip_modules && pip3 install <modulename1> <modulename2> --target /px-storage/pip_modules –user
  4. Save and execute the flow.
  5. To enable non-root users to execute your script, append the file path of the module directory at the top of your Python script. Following the previous example:
    import sys
    sys.path.append("/px-storage/pip_modules")
  6. Replace the command in the Shell command text box with a command to call the Python script with its filepath. This example calls the script test_data.py from /ds-storage/ing_py_env.
    python3 /ds-storage/ing_py_env/test_data.py
  7. Save and execute the flow.
Generative AI search and answer
These answers are generated by a large language model in watsonx.ai based on content from the product documentation. Learn more