Creating a custom component for use in the pipeline

Creating a component as a project asset

To create a custom component, use the Python client to authenticate with IBM Watson Pipelines, code the component, then publish the component to the specified project. After it is available in the project, you can assign the component to a node in a pipeline and run it as part of a pipeline flow.

This example demonstrates the process of publishing a component that adds two numbers together, then assigns the component to a pipeline node.

1. Publish a function as a component with the Python client. Run the following code in a Jupyter Notebook in a project of Cloud Pak for Data as a Service.

   # Install libraries
   ! pip install ibm-watson-pipelines
   
   # Authentication
   from ibm_watson_pipelines import WatsonPipelines
   
   apikey = ''
   project_id = 'your_project_id'
   
   client = WatsonPipelines.from_apikey(apikey)
   
   # Define the function of the component
   
   # If you define the input parameters, users are required to 
   # input them in the UI
   
   def add_two_numbers(a: int, b: int) -> int:
       print('Adding numbers: {} + {}.'.format(a, b))
       return a + b + 10
   
   # Other possible functions might be sending a Slack message,
   # or listing directories in a storage volume, and so on.
   
   # Publish the component    
   client.publish_component(
       name='Add numbers', # Appears in UI as component name 
       func=add_two_numbers,
       description='Custom component adding numbers', # Appears in UI as component description 
       project_id=project_id,
       overwrite=True, # Overwrites an existing component with the same name 
   )
   

   To generate a new API key:

   1. Go to the IBM Cloud home page
   2. Click Manage > Access (IAM)
   3. Click API keys
   4. Click Create

2. Drag the node called Run Pipelines component under Run to the canvas.
   

3. Choose the name of the component that you want to use.
   

4. Connect and run the node as part of a pipeline job.
   
   

Manage pipeline components

Use these Python client methods to manage custom pipeline components.

Parent topic: Creating a pipeline