Read about pipelines

The Watson Pipelines editor provides a graphical interface for orchestrating an end-to-end flow of assets from creation through deployment. Assemble and configure a pipeline to create, train, deploy, and update machine learning models and Python scripts. Putting a model into production is a multi-step process. Data must be loaded and processed, models must be trained and tuned before they are deployed and tested. Machine learning models require more observation, evaluation, and updating over time to avoid bias or drift.

Read more about pipelines

Learn about other ways to build models

Watch a video about pipelines

Watch this video to preview the steps in this tutorial. You might notice slight differences in the user interface that is shown in the video. The video is intended to be a companion to the written tutorial.

This video provides a visual method to learn the concepts and tasks in this documentation.

Try a tutorial to create a model with Pipelines

This tutorial guides you through exploring and running an AI pipeline to build and deploy a model. The model predicts if a customer is likely subscribe to a term deposit based on a marketing campaign.

In this tutorial, you will complete these tasks:

• Task 1: Create a deployment space.
• Task 2: Create the sample pipeline.
• Task 3: Explore an existing pipeline.
• Task 4: Run the pipeline.
• Task 5: View the assets, deployed model, and online deployment.

This tutorial takes approximately 30 minutes to complete.

Task 1: Create a deployment space

Deployment spaces help you to organize supporting resources such as input data and environments; deploy models or functions to generate predictions or solutions; and view or edit deployment details. Follow these steps to create a deployment space.

1. From the watsonx navigation menu , choose Deployments. If you have an existing deployment space, you can skip to Task 2.

2. Click New deployment space.

3. Type a name for your deployment space.

4. Select a storage service from the list.

5. Select your provisioned machine learning service from the list.

6. Click Create.

Task 2: Create the sample pipeline

To preview this task, watch the video beginning at 00:08.

You create and run pipelines in a project. Follow these steps to create a pipeline based on a sample in a project:

1. On the watsonx home page, select your sandbox or a different existing project from the drop down list.

   

2. Click Customize my journey, and then select View all tasks.

3. Select Automate model lifecycle.

4. Click Samples.

5. Select Orchestrate an AutoAI experiment, and click Next.

6. Optional: Change the name for the pipeline.

7. Click Create. The sample pipeline gets training data, trains a machine learning model by using the AutoAI tool, and selects the best pipeline to save as a model. The model is deployed to a space.

Task 3: Explore the existing pipeline

To preview this task, watch the video beginning at 00:30.

The sample pipeline includes several nodes to create assets and use those assets to build a model. Follow these steps to view the nodes:

1. Click the Global objects icon to view the pipeline parameters. Expand the deployment_space parameter. This pipeline includes a parameter to specify a deployment space where the best model from the AutoAI experiment is stored and deployed. Click the X to close the window.

2. Double-click the Create data file node to see that it is configured to access the data set for the experiment. Click Cancel to close the properties pane.

3. Double-click the Create AutoAI experiment node. View the experiment name, the scope, which is where the experiment is stored, the prediction type (binary classification, multiclass classification, or regression), the prediction column, and positive class. The rest of the parameters are all optional. Click Cancel to close the properties pane.

4. Double-click the Run AutoAI experiment node. This node runs the AutoAI experiment onboarding-bank-marketing-prediction, trains the pipelines, then saves the best model. The first two parameters are required. The first parameter takes the output from the Create AutoAI experiment node as the input to run the experiment. The second parameter takes the output from the Create data file node as the training data input for the experiment. The rest of the parameters are all optional. Click Cancel to close the properties pane.

5. Double-click the Create Web service node. This node creates a deployment with the name onboarding-bank-marketing-prediction-deployment. The first parameter takes the best model output from the Run AutoAI experiment node as the input to create the deployment with the specified name. The rest of the parameters are all optional. Click Cancel to close the properties pane.

Task 4: Run the pipeline

To preview this task, watch the video beginning at 03:43.

Now that the pipeline is complete, follow these steps to run the pipeline:

1. From the toolbar, click Run pipeline > Trial run.

2. In the Values for pipeline parameters section, select your deployment space:

   1. Click Select Space.

   2. Click Spaces.

   3. Select your deployment space from Task 1.

   4. Click Choose.

3. Provide an API key if this occasion is your first time running a pipeline. Pipeline assets use your personal IBM Cloud API key to run operations securely without disruption.

   • If you have an existing API key, click Use existing API key, paste the API key, and click Save.

   • If you don't have an existing API key, click Generate new API key, provide a name, and click Save. Copy the API key, and then save the API key for future use. When you're done, click Close.

4. Click Run to start running the pipeline.

5. Monitor the pipeline progress.

   1. Scroll through consolidated logs while the pipeline is running. The trial run might take up to 10 minutes to complete.

   2. As each operation completes, select the node for that operation on the canvas.

   3. On the Node Inspector tab, view the details of the operation.

   4. Click the Node output tab to see a summary of the output for each node operation.

Task 5: View the assets, deployed model, and online deployment

To preview this task, watch the video beginning at 04:27.

The pipeline created several assets in the deployment space. Follow these steps to view the assets:

1. From the watsonx navigation menu , choose Deployments.

2. Click the name for your deployment space.

3. On the Assets tab, view All assets.

4. Click the bank-marketing-data.csv data asset. The Create data file node created this asset.

5. Click the model beginning with the name onboarding-bank-marketing-prediction. The Run AutoAI experiment node generated several model candidates, and chose this as the best model.

6. Click the Model details tab, and scroll through the model and training information.

7. Click the Deployments tab, and open the onboarding-bank-marketing-prediction-deployment.

   1. Click the Test tab.

   2. Click the JSON input tab, and replace the sample text with the following JSON text.

      {
        "input_data": [
                {
                        "fields": [
                                "age",
                                "job",
                                "marital",
                                "education",
                                "default",
                                "balance",
                                "housing",
                                "loan",
                                "contact",
                                "day",
                                "month",
                                "duration",
                                "campaign",
                                "pdays",
                                "previous",
                                "poutcome"
                        ],
                        "values": [
                                [
                                        35,
                                        "management",
                                        "married",
                                        "tertiary",
                                        "no",
                                        0,
                                        "yes",
                                        "no",
                                        "cellular",
                                        1,
                                        "jun",
                                        850,
                                        10,
                                        -1,
                                        4,
                                        "unknown"
                                ]
                        ]
                }
        ]
      }
      

   3. Click Predict. The results show that the prediction is to approve the applicant.

Next steps

• Try these other methods to build models:

  • Build and deploy a model with AutoAI
  • Build and deploy a model in a Jupyter notebook
  • Build and deploy a model with SPSS Modeler
  • Build and deploy a Decision Optimization model

Learn more

• Watson Pipelines

Parent topic: Quick start tutorials