Creating a streams flow manually

You can manually design a streams flow that best meets your data and analytic needs. Creating a streams flow manually gives you the greatest control over its design. This method gives you maximum flexibility to get the most value from your data. Use this method rather than the wizard when you have more complicated streams flow.

You don't need to get involved in creating the streams flow - we do that for you. You need to set up the source, target, analytic, and alert operators.

For example, you can add multiple sources and targets to your streams flow. Based on real-time analytics that you need to perform, you can aggregate, filter, and run your own code. You can even send email notification to interested parties from strategic places in your streams flow! If needed, you can do real-time analysis on the streaming data, and then store selected data in a target for further analysis later.

Watch the following video to see how to create a streams flow manually in the streams flow canvas.

Figure 1. Video iconCreate a streams flow manually in the canvas.
This video will demonstrate how to create a streams flow in IBM Streams Designer.

 

Prerequisites

The source data must be in JSON format. If a time stamp attribute exists, it must be in ISO-8601 format, with any delimiter. Milliseconds are optional, and the time zone must not be present. Valid time stamp examples are "2018-01-08T07:11:36", "2018-01-08 07:11:36.877". For more information about date formats that are supported in a streams flow, see Date formats.

You must have either the Editor role or the Admin role.

The following instances must be associated with the project where the streams flow runs:

  • Cloud Object Storage instance

  • Streaming Analytics service instance

To check that the instances are associated, go to the Projects page of the project, and then click the Settings tab. Settings tab

To create an instance, go to your account in IBM Cloud Dashboard. Click Create resource, and then follow the prompts.

Watch this video to see how to provision the services necessary to create, edit, and run a streams flow.

Figure 2. Video iconProvision the prerequisite services to create, edit, and run a streams flow
This video will demonstrate how to provision the prerequisite IBM Cloud services.

To manually create a streams flow, do the following steps:

  1. In the Projects menu, click View All Projects, and then click the name of the project to add your streams flow to. Alternatively, to create a project for the streams flow, click New project.

  2. In the Project page, click the Assets tab, and then click New streams flow.

  3. In the Blank tab of the Create Streams Flow page, type in a unique name. Click Manual, and then click Create.

  4. Construct your streams flow by using the streams flow editor.

  5. Click the Save icon (Save) to save the streams flow, and then click the Metrics icon (Metrics page) to open the streams flow in the Metrics page.

  6. In the Metrics page, the newly created streams flow is shown. Click the Run icon (Run) to begin streaming.

Learn more