Troubleshooting a streams flow

When you have a problem with a streams flow, use this guide to find answers to get it up and running again.

   

Table of contents

  1. Notification errors in canvas
  2. Notification errors in Metrics page
  3. Other streams flow problems
  4. Display problems in the canvas or in the Metrics page
  5. Learn more about operators

   

Notification errors in canvas

Your streams flow is validated as you edit. If errors exist in the streams flow, the Notifications icon (Notifications icon) is displayed in the upper-right corner of the canvas.

What’s happening

When the Notifications icon is displayed, the icon indicates validation errors in the streams flow operators or their links.

Why it’s happening

The streams flow has validation errors that might include the following issues:

  • Missing required fields
  • Missing input or output stream
  • Schema validation: paths, duplicate names, data type mismatch
  • Multiple links on the same port must have the same schema
  • Python code validation
  • Filter expression is not valid

How to fix it

Perform the following steps:

  1. Click the Notifications icon (OK) to display general error messages. For example, the error in a Sample Data operator might be Sample Data - Required parameters are missing.

  2. A red circle (red circle) is shown in the operator that has the error. Hover your mouse pointer over the red dot to see specific errors.

  3. Click the operator that has errors to open its Properties pane. A red circle is shown next to the problematic properties. Correct the problem. If there are no errors, no red circle is shown in the Notifications icon (OK).

 

Here is a screen capture that shows the three areas in yellow highlight where error messages are shown in the canvas:

All errors in an operator

 


 

Notification errors in Metrics page

If errors exist in the streams flow, the Notifications icon (Notifications icon) in the upper-right corner of the Metrics page is displayed with a red dot. This icon can indicate different types of errors: validation, compilation, or runtime.

What’s happening

The Notifications icon in the upper-right corner of the Metrics page is displayed with a red dot indicating that there are errors.

Why it’s happening

  • Streams flow validation error
    • The streams flow is not configured properly
  • Streams flow compilation errors

    • Incorrect Python library was imported in the Code operator
    • Schema path is not valid
    • Data type mismatch
  • Streams flow runtime errors
    • Events are not flowing
    • Data type mismatch
    • Python runtime exceptions
    • The Object Storage container does not exist
    • Connectivity problem to the source or target operator
    • Credentials are not valid or have expired
    • The source or target instance no longer exists
    • Streaming Analytics service instance is stopped

How to fix it

  1. In the Metrics page, click the Notifications icon to open the Notification pane. You can download log files and code files to help you identify your streams flow problem.

    Download logs in Notification icon

    Click any of the following icons:

    • Streaming Analytics instance icon (Streaming Analytics icon) to check the instance in IBM Cloud. In the Manage page of IBM Cloud, you can start or stop the instance.

    • Download user log icon (Download user log icon) to download the user log file. The user log contains logging messages that you put into the Code operator and the Python Machine Learning operator.

    • Download logs icon (Download logs icon) to download system log files. The log files are saved to your local disk in a compressed format. These logs are needed if you contact Support.

    • Download code archive icon (Download code icon) to download the code that generates the streams flow. The code can help you to identify the cause of runtime and compilation errors.

2.In the Notifications pane, click an error to expand it and get details.

3.In the Metrics page, click the Edit icon (Edit icon) to open the canvas.

4.Follow the steps in the following table to correct problems. After you make your corrections, save the streams flow, and then rerun it.

Error type Error message How to fix it
Validation The streams flow is not configured properly 1. In the canvas, click the Notifications icon (Notifications icon) to locate the operator with errors.
2. Click the operator to open its Properties pane, and then fix the errors.
Compilation Incorrect Python library was imported in the Code operator See Python libraries.
In the Code operator, use only Python libraries that are supported.
Compilation Schema path is not valid 1. Click the operator with the invalid schema to open its Properties pane, and then click Edit Output Schema.
2. Correct the path.
Compilation Invalid filter expression 1. Click the Filter operator with the error to open its Properties pane.
2. Check that all expression operators are schema attributes. Use Python syntax.
Learn more about the Filter operator.
Compilation, Runtime Error accessing Streaming Analytics service instance. 1. In the Metrics page, click the Download icon (Download icon) to save the streams flow in STP format to disk. You will import this file in step 6.
2. If another instance does not exist, then in the Watson Studio task bar click Data Services > Services. Click Create new, and then select Streaming Analytics to create an instance and to associate it with this project.
If another Streaming Analytics service instance does exist, go to Project > settings. In the Associated Services section, click New associated service to associate it with this project.
3. Go to the Project page where the streams flow is located.
4. In the Assets tab, go to the Streams Flow section, and then delete the streams flow.
5. Click New streams flow.
6. In the Create Streams Flow page, click the From File tab, and then drag the downloaded streams flow to the Drop area. The name and description of the original streams flow are used.
7. Select the new Streaming Analytics Service.
8. Click Create.
10. In the Metrics page, click the Run icon (Run icon).
Runtime Events are not flowing Check that all source instances are running.
Runtime Data type mismatch 1. Click the operator with the data type mismatch to open its Properties pane, and then click Edit Output Schema.
2. Check that the data type of each attribute is correctly set.
3. Check that the date is in a format that the streams flow supports. See Date formats.
Runtime Python runtime exceptions Click the Code operator with the error to open its Properties pane, and then correct the code. Also, check that all attributes that you are returning as output are listed in the schema.
Learn more about the Code operator.
Runtime The Object Storage container does not exist 1. In the Watson Studio task bar, click Data Services > Object Storage.
2. Select the Region, and then click Add a Container.
Runtime Credentials are not valid or have expired Click the operator whose credentials are not valid or have expired, and then correct the credentials.
Runtime The source or target instance no longer exists 1. In the Watson Studio task bar, click Data Services > Services, and then add the missing instance.
2. Make sure that the operator is configured correctly with the new instance.
Runtime Unable to send email Some email servers block attempts to send email from third-party apps. In this case, log into the email account, open the email, and follow any instructions given by the server.

 


 

Other streams flow problems

What’s happening

A streams flow is stopped.

Why it’s happening

If you restart the Streaming Analytics service, the stream flows are stopped.

How to fix it

Restart the stream flows.

 

What’s happening

Streams flow does not run because the Streaming Analytics service instance is in the wrong state.

Why it’s happening

You exceed the limit of your plan for Streaming Analytics service. The Lite plan is for 50 hours per month. If you have this plan and exceed the limit, you are given email notification.

How to fix it

Either upgrade your plan or wait to run your stream flows.

 

What’s happening

Some problems cannot be fixed by the user, or they might require investigation to determine the cause.

Why it’s happening

The following error messages require further intervention:

  • Unable to start the streams flow due to errors. Please contact IBM Support.
  • The service is down.
  • Connectivity problem to the source or target operator.

How to fix it

If the streams flow cannot start due to error, perform the following steps:

  1. In the Metrics page, perform the following steps:

    a. Click the Download icon (Download icon) to save the streams flow to a file in STP format on your local disk.

    b. Note the URL of the Metrics page.

  2. Click the chat button in the lower-right corner of the Metrics page. Type in the URL of the Metrics page, your contact information, and any pertinent information about the streams flow. Attach the downloaded streams flow file.

 

What’s happening

No data is flowing in the streams flow.

Why it’s happening

Some causes might be runtime errors, incorrect connection information, or the data source instance is missing or stopped.

How to fix it

Perform the following steps:

  1. In the Metrics page of the streams flow, check for any runtime errors. If errors exist, click the Notification icon, and then click the Download logs icon (Download logs icon) and the Download user logs icon (Download user logs icon) to save the logs to your local disk. Check the logs for errors. If the Streaming Analytics service instance is stopped, click the Streaming Analytics icon (Streaming Analytics icon) to check the instance in IBM Cloud.

  2. In the Metrics page, click the Edit icon (Edit icon) to open the canvas. In the canvas, click the problematic operator. In the Properties pane, check that the connection information is correct.

  3. Check the health of the data sources in your IBM Cloud Dashboard:

    • Does an instance of the data source exist? If it was deleted, you must create a new instance.
    • Is the instance stopped? If so, restart it.
    • Can you browse assets in the instance? Click the name of the instance to open the instance. For example, check for topics in an Event Streams instance, or for buckets in a Cloud Object Storage instance.

 

What’s happening

A streams flow can consume large volumes of data, but if an operator cannot handle the large volume, the flow will back up to the operators feeding into it.

Why it’s happening

Congestion occurs when tuples are being delivered to an operator faster than they can be processed and sent to the next operator.

How to fix it

You can take several measures to reduce operator congestion.

 


 

Display problems in the canvas or in the Metrics page

What’s happening

The operators and their links do not show up properly in the canvas or in the Metrics page.

Why it’s happening

Your web browser is not the latest version of Google Chrome, Mozilla Firefox, Microsoft Internet Explorer, or Apple Safari.

How to fix it

Update your web browser with the latest version.

 


 

Learn more