Streams (Source and Target) operators

The Streams operator publishes and subscribes messages to topics in the Streaming Analytics instance that is associated with your streams flow.

The Streams operator can connect a streams flow in IBM Watson Studio with a flow in IBM Streams Studio if both flows use the same Streaming Analytics instance.

This feature is useful when you want to process data in Streams Studio in a way that cannot be done in a IBM Watson Studio streams flow.

In a streams flow, the Streams operator can be of type Source or of type Target:

  • Streams as a Source operator subscribes to topics in the Streaming Analytics instance.

  • Streams as a Target operator publishes to topics in the Streaming Analytics instance.

Prerequisites

  • A Streaming Analytics instance must be associated with the streams flow.

  • If you intend to subscribe to a flow in Streams Studio, it must use the same Streaming Analytics instance as your streams flow in Watson Studio.

Topic

A topic is a string that can have a hierarchy of levels that are separated by a slash (/). This string is the routing information for the Streaming Analytics instance.

You create a topic in the Streams as Target operator by typing in a name. For example, you might type in Myhome/first_floor. The topic Myhome/first_floor is then created and published in the Streaming Analytics instance.

In the Streams as Source operator, when you select a topic from a list of existing topics, you are subscribing to a published topic. In the previous example, when you select Myhome/first_floor, you subscribe to that topic.

When subscribing to topic, you have to specify the exact schema. If only one schema matches the topic, that one will be used. However, if more schemas match the topic, you have to select the one you want to subscibe to. Only data matching both, topic pattern and schema, will be ingested.

Persistence

Data is not persisted in Streaming Analytics.

If you subscribe, you get new tuples.

Examples

Let’s say that the following topics are defined in the Streaming Analytics instance:

  • Myhome/first_floor/bedroom/temp_detector

  • Myhome/first_floor/kitchen/temp_detector

  • Myhome/first_floor/kitchen/smoke_detector

The following examples of Streams of type Source show how to subscribe to different topics:

Subscription to a specific topic

To subscribe to the topic for kitchen temperature data, the Streams as Source operator would subscribe to Myhome/first_floor/kitchen/temp_detector.

Subscription to a single level +

Use a wildcard to subscribe to all topics in the same topic level.

When Streams as Source operator subscribes to the topic Myhome/first_floor/+/temp_detector, the operator subscribes to the following topics:

  • Myhome/first_floor/kitchen/temp_detector

  • Myhome/first_floor/bedroom/temp_detector

Subscription to multi-level #

If you need to subscribe to an arbitrary number of topic levels, use a multi-level wildcard (#). You can subscribe to all underlying hierarchy levels. The # must be the last character in the topic and it must be preceded by a slash (/).

When Streams as Source operator subscribes to the topic Myhome/first_floor/#, the operator subscribes to all topics that begin with Myhome/first_floor:

  • Myhome/first_floor/bedroom/temp_detector

  • Myhome/first_floor/kitchen/temp_detector

  • Myhome/first_floor/kitchen/smoke_detector