HTTP operator

The HTTP operator is of type Sources. The operator has no input parameter, but it does have a return value.

This operator calls the following REST services:

  • GET – retrieves web pages, JavaScript, CSS, images, and web forms from any website. The website can be secured (https) or not secured (http).
  • POST – sends data, usually web forms, to any website, secured or not secured.

You can use HTTP as a Source operator if you want to generate some test data without having to set up an IBM Event Streams instance. You might also use HTTP if you want to bring in data from a web socket or from your proprietary data source.

 


Components

Headers

Headers are the core part of HTTP requests and responses. They carry information about the client browser, the requested page, the server, and so forth.

You do not need to supply any headers. Streams flow supplies the following default headers:

Host: my_host.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) 
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Set-Cookie: PHPSESSID=r2t5uvjq435r4q7ib3vtdjq120
Transfer-Encoding: chunked
Date: Sat, 28 Nov 2017 04:36:25 GMT
Server: MyServer
X-Powered-By: W3 Total Cache/0.8
Expires: Sat, 28 Nov 2017 05:36:25 GMT
Cache-Control: max-age=3600, public
Content-Type: text/html; application/json; charset=UTF-8
Content-Length: 1000000
Last-Modified: Sat, 28 Nov 2017 03:50:37 GMT
Location: http://my_server.com
Content-Encoding: gzip

Headers must be written in JSON format. The following example shows a header in JSON format:

{
“Content-Type”: ”application/json”,
“User-Agent”: ”Streams Designer Http/1.0”
}

 

Body

HTTP Body is the data that is transmitted immediately after the HTTP header. It carries POST requests from the client to the server, and responses from the server to the client.

For example, suppose that the HTTP returns a JSON object with the current date and time in human-readable form, and the current number of milliseconds since the UNIX epoch. The HTTP Body contains the time, milliseconds, and date.

HTTP Body must be written in JSON format. The following example shows a Body in JSON format:

{
   "time": "03:53:25 AM",
   "milliseconds_since_epoch": 1362196405309,
   "date": "03-02-2013"
}

 

Response body parsing

Parsing divides the HTTP Body into its individual fields. Parsing is useful if you need to use individual fields in your streams flow.

When you select None, the attribute http_body comes through as a single attribute. Its type is either Text or Binary, detected from its content when editing the output schema. In the Properties pane of the HTTP operator, you can see the http_body attribute in the Output Schema:

Output schema of unparsed HTTP Body

When you select JSON, the parsed attribute http_body comes through as individual attributes in JSON format.

{
   name: “John Doe”,
   age: “50",
   city: “Rome”,
   country: “Italy”
}

In the Properties pane of the HTTP operator, you can see individual attributes rather than the http_body attribute in the Output Schema:

Output schema of unparsed HTTP body

 

Polling interval

HTTP uses a client-driven protocol that initiates communication with the server by sending a request to a server. The client decides the timing of how frequently to check for updates from the server and of how frequently to retrieve data from the server. This timing is the polling interval.