JSON path

In operators whose data schema is in JSON format, you must edit the operator’s schema.

The Edit Schema window opens when you click Edit Output Schema (edit_schema) in the Properties pane of those operators. You can reorder, add, and remove attributes from the data schema as it flows between operators.

In the Edit Schema window, JSON Path is a path according to the JSON Pointer specification. It defines a string syntax for identifying a specific value within a JSON.

JSON path in Edit Schema window

The following code snippet is an example JSON document.

	{  
		"title": "Some Book",
		"publisher": {
    		"name": "Wiki"
  		},  
  		"authors": [
    		{ "names": ["Smith", "John"] },
    		{ "names": ["Doe", "Jane"] }
  		 ]
	} 

The JSON Path defines a string syntax that is prefixed by ‘/’. The path is used to identify a specific value within a JSON document.

Using the example JSON document, the following code snippet shows what the JSON Path is to extract values.

/title                  "Some Book"
/publisher/name         "Wiki"
/authors/0/names/0      "Smith"
/authors/0/names/1      "John"
/authors/1/names/1      "Jane"
/authors/1/names/42     ""  (this is an empty value, not an error) 

Preset attributes

Some source operators have preset metadata attributes in their output, in addition to a preset data attribute. The schema editor presents them along with any JSON-based attributes, but their path expressions are identifying preset attributes rather than values within JSON. You can change the Attribute Name of preset metadata attributes, but not the Path or the Type.

Depending on the selected parse format, the data attribute is either automatically parsed at ingestion, or it is streamed as raw data. The following table describes the impact of parse format selection on schema detection:

Parse
setting
Detected (default) output schema attributes
Preset metadata attributes
Preset data attribute Extracted JSON attributes
JSON yes no yes
None yes yes no

For an illustrative example with the HTTP source operator, see HTTP response body parsing.

The following table depicts the parse selection property and the preset attributes of various source operators:

Source operator Parse selection property Preset data attribute Preset metadata attributes
Kafka, Event Streams Record value parsing event_message event_key
event_topic
event_offset
event_partition
event_timestamp
HTTP Response body parsing http_body http_status_code
http_reason
http_headers

Learn more