Security for remote runtime engines with DataStage as a Service Anywhere

With remote runtime engines for DataStage®as a Service Anywhere, you retain the full privacy of your sensitive data.

Security for remote runtime engines overview

When you run a DataStage flow, the flow compiles as the Orchestrate Shell code (OSH). At runtime, OSH code is converted into JSON data and is pushed to the remote engine where the job execution occurs. Since the remote engine sits within your VPC, data pipelines can be ran behind the secure firewall, without exposing any data to the Cloud Pak for Data as a Service. You retain full data privacy and remain in compliance with applicable sovereignty regulations. Once a DataStage project is tied to a remote engine runtime execution, you cannot revert it back to Cloud Pak for Data as a Service as this could introduce security vulnerabilities.

Communication

Since the Control Plane (DataStage application on Cloud Pak for Data as a Service) and the remote engine are separated, API calls are necessary for communication. Communication always goes from the remote engine to the Control Plane on Cloud Pak for Data as a Service including API calls and the basic communication protocol (TLS1.2). Each remote engine instance has its own encryption key (AES 256 encryption algorithm) automatically generated for data connections and credentials. Logs are transmitted back to Cloud Pak for Datato track entitlement (VPC) usage but are also available where the remote engine is located.

Sensitive data

In DataStage environments, data can be written to the logs and exposed to Cloud Pak for Data. Therefore, users with potentially sensitive data are advised not to use the Peek Stage or the Asset Browser functions. Instead, users can use sequential files to analyze the actual data and data types that are used in the job design and to parametrize all file names and connections.