You can replicate data from other databases to Apache Kafka with the Data Replication service.
Restriction
Apache Kafka can only be used as a target data store for Data Replication.
Before you begin
Ask your Kafka administrator for Kafka credentials that Data Replication can use to write to the cluster. Ask for Kafka producer properties for the cluster.
You can replicate tables without a primary key or unique constraint to an Apache Kafka database. To connect to Apache Kafka in a project in Cloud Pak for Data as a Service, see Apache Kafka connection.
Configuring a replication asset to use Apache Kafka as a target data store
To create a Data Replication asset with Apache Kafka as a target data store:
-
Click the Assets tab in the project.
-
Click New asset > Replicate data.
-
Enter a name.
-
Click Connections.
-
On the Source options page, select a connection from the list of connections or click Add connection to create a new connection.
-
Click Select data, select a schema, and optionally a table from the schema.
-
On the Target options page, select the Kafka connection from the list.
-
On the Review page, review the summary, then click Create.
When you configure Data Replication to replicate a source schema, tables you create on the source database replicate to the default topic when you run the Data Replication asset. When you replicate a source schema and you change a source table's structure, Data Replication updates the target topic headers accordingly. When you start a replication, the topics that are configured in the replication are truncated. When you pause and resume a replication, the topics are not truncated.
Parent topic: Supported Data Replication connections