Confluent Cloud

Synopsis

This quickstart guide shows how to set up the OData V4 Connectors on Confluent Cloud.

Preliminary Setup

1. Make sure to have a working Confluent Cloud environment as described in Confluent Cloud Quickstart Guide.
2. Ensure that you have a running cluster with AWS or Microsoft Azure as the provider.
3. Confirm that Confluent Cloud has a network connection to the publicly available (read/write) TripPin service.

Connector Installation

1. Go to the menu item Connectors via the left navigation bar and click on the button “Add plugin”.
2. Give the connector a name, a description, specify the path to the connector class and select the type of connector.
3. Upload the zipped connector package init-kafka-connect-odatav4-x.x.x.zip via the Connector archive.
4. Click “Submit”. Now the connector should be displayed as a plugin via the Connector menu list.

Connector Configuration

The OData V4 Connectors can be configured via the Connector menu list.

1. Select the connector and generate the API Key.
2. A key and a secret will be generated, write it down/copy it, as it will not be displayed later. Then select “Continue”.
3. In this overview, the properties can be inserted via the JSON submenu. Nested layers are not recognized, only the pure config is inserted.

OData V4 Source Connector

Extract data from the TripPin service using the OData V4 Source Connector:

1. Open TripPin service in a browser to obtain a temporary service instance URL. The URL will include a temporary service ID that replaces the S(readwrite) part of the URL.

2. Transfer the properties including a minimal configuration to the JSON submenu. Remember to include the service ID in the service path and to include your license key.

   name = odata-source-connector
   connector.class = org.init.ohja.kafka.connect.odatav4.source.OData4SourceConnector
   tasks.max = 1
   sap.odata.license.key = "Your license key here"
   sap.odata.host.address = services.odata.org
   sap.odata.host.port = 443
   sap.odata.host.protocol = https
   sap.odata#00.service = /V4/{TEMPORARY_SERVICE_ID}/TripPinServiceRW
   sap.odata#00.entityset = People
   sap.odata#00.topic = People
   

3. Click “Continue” to select how many tasks should be used and launch the connector.
4. Click “Continue” to get to the OData V4 Connector overview. The entries can be checked here.
5. Click “Continue” to launch the connector. It may take a few minutes for the connector to start. After the connector is launched, the new topics and a new topic for the logs appear under the Topics submenu.

OData V4 Sink Connector

Export data from the OData V4 Sink Connector to the TripPin service:

1. Open TripPin service in a browser to obtain a temporary service instance URL that allows write operations. The URL will include a temporary service ID that replaces the S(readwrite) part of the URL.

2. Transfer the properties including a minimal configuration to the JSON submenu. Remember to include the service ID in the service path and to include your license key.

   name = odata-sink-connector
   connector.class = org.init.ohja.kafka.connect.odatav4.sink.OData4SinkConnector
   topics = People
   sap.odata.license.key = "Your license key here"
   tasks.max = 1
   sap.odata.host.address = services.odata.org
   sap.odata.host.port = 443
   sap.odata.host.protocol = https
   sap.odata#00.service = /V4/{TEMPORARY_SERVICE_ID}/TripPinServiceRW
   sap.odata#00.entityset = People
   sap.odata#00.topic = People
   

3. Launch the sink connector.
4. To test the sink connector, you need sample data. You can obtain sample data for all entities from the target TripPin service instance. You can then import this data into Kafka using the OData V4 Source Connector.