Confluent Cloud

Synopsis

This quickstart guide shows how to set up the OData V2 Connectors on Confluent Cloud.

Preliminary Setup

1. Make sure to have a working Confluent Cloud environment as described in Confluent Cloud Quickstart Guide.
2. Ensure that you have a running cluster with AWS or Microsoft Azure as the provider.
3. Confirm that Confluent Cloud has a network connection to the publicly available (read/write) OData V2 service.

Connector Installation

1. Go to the menu item Connectors via the left navigation bar and click on the button “Add plugin”.

   

2. Give the connector a name and a description. Specify the path to the connector class and select the type of connector.

   

3. Upload the zipped connector package init-kafka-connect-odatav2-x.x.x.zip via the Connector archive.
4. Click “Submit”. Now the connector should be displayed as a plugin via the Connector menu list.

Connector Configuration

The OData V2 Connectors can be configured via the Connector menu list.

1. Select the connector and generate the API key.

   

2. A key and a secret will be generated, write it down and copy it, as it will not be displayed later. Then select “Continue”.
3. In this overview, the properties can be inserted via the JSON submenu. Nested layers are not recognized, only the pure config is inserted.

   

OData V2 Source Connector

For the OData V2 Source Connector to extract data from the OData V2 service, you need to follow these steps:

1. Open OData V2 service in a browser to obtain a temporary service instance URL that allows read and write operations. The URL will include a temporary service ID that replaces the S(readwrite) part of the URL.

2. Transfer the properties including a minimal configuration to the JSON submenu. Remember to include the temporary service ID as well as your license key.

   name = odata-source-connector
   connector.class = org.init.ohja.kafka.connect.odatav2.source.OData2SourceConnector
   tasks.max = 1
   sap.odata.license.key = "Your license key here"
   sap.odata.host.address = services.odata.org
   sap.odata.host.port = 443
   sap.odata.host.protocol = https
   sap.odata#00.service = /{ENTER SERVICE ID HERE}/V2/OData/OData.svc/
   sap.odata#00.entityset = Suppliers
   sap.odata#00.topic = Suppliers
   

3. Click “Continue” to select how many tasks should be used and launch the connector.
4. Click “Continue” to get to the OData V2 Connector overview. The entries can be checked here.
5. Click “Continue” to launch the connector. It may take a few minutes for the connector to start. After the connector is launched, the new topics and a new topic for the logs appear under the Topics submenu.

OData V2 Sink Connector

To test the sink connector, you need sample data. You can obtain sample data for all entities from the target OData service instance. You can import this data into Kafka using the OData V2 Source Connector.
Follow these steps to export data to the OData V2 service using the OData V2 Sink Connector:

1. Open OData V2 service in a browser to obtain a temporary service instance URL that allows write operations. The URL will include a temporary service ID that replaces the S(readwrite) part of the URL.

2. Transfer the properties with a minimal configuration to the JSON submenu. Remember to include the temporary service ID in the service path as well as including your license key.

   name = odata-sink-connector
   connector.class = org.init.ohja.kafka.connect.odatav2.sink.OData2SinkConnector
   topics = Suppliers
   tasks.max = 1
   sap.odata.license.key = "Your license key here"
   sap.odata.host.address = services.odata.org
   sap.odata.host.port = 443
   sap.odata.host.protocol = https
   sap.odata.content.type = application/atom+xml
   sap.odata#00.service = /{ENTER SERVICE ID HERE}/V2/OData/OData.svc/
   sap.odata#00.entityset = Suppliers
   sap.odata#00.topic = Suppliers
   

3. Launch the sink connector.