Confluent Platform

Synopsis

This section shows how to launch the ODP Source Connector on a Confluent Platform running locally within a docker environment.

Preliminary Setup

1. Install Docker Engine and Docker Compose on the machine where you plan to run the Confluent Platform.
   1. If you’re using a Mac or Windows, Docker Desktop includes both.
2. Download and launch a ready-to-go Confluent Platform Docker image as described in Confluent Platform Quickstart Guide.
3. Make sure that the machine running the Confluent Platform is connected to the SAP® source.
4. Make sure you have a licensed version of the SAP® Java Connector 3.1 SDK before proceeding.

Connector Installation

The connector can be installed either manually or through the Confluent Hub Client.

Manual Installation

To install the ODP Source Connector, follow these steps:

1. Unzip the downloaded file, init-kafka-connect-odp-x.x.x.zip.
2. Get a copy of SAP® Java Connector v3.1.11 SDK and move it to the lib/ folder inside the unzipped connector folder. SAP® JCo consists of sapjco3.jar, and the native libraries like sapjco3.dll for Windows OS or sapjco3.so for Unix. For instance include the native lib sapjco3.so next to the sapjco3.jar.
3. Move the unzipped connector folder into the configured CONNECT_PLUGIN_PATH of the Kafka Connect service.

4. Within the directory where the docker-compose.yml of the Confluent Platform is located you can start the Confluent Platform using Docker Compose.

   docker-compose up -d
   

Confluent CLI

Install the zipped connector init-kafka-connect-odp-x.x.x.zip using the Confluent Hub Client from outside the Kafka Connect docker container:

confluent connect plugin install {PATH_TO_ZIPPED_CONNECTOR}/init-kafka-connect-odp-x.x.x.zip

Connector Configuration

The connector can be configured and launched using the control-center service of the Confluent Platform.

1. In the control-center (default: localhost:9091) select a Connect Cluster in the Connect tab.

   

2. Click the “Add connector” button and select the ODPSourceConnector.

   

3. Enter a unique name for the connector instance and provide any other required configuration.

For the ODP Source Connector to extract data from a SAP® source, you need to follow these steps:

1. Transfer the properties including a minimal configuration to the Confluent Control Center user interface. Remember to add in your license key.

   name = odp-source-connector
   connector.class = org.init.ohja.kafka.connect.odp.source.ODPSourceConnector
   tasks.max = 1
   sap.odp.license.key = "Your license key here"
   sap.odp.subscriber-name = OhJaODPKafkaConnector
   sap.odp#00.name = Test
   sap.odp#00.context = SAPI
   sap.odp#00.topic = ODPSAPITEST
   jco.client.ashost = 127.0.0.1
   jco.client.sysnr = 20
   jco.client.client = 100
   jco.client.user = user
   jco.client.passwd = password
   

2. Launch the source connector.

Starting a Connector via REST Call

1. Save the example configuration JSON file into a local directory, e.g. named as source.odp.json. Remember to add in your license key.

   {
       "name": "odp-source-connector",
       "config": {
           "connector.class": "org.init.ohja.kafka.connect.odp.source.ODPSourceConnector",
           "tasks.max": "1",
           "sap.odp.license.key": "Your license key here",
           "sap.odp.subscriber-name": "OhJaODPKafkaConnector",
           "sap.odp#00.name": "Test",
           "sap.odp#00.context": "SAPI",
           "sap.odp#00.topic": "ODPSAPITEST",
           "jco.client.ashost": "127.0.0.1",
           "jco.client.sysnr": "20",
           "jco.client.client": "100",
           "jco.client.user": "user",
           "jco.client.passwd": "password"
       }
   }
   

2. Once the configuration JSON is prepared, you can start the connector by sending it via a REST call to the Kafka Connect REST API. Use the following command to send a POST request:

   curl -X POST http://localhost:8083/connectors \
   -H "Content-Type:application/json" \
   -H "Accept:application/json" \
   -d @source.odp.json
   

3. Once the connector is started successfully, the Kafka Connect REST API will return a response in JSON format with details about the connector, including its status and any potential errors. You can verify that the connector is running by checking its status:

   curl -X GET http://localhost:8083/connectors/odp-source-connector/status
   

   This will return a JSON object indicating whether the connector is running, its tasks, and any associated errors.