Confluent Platform

Synopsis

This section shows how to launch the Business Events Source Connector on a Confluent Platform running locally within a docker environment.

Preliminary Setup

  1. Make sure to have Docker Engine and Docker Compose installed on the machine where you want to run the Confluent Platform.
    1. Docker Desktop available for Mac and Windows includes both.
  2. Download and launch a ready-to-go Confluent Platform Docker image as described in Confluent Platform Quickstart Guide.
  3. Ensure that the machine where the Confluent Platform is running on has a network connection to the OData services published by your SAP system.

Connector Installation

The Business Events Source Connector can be installed either manually or through the Confluent Hub Client.

Note

In both scenarios, it is beneficial to use a volume to easily transfer the connector file into the Kafka Connect service container. If running Docker on a Windows machine, make sure to add a new system variable COMPOSE_CONVERT_WINDOWS_PATHS and set it to 1.

Manual Installation

  1. Unzip the zipped connector package init-kafka-connect-odatabusevent-x.x.x.zip.
  2. Move the unzipped connector folder into the configured CONNECT_PLUGIN_PATH of the Kafka Connect service.
  3. Within the directory where the docker-compose.yml of the Confluent Platform is located, you can start the Confluent Platform using Docker Compose. 
    docker-compose up -d

Confluent CLI

Install the zipped connector package init-kafka-connect-odatabusevent-x.xx-x.x.x.zip using the Confluent Hub Client from outside the Kafka Connect docker container.

confluent connect plugin install {PATH_TO_ZIPPED_CONNECTOR}/init-kafka-connect-odatabusevent-x.x.x.zip
Note

Further information on the Confluent CLI can be found in the Confluent CLI Command Reference.

Connector Configuration

The Business Events Source Connector can be configured and launched using the control-center service of the Confluent Platform.

  1. In the Confluent Control Center (default: localhost:9091) select a Connect Cluster in the Connect tab.
    Choose a Connect Cluster.
    Choose a Connect Cluster.
  2. Click the button “Add connector” and select ODataBusinessEventsSourceConnector.
    Add a connector.
    Add a connector.
  3. Enter a name and further required configuration for the connector.
Note

When using a single connector instance to poll data from different business events, an additional OData Business Events Source configuration block will appear once you provided the required information for the previous OData Business Events Source configuration block. The amount of configurable sources in the Confluent Control Center UI for one connector instance is restricted. Furthermore, sources can be configured in the UI without recommendations in the Additional Properties section.

Sales Order Business Events

For the connector to extract data for business events of type Sales Order, you need to follow these steps:

  1. Transfer the properties including a minimal configuration to the Confluent Control Center user interface. Remember to add in your license key.

    name = busevent-source-connector
connector.class = org.init.ohja.kafka.connect.odata.busevent.source.ODataBusinessEventsSourceConnector
tasks.max = 1
sap.busevent.license.key = "Your license key here"
sap.odata.host.address = services.odata.org
sap.odata.host.port = 443
sap.odata.host.protocol = https
sap.odata.user.name = anonymous
sap.odata.user.pwd = anonymous
sap.busevent.subscriber.code = KBES
sap.busevent#00.object.type = SalesOrder
sap.busevent#00.object.task-code = Deleted,Created,Changed
sap.busevent#00.topic = Order_Details
sap.busevent#00.event.priority = 1
sap.busevent#00.object.resolve = 0
sap.busevent#00.service = /sap/opu/odata/sap/API_SALES_ORDER_SRV
sap.busevent#00.entityset = A_SalesOrder
  2. Launch the source connector.

Starting a Connector via REST Call

  1. Save the example configuration JSON file into a local directory, e.g. named as source.odatabusevent.json. Remember to add in your license key.

    {
    "name": "busevent-source-connector",
    "config": {
        "connector.class": "org.init.ohja.kafka.connect.odata.busevent.source.ODataBusinessEventsSourceConnector",
        "tasks.max": "1",
        "sap.busevent.license.key": "Your license key here",
        "sap.odata.host.address": "services.odata.org",
        "sap.odata.host.port": "443",
        "sap.odata.host.protocol": "https",
        "sap.odata.user.name": "anonymous",
        "sap.odata.user.pwd": "anonymous",
        "sap.busevent.subscriber.code": "KBES",
        "sap.busevent#00.object.type": "SalesOrder",
        "sap.busevent#00.object.task-code": "Deleted,Created,Changed",
        "sap.busevent#00.topic": "Order_Details",
        "sap.busevent#00.event.priority": "1",
        "sap.busevent#00.object.resolve": "0",
        "sap.busevent#00.service": "/sap/opu/odata/sap/API_SALES_ORDER_SRV",
        "sap.busevent#00.entityset": "A_SalesOrder"
    }
}

  2. Once the configuration JSON is prepared, you can start the connector by sending it via a REST call to the Kafka Connect REST API. Use the following command to send a POST request:

    curl -X POST http://localhost:8083/connectors \
-H "Content-Type:application/json" \
-H "Accept:application/json" \
-d @source.odatabusevent.json

  3. Once the connector is started successfully, the Kafka Connect REST API will return a response in JSON format with details about the connector, including its status and any potential errors. You can verify that the connector is running by checking its status:

    curl -X GET http://localhost:8083/connectors/busevent-source-connector/status

    This will return a JSON object indicating whether the connector is running, its tasks, and any associated errors.

1.3.6