Quickstart guide

This section contains two types of guides: one for local standalone mode using Apache Kafka and the other for operating a confluent platform inside docker.

Kafka Connect Standalone

Synopsis

This quickstart guide will show you how to set up the OData Business Events Source Connector on a local Kafka Connect instance in standalone mode using Apache Kafka, extracting business event data from a SAP system. The publicly available business event object type used for this scenario is Sales Order. The corresponding BOR object type is BUS2032.

Preliminary setup

1. Download and extract Apache Kafka
2. Copy and extract the Business Events Source Connector package into the Kafka Connect plugins directory

Configuration

1. Edit the contents of file /config/connect-standalone.properties like this:

   bootstrap.servers=localhost:9092
   key.converter=org.apache.kafka.connect.json.JsonConverter
   value.converter=org.apache.kafka.connect.json.JsonConverter
   key.converter.schemas.enable=true
   value.converter.schemas.enable=true
   offset.storage.file.filename=/tmp/connect.offsets
   offset.flush.interval.ms=10000
   plugin.path=/kafka_2.12-2.0.0/plugins,
   

   Note

   Make sure the plugin path exists

2. Extract the Business Events Source Connector property template etc/busevent-source-connector.properties from the connector package and copy it to /config/busevent-source-connector.properties

Execution

1. Maintain subscriber code KBES in your SAP system, as described in section SAP subscriber configuration (in the doc/readme.md)

2. Start a local Zookeeper instance from the shell, e.g. for Windows OS type:

   cd <KAFKA_ROOT>
   set KAFKA_LOG4J_OPTS=-Dlog4j.configuration=file:<KAFKA_ROOT>/config/tools-log4j.properties
   bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties
   

3. Start a local Kafka server instance from another shell, e.g. for Windows OS type:

   bin\windows\kafka-server-start.bat .\config\server.properties
   

4. Start a simple kafka consumer from another shell, e.g. for Windows OS type:

   bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic Order_Details --from-beginning
   

5. Start a local standalone Kafka Connect instance and execute the Business Events Source Connector from another shell, e.g. for Windows OS type:

   bin\windows\connect-standalone.bat .\config\connect-standalone.properties .\config\busevent-source-connector.properties > log.txt 2>&1
   

   The logging outputs will be written to the file log.txt

6. Execute transaction code SWUE in your SAP system and create a business event for object type BUS2032.

Switch to the Kafka consumer shell and if everything goes right you should, see a json representation of the sales order business event messages together with its schema printed to the standard console output.

Logging

Check the log outputs by opening file log.txt in an editor of your choice or for Windows OS just type:

type log.txt

Confluent Platform

Synopsis

This section shows how to launch the Business Events Source Connector on a Confluent Platform running locally within a docker environment.

Preliminary setup

1. Make sure to have Docker Engine and Docker Compose installed on the machine where you want to run the Confluent Platform
   1. Docker Desktop available for Mac and Windows includes both
2. Download and launch a ready-to-go Confluent Platform Docker image as described in Confluent Platform Quickstart Guide
3. Ensure that the machine where the Confluent Platform is running on has a network connection to the OData services published by your SAP system.

Connector Package Installation

The Business Events Source Connector can either be installed manually or through the Confluent Hub Client.

Manual Installation

1. Unzip the zipped connector package init-kafka-connect-odatabusevent-x.xx-x.x.x.zip
2. Move the unzipped connector folder into the configured CONNECT_PLUGIN_PATH of the Kafka Connect service
3. Within the directory where the docker-compose.yml of the Confluent Platform is located, you can start the Confluent Platform using Docker Compose

   docker-compose up -d
   

Confluent CLI

Install the zipped connector package init-kafka-connect-odatabusevent-x.xx-x.x.x.zip using the Confluent Hub Client from outside the Kafka Connect docker container

docker-compose exec connect confluent-hub install {PATH_TO_ZIPPED_CONNECTOR}/init-kafka-connect-odatabusevent-x.xx-x.x.x.zip

Further information on the Confluent CLI can be found in the Confluent CLI Command Reference

Configuration

The Business Events Source Connector can be configured and launched using the control-center service of the Confluent Platform.

1. In the Confluent Control Center (default: localhost:9091) select a Connect Cluster in the Connect tab
2. Click the button “Add connector” and select ODataBusinessEventsSourceConnector
3. Enter a name and further required configuration for the connector

Hints

When you use a single connector instance to poll data from different business events, an additional OData Business Events Source configuration block will appear once you provided the required information for the previous OData Business Events Source configuration block. The amount of configurable sources in the Confluent Control Center UI for one connector instance is restricted. Further, sources can be configured in the UI without recommendations in the Additional Properties section.

Sales Order Business Events

For the connector to extract data for business events of type Sales Order, you can transfer the properties from the Business Events Source Connector property template etc/odatabusevent-source-connector.properties in the connector package to your Kafka Connect environment and launch it.