Quickstart Guide

This section contains two types of guides: one for local standalone mode using Apache Kafka and the other for operating a Confluent Platform inside docker.

Note

For using all the functionalities of the connector, we recommend going for the Confluent Platform.

Kafka Connect standalone

Synopsis

This quickstart will show how to set up the RFC Source Connector against an existing SAP Netweaver system and run it locally in Kafka Connect standalone mode using Apache Kafka. We will use a non encrypted communication channel with basic SAP username/password authentication. In productive environments, it is recommended to use a SAP® Secure Network Communication (SNC) setup with optional Single Sign On (SSO).

Preliminary setup

Download and extract Apache Kafka
Copy the RFC source connector jar into the Kafka Connect plugins directory
Get a copy of SAP JCo v3.1.8 (sapjco3.jar and native lib sapjco3.dll or sapjco3.so) and copy it to the plugins directory

Configuration

Edit the contents of file /config/connect-standalone.properties like this:

bootstrap.servers=localhost:9092
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=true
value.converter.schemas.enable=true
offset.storage.file.filename=/tmp/connect.offsets
offset.flush.interval.ms=10000
plugin.path=/kafka_2.12-2.0.0/plugins,

Note

Make sure that the plugin path exists

Extract the RFC source connector properties template etc/ohja-rfc-source-connector.properties from the connector package and copy it to /config/ohja-rfc-source-connector.properties
Get in contact with your administration team for the connection properties of your SAP Netweaver installation and maintain the following minimum connector properties:
```
# SAP Netweaver application host DNS or IP
jco.client.ashost = 127.0.0.1
# SAP system number
jco.client.sysnr = 20
# SAP client number
jco.client.client = 100
# SAP RFC user
jco.client.user = user
# SAP user password
jco.client.passwd = password
```
Note

Make sure the SAP user has enough privileges to access RFC-enabled function modules and the SAP Netweaver Gateway is configured to accept RFC connections from your host.

Maintain the output Kafka topic name, and the following connector configs according to the RFM of your choice in SAP:

sap.rfc#00.name = ZFUNCTIONNAME
sap.rfc#00.group = ZFUNCTIONGROUP
sap.rfc#00.offset-parameter.in = IS_OFFSET
sap.rfc#00.offset-parameter.out = ES_OFFSET
# Kafka output topic name
sap.rfc#00.topic = ZFUNCTIONNAME

Execution

Start a local Zookeeper instance from the shell, e.g. for Windows OS type:

cd <KAFKA_ROOT>
set KAFKA_LOG4J_OPTS=-Dlog4j.configuration=file:<KAFKA_ROOT>/config/tools-log4j.properties
bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

Start a local Kafka server instance from another shell, e.g. for Windows OS type:
```
bin\windows\kafka-server-start.bat .\config\server.properties
```

Start a simple kafka consumer from another shell, e.g. for Windows OS type:

bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic ZFUNCTIONNAME --from-beginning

Start a local standalone Kafka Connect instance and execute the RFC source connector from another shell, e.g. for Windows OS type:
```
bin\windows\connect-standalone.bat .\config\connect-standalone.properties .\config\ohja-rfc-source-connector.properties > log.txt 2>&1
```
The logging outputs will be written to the file log.txt

Switch to the Kafka consumer shell and if everything goes right you should see json representations of the RFC messages together with their schema being printed out to the standard console output.

Logging

Check the log outputs by opening file log.txt in an editor of your choice or for Windows OS just type:

type log.txt

Confluent Platform

Synopsis

This section shows how to launch the RFC Source Connector on a Confluent Platform running locally within a docker environment.

Preliminary setup

Make sure to have Docker Engine and Docker Compose installed on the machine where you want to run the Confluent Platform
1. Docker Desktop available for Mac and Windows includes both
Download and launch a ready-to-go Confluent Platform Docker image as described in Confluent Platform Quickstart Guide
Ensure that the machine where the Confluent Platform is running on has a connection to the SAP source.
Ensure that you have a licensed version of the SAP Java Connector 3.1 SDK available

Connector Installation

The RFC source connector can either be installed manually or through the Confluent Hub Client.

Note

In both scenarios, it is beneficial to use a volume to easily transfer the connector file into the Kafka Connect service container. If running Docker on a Windows machine make sure to add a new system variable COMPOSE_CONVERT_WINDOWS_PATHS and set it to 1.

Manual Installation

Unzip the zipped connector init-kafka-connect-rfc-x.x.x.zip
Get a copy of SAP Java Connector v3.1.8 SDK and move it to the lib folder inside the unzipped connector folder. SAP JCo consists of sapjco3.jar and the native libraries like sapjco3.dll for Windows OS or sapjco3.so for Unix. Include e.g. the native lib sapjco3.so next to the sapjco3.jar
Move the unzipped connector folder into the configured CONNECT_PLUGIN_PATH of the Kafka Connect service
Within the directory where the docker-compose.yml of the Confluent Platform is located, you can start the Confluent Platform using Docker Compose
```
docker-compose up -d
```

Confluent Hub Client

Install the zipped connector init-kafka-connect-rfc-x.xx-x.x.x.zip using the Confluent Hub Client from outside the Kafka Connect docker container

docker-compose exec connect confluent-hub install {PATH_TO_ZIPPED_CONNECTOR}/init-kafka-connect-rfc-x.x.x.zip

Further information on the Confluent CLI can be found in the Confluent CLI Command Reference

Configuration

The RFC Source Connector can be configured and launched using the control-center service of the Confluent Platform.

In the control-center (default: localhost:9091) select a Connect Cluster in the Connect tab
Click the button “Add connector” and select RFCSourceConnector
Enter a name and further required configuration for the connector

Hints

To prevent from SAP® user from being locked out during the connector configuration process, it’s important to be cautious when entering the jco.client.user and jco.client.passwd properties in the JCo Destination configuration block with caution since the control-center validates the configuration with every user input, which includes trying to establish a connection to the SAP® system
If you enter the properties in the JCo Destination configuration block first, value recommendations for some other properties will be loaded directly from the SAP source system
To display the advanced JCo configuration properties in the Confluent Control Center UI, you can set the configuration property Display advanced properties to 1.
Since you can poll data from different RFMs in one connector instance, an additional RFC Source configuration block will appear once you provided the required information for the first RFC Source configuration block
The number of configurable RFMs in the Confluent Control Center UI for one connector instance is restricted. The same applies for the amount of selection conditions in full initialization mode.

1.3.5