Configuration

The ODP Source Connector package includes detailed configuration documentation, which can be found in the following locations:

Additionally, the connector package includes a configuration template file in the etc/ subfolder:

  • odp-source-connector.properties

The connector utilizes extensive validators and recommenders and has been tested with the Confluent Control Center UI.

Benefits of Configuring through Confluent Control Center

The ODP Source Connector fully supports configuration through the Confluent Control Center UI. Compared to using property files, configuring through this UI offers several advantages:

  • Suggested property values
  • A comprehensive list of recommended settings
  • Incremental visibility into relevant configuration options
  • A wide range of interactive validations
  • And more

To maintain optimal performance, the Confluent Control Center UI displays a maximum of 1000 recommendations at a time. Once the required fields for a service entity are filled out in the configuration UI, a new section will appear, allowing you to configure additional service entities.

To see how easy it is to set up the ODP Connector using the Confluent Control Center, watch the screen recordings.

Installation

Configuration

SAP® System Connection

The ODP Connector includes configuration properties from SAP® JCo (prefixed with jco.). For a detailed description of these properties, please refer to the Java documentation of the com.sap.conn.jco.ext.DestinationDataProvider interface. The JCo JavaDoc can be found within the sapjco3.jar package.

A basic configuration for a SAP® JCo client destination is as follows:

# SAP Netweaver application host DNS or IP
jco.client.ashost = 127.0.0.1
# SAP system number
jco.client.sysnr = 20
# SAP client number
jco.client.client = 100
# SAP RFC user
jco.client.user = user
# SAP user password
jco.client.passwd = password

If you’re using SAP’s Message Server for load balancing, replace jco.client.ashost with jco.client.mshost.

To ensure encryption and data integrity, use SAP® Secure Network Connection (SNC) properties. The connector also supports various authentication types like user/password, SNC SSO, and SAP® cookie logon tickets.

SAP® RFC Communication Ports

Refer to the TCP/IP ports for SAP® products. Common ports:

  • 33<NN> (unencrypted)
  • 48<NN> (encrypted)

SAP® Authorizations

To be able to establish communication between the connector and the SAP® system and within the framework of authorization management, you need an SAP® user. The SAP® user must be of type Communications or Dialog. To save SAP® dialog resources, the Communications type is recommended.

Minimum Required Authorizations

  • Object S_RFC:
    ACTVT: 16
    RFC_NAME: RFCPING, SMLG_GET_DEFINED_GROUPS, RFC_GET_FUNCTION_INTERFACE, RFC_METADATA_GET
    RFC_TYPE: FUNC

  • Object S_RFC:
    ACTVT: 16
    RFC_NAME: RFC1, RODPS_REPL, BUS1090
    RFC_TYPE: FUGR

  • Object S_BTCH_JOB: JOBACTION: RELE
    JOBGROUP: *

  • Object S_RO_OSOA:
    ACTVT: 03
    OLTPSOURCE: [odp names]
    OSOAAPCO: *
    OSOAPART: DATA, DEFINITION

  • Object S_TABU_NAM:
    ACTVT: 03
    TABLE: V_CURC

Note

For more detailed information, see the following SAP® support notes:

SAP® System Requirements

For a comprehensive overview of ODP and a list of supported ODP extractors, please refer to the ODP Introduction and ODP enabled extractors pages.

Here are some crucial SAP® support notes to consider:

ODP Contexts

A context represents a source of ODPs. Context identifiers exist for all SAP® technologies whose analytical views can be exposed as ODPs. Please refer to the relevant links and support notes for information on the specific prerequisites and configuration procedures for your ODP context. Currently, the following ODP-contexts are available (depending on release):

ODP Suffixes

For instance, when being used to extract data from SAP® BW InfoProviders, ODP data sources receive a suffix describing the type of data:

Suffix Meaning
$F transaction fata/facts
$P time-independant master data/attributes
$Q time-dependent master data/attributes
$T texts
$H hierarchies
$M joined time-(in)dependant master data view

ODP Source Configuration

ODP being a publish/subscribe oriented component with potentially multiple subscribers, each subscriber must provide a unique name, referred to as sap.odp.subscriber-name, specifically for delta extractions. This name serves as the primary key used for tracking pointers that indicate which delta data has already been successfully extracted.

The name and context properties must correspond to the ODP name and ODP context configured in SAP®. The available contexts vary depending on the source system. To get a dropdown list of valid values in the Confluent Control Center, a prefix must be entered. A list of possible contexts can be found above in section ODP contexts.

topic defines the Kafka output topic the connector producer will use to publish extracted data.

# The remaining configs are specific to the ODP source connector.
# Unique subscriber name used to subscribe to ODP sources in SAP.
# This identifier will be used to calculate delta requests.
sap.odp.subscriber-name = OhJaODPKafkaConnector
sap.odp#00.name = Test
sap.odp#00.context = SAPI
sap.odp#00.topic = ODPSAPITEST

Configuration Support with our CLI Tool

The ODP connector’s package includes a command line interface (CLI) to help with various tasks such as:

  • Validating connector properties
  • Testing SAP® connections
  • Listing and extracting available ODPs as well as query details of an ODP
  • Extracting schemas for ODP sources

Prerequisites

To run the CLI you need to provide the following dependencies in the CLASSPATH:

  1. Scala runtime libraries
  2. kafka-clients
  3. kafka-connect-odp connector libraries
  4. SAP® Java Connector 3.1 SDK

Since a Java runtime is provided by the Confluent Platform and the Scala runtime libraries are part of the connector package, executing the CLI with Java would not require installing Scala.

Java
java -cp <kafka-connect-odp>:<kafka-clients>:\
<kafka-avro-serializer>:<sapjco3> \
org.init.ohja.kafka.connect.odp.source.ODPDetailsApp \
<command> <options>
Scala
scala -cp <kafka-connect-odp>:<kafka-clients>:\
<kafka-avro-serializer>:<sapjco3> \
org.init.ohja.kafka.connect.odp.source.ODPDetailsApp \
<command> <options>

The required Kafka libraries are usually located in:

  • /usr/share/java/kafka/
  • /usr/share/java/kafka-serde-tools/

Available Commands

  • ping: Test the SAP® connection.
  • list-odp: List all available ODP sources from the SAP system.
  • odp-details: Print meta information of an ODP source.
  • extract-schema: Print the avro schema of an ODP source. Extracting Avro schemas requires the presence of io.confluent.kafka-connect-avro-converter in the classpath.
usage:
    ODPDetailsApp <command> <options>

commands:
    ping
    list-odp
    odp-details -n <object name> -c <context name>
    extract-schema -n <object name> -c <context name>

mandatory options:
    -p <path to connector properties file>
Note

Avro schemas may differ if the Single Message Transform(s) in the connector configuration are used.

Using the ODP Source Connector CLI

1.3.6