Configuration

The OData V2 Connectors package includes detailed configuration documentation for both the source and sink connector, which can be found in the following locations:

• Source Configuration
• Sink Configuration
• Examples

Additionally, the connectors package includes configuration template files in the etc/ subfolder:

• odata-source-connector.properties
• odata-sink-connector.properties

These connectors utilize extensive validators and recommenders and have been tested with the Confluent Control Center UI.

Benefits of Configuring through Confluent Control Center

The OData V2 Connectors fully support configuration through the Confluent Control Center UI. Compared to using property files, configuring through this UI offers several advantages:

• Suggested property values
• A comprehensive list of recommended settings
• Incremental visibility into relevant configuration options
• A wide range of interactive validations
• And more

To maintain optimal performance, the Confluent Control Center UI displays a maximum of 1000 recommendations at a time.

Once the required fields for a service entity are filled out in the configuration UI, a new section will appear, allowing you to configure additional service entities.

OData V2 Source Connector

OData V2 Sink Connector

Activating SAP OData Service Recommendations

To enable SAP OData service recommendations in the Confluent Control Center’s connector configuration, you need to activate the /sap/opu/odata/iwfnd/CATALOGSERVICE in the corresponding SAP system.

Additionally, the OData V2 Source Connector includes built-in support for ODP-based extraction.

Minimal Configuration Parameters

Service Destination Configuration

A minimal OData V2 service destination configuration looks like this:

# OData V2 server host as either DNS or IP
sap.odata.host.address = services.odata.org
# OData V2 server port
sap.odata.host.port = 443
# OData V2 protocol (supported values are http or https)
sap.odata.host.protocol = https
# OData V2 user name for basic authentication.
# For services not requiring authentication this can be set to any value.
sap.odata.user.name = anonymous
# OData V2 user password for basic authentication.
# For services not requiring authentication this can be set to any value.
sap.odata.user.pwd = anonymous

Encrypted communication is supported by using HTTPS.
The supported authentication types are basic authentication with a username and password and OAuth client credentials flow.

Service Entity Set Configuration

A minimal service entity set configuration looks like this:

# OData V2 URL service path
sap.odata#00.service = /V2/Northwind/Northwind.svc
# OData V2 entity set name
sap.odata#00.entityset = Order_Details
# Kafka topic name
sap.odata#00.topic = Order_Details

• The combination of service and entityset properties uniquely identifies the data set.
• topic defines the Kafka output topic the connector producer will use to publish extracted data.

Modern web browsers allow for direct use and querying of OData services. This feature can be used, e.g. to test the service or to identify the properties and values required for the connectors’ configuration.

Configuration Support with our CLI Tool

The kafka-connect-odatav2 connector`s package includes a command line interface (CLI) to help with various tasks such as:

• Validating connector properties
• Ping OData V2 services
• Testing SAP® connections
• Listing OData V2 services of an SAP system
• Retrieving a list of entity set names for a service
• Extracting schemas for entity sets of OData V2
• Listing and deleting active subscriptions from the SAP source system

Since the CLI is written in Scala you can execute it in Scala or Java.

Prerequisites

To run the CLI you need to provide the following dependencies in the CLASSPATH:

1. Scala runtime libraries
2. kafka-clients
3. kafka-connect-odata connector libraries

As a Java runtime is provided by the Confluent Platform and the Scala runtime libraries are part of the connector package, executing the CLI with Java would not require installing Scala.

Java

copyjava -cp <kafka-connect-odatav2>:<kafka-clients>:\
<kafka-avro-serializer> \
org.init.ohja.kafka.connect.odatav2.OData2App \
<command> <options>

Scala

copyscala -cp <kafka-connect-odatav2>:<kafka-clients>:\
<kafka-avro-serializer> \
org.init.ohja.kafka.connect.odatav2.OData2App \
<command> <options>

The required Kafka libraries are usually located in:

• /usr/share/java/kafka/
• /usr/share/java/kafka-serde-tools/

Available Commands

• ping: Test the SAP® connection.
• list-entitysets: List all available entity sets from the SAP system.
• list-services
• active-subscriptions
• delete-subscription
• extract-schema: Print the avro schema of an OData source. Extracting Avro schemas requires the presence of io.confluent.kafka-connect-avro-converter in the classpath.

usage:
  OData2App <command> <options>

commands:
  ping -s <relative service path>
  list-entitysets -s <relative service path>
  extract-schema -s <relative service path> -e <entity-set>
  list-services                                              (SAP only)
  active-subscriptions -s <relative service path>            (SAP only)
  delete-subscription -u <subscription URL>                  (SAP only)

mandatory options:
  -p <path to connector properties file>