Kafka Connect - ODP Source Connector

Product Description

The ODP Source Connector is a powerful Kafka Connector designed to seamlessly extract and replicate data from SAP® Operational Data Provisioning (ODP) sources into Kafka.

image

Key Features

SAP® Operational Data Provisioning (ODP) provides a robust infrastructure for efficient data extraction and replication across various SAP® systems. Leveraging ODP, the connector enables you to:

  • Full Data Extraction: Retrieve all data from specific ODP sources in a single batch.
  • Delta Data Extraction: Continuously capture and extract only the changes (deltas) via the Operational Data Queue (ODQ), allowing multiple independent subscribers to consume changes as they occur.
  • Real-Time Data Acquisition: Stream data in real-time for immediate access and processing.

Flexible Data Initialization

When connecting to a new SAP® ODP data source, the connector supports flexible initialization methods:

  • Delta Initialization: Choose between extracting all existing data or starting fresh with delta updates from the moment the connector begins running.
  • Full Initialization: Apply complex filters during initialization for more tailored data extraction. This option provides greater control over the data captured during the initial load, but direct full extractions via the ODP API are not yet supported by the connector.

Key Benefits of the SAP® ODP Infrastructure

  • Monitoring and Management: Built-in services to monitor and track delta data requests.
  • Efficient Data Storage: Data is stored in a compressed format within the ODQ of the source system, optimizing storage and performance.
  • Multiple Subscriptions: Multiple consumers can access and process data changes concurrently.
  • Data Retention: Data in the ODQ is retained for a configurable period, allowing for re-extraction or recovery.
  • Reusability: Data can be extracted multiple times, enhancing flexibility for data recovery or reprocessing.

The ODP Source Connector integrates with Kafka Connect’s offset management to ensure exactly-once delivery semantics, ensuring reliable and consistent data flow.

Data Format and Schema Support

ODP provides data in a tabular format, ensuring simplicity and compatibility with most data pipelines. While nested data types are not supported, the connector fully supports a wide range of SAP® data types. Additionally, metadata for the extracted fields is provided directly by the SAP® system, allowing the connector to automatically generate a detailed Kafka Connect schema for each SourceRecord, simplifying downstream data handling.

The ODP Source Connector is a reliable solution for integrating SAP® data into Kafka, supporting a variety of extraction modes, ensuring real-time data processing, and offering flexibility to meet different initialization needs.

Versioning and Compatibility

ODP Source Connector

  • Connector name: ODP Source Connector
  • Version: 1.3.6
  • Type: Kafka Connect source connector

ODP Source System

The ODP Source Connector uses the SAP® ODP API V2 (see SAP® Note 1931427 - ODP Data Replication API 2.0), which is available in the following Netweaver versions:

  • PI_BASIS 730 Support Package 14 (Part of SAP® NetWeaver 7.30 Support Package 14)
  • PI_BASIS 731 Support Package 16 (Part of SAP® NetWeaver 7.03 Support Package 16 und SAP® NetWeaver 7.31 Support Package 16)
  • PI_BASIS 740 Support Package 11 (Part of SAP® NetWeaver 7.40 Support Package 11)
  • SAP_BW 750 Support Package 0 (including earlier PI_BASIS-Packages)
  • DW4CORE 100 SP 0 (including earlier PI_BASIS-Packages)
  • DW4CORE 200 SP 0 (including earlier PI_BASIS-Packages)

Kafka

  • Tested on Confluent Platform v5.5.1, v6.0.1, v7.0.1, v7.3.0, v7.5.0, v7.6.0 and v7.7.0
  • Tested with Apache Kafka and Connect API v2.5, v2.6, v2.8, v3.3, v3.5, v3.6 and v3.8

Dependencies

The connectors make use of SAP® Java Connector (JCo) v3.1.11 released by SAP®.

According to the license of SAP®, it is not allowed to bundle the library with other software. Therefore, the JCo library needs to be obtained separately from the SAP® Marketplace, see SAP JCo.

Licensing

See licenses for INIT’s evaluation license (DE and EN) or for more information on the dependencies` license.

Support and Contact Information

Company

INIT Individuelle Softwareentwicklung & Beratung GmbH
Rotebühlplatz 21
70178 Stuttgart - Deutschland
Amtsgericht Stuttgart, HRB 224214
Geschäftsführer: Thomas Winkelsdorf
https://init-software.de/

Business and technical Contact

Ronald Schertle
connector-contact@init-software.de
+49 173 81 49 709

Full Enterprise Support

Our full enterprise support offers you access to top-tier support from our connector developers, tailored to meet your specific needs. This includes:

  • 8/5 support
  • 60-minutes response time depending on support plan
  • Full application lifecycle support from development to operations
  • Access to expert support engineers
  • Consultancy in case of individual SAP integration requirements

Software Evaluation

An evaluation version of the connectors, limited to 30 days of use, can be downloaded from the INIT repository. This allows a prospective customer to test the connectors at an early stage.

Please contact connector-contact@init-software.de for more information.

1.3.6