Error Handling Mechanisms

The connector applies different kind of validations and error handling mechanisms like configuration validation, offset recovery, upstream connection tests, connection retries with graceful back-off and repeatable source request extractions.

The connector maps known exceptions into ConnectException, allowing them to be handled appropriately by Kafka Connect. Any errors or warnings are logged using SLF4J, as described in the Logging section.

Configuration Validation and Recommenders

Single configuration parameter validation extending the Validator class will throw exceptions of type ConfigException in case of invalid configuration values. Additionally, the connector overrides the validate method to validate interdependent configuration parameters and adds error messages to class ConfigValue in case of any invalid parameter values. The corresponding parameters containing invalid values will be framed in red in the Confluent Control Center together with an appropriate error message.

Graceful Backoff Retry Mechanism

When the connector encounters connection or communication issues with the SAP® system, it uses a retry backoff strategy to handle the situation. Here’s how it works:

  1. Retry Attempts: The connector will try reconnecting a maximum number of times, which you can set using the sap.odp.max.retries property.

  2. Backoff Intervals: After each failed attempt, the connector pauses for a random duration. This pause is between the minimum and maximum values you set with sap.odp.min.retry.backoff.ms and sap.odp.max.retry.backoff.ms.

  3. Exception Handling: Connection and communication issues are grouped into exception categories. You can configure which exception groups should trigger the backoff retry strategy using the sap.odp.retry.exception.groups property. For more details, refer to the JCo JavaDocs for com.sap.conn.jco.JCoException. You can also include BAPI return messages from any message class.

Error Tolerance

Kafka’s errors.tolerance is not applied to the internal logic of a Kafka Connect source connector, therefore the ODP Source Connector provides its own configuration errors.tolerance to define the behaviour of error handling:

  • none (default): Any error will cause an immediate task failure.
  • skiprow: Skips over problematic records instead of failing.
  • initializefield: Initializes values for fields with issues.
  • deadletterqueue: Sends erroneous records to a Dead Letter Queue (DLQ), though delivery guarantees are not provided for DLQ records.

This error handling behaviour does not cover all logical areas of the connector and only applies to issues in data type and value conversion from SAP® to Kafka.

In addition to the error tolerance, configure the following options:

  • errors.tolerance.row-regexp: Define a regular expression to select specific datasets for error tolerance. Check the Headers section for information on unique identifiers.

  • errors.tolerance.fields (optional): Specify a list of field names to apply error tolerance behavior. If a problematic field isn’t on this list, the connector defaults to none.

1.3.6