Error Handling

The Business Events Source Connector applies various validations and error handling mechanisms:

  • Configuration validation
  • Offset recovery
  • Upstream connection tests
  • HTTP status checks
  • Connection retries

The connector maps known exceptions to exception type ConnectException, which can be handled by Kafka Connect accordingly. Errors and warnings are logged using SLF4J, as described in section Logging.

Configuration Validation and Recommenders

Single configuration parameter validation extending the Validator class will throw exceptions of type ConfigException in case of invalid configuration values. Additionally, the connector overrides the validate method to validate interdependent configuration parameters and adds error messages to class ConfigValue in case of any invalid parameter values. The corresponding parameters containing invalid values will be framed in red in the Confluent Control Center together with an appropriate error message.

Offset Handling

The logic offset for each message produced by the source connector contains:

  • business event identifier (GUID)
  • business event creation timestamp

In case of issues or task rebalances:

  1. The connector retrieves the latest offsets from Kafka Connect.
  2. It restarts extracting the last processed requests from the OData V2 service, to ensure no data loss.
  3. All the messages up to the record number of the corresponding logical offset are regarded as duplicates and are filtered out.
  4. Only new messages are sent to Kafka to avoid duplicates in the output topic.

This means that in all cases of a restart the source connector will immediately try to recover from the last request, before going into a wait-loop for the next execution interval configured by exec-period.

Graceful Backoff Retry Mechanism

When the connector encounters connection or communication issues with the configured OData V2 service endpoint, it uses a retry backoff strategy to handle the situation. Here’s how it works:

  1. Retry Attempts: The connector will try reconnecting a maximum number of times, which you can set using the sap.odata.max.retries property.

  2. Backoff Intervals: After each failed connection attempt, the connector pauses for a random number of milliseconds in the range between the values of configuration properties sap.odata.min.retry.backoff.ms and sap.odata.max.retry.backoff.ms.

1.3.6