Key Features

Delivery Semantics

• At-least-once semantics: Can be achieved if the RFM makes proper use of the offset parameters configured by offset-parameter.in and offset-parameter.out. These parameters can be used to implement a kind of source system change data capture and will automatically be synchronized with Kafka offsets by the connector.

• Exactly-once-semantics: Can only be achieved if a function call is regarded as a single message, as offsets are related to single function calls and not e.g., to single records as part of a table parameter of the RFM.

The RFC Source Connector stands out by ensuring that data is processed exactly once when offset-parameter.in and offset-parameter.out are not empty.

Enabling Exactly-Once in Kafka

To enable exactly-once support, update the Kafka worker configuration property exactly.once.source.support to preparing or enabled. Refer to KIP-618 for details.

When using a source connector, you can configure the exactly.once.support property to make exactly-once delivery either requested or required (default is requested).

• Required: Ensures exactly-once delivery, with a preflight check. The connector fails if it can’t support exactly-once delivery.
• Requested (default): No preflight check is performed.

Offset Handling

The RFM configured to be called by the connector can return an arbitrary offset using an exporting parameter of the RFM, configured using configuration offset-parameter.out. The offset parameters can have an arbitrary type, as long as contained data types are supported by the connector. In a succeeding function call to the RFM, the connector will pass the offset value returned to the preceding call using the RFM’s offset importing parameter, configured using configuration offset-parameter.in.