Monitoring and Logging

Logging

The connector makes use of SLF4J for logging integration. The logger uses the logger name org.init.ohja.kafka.connect.odp.source and can be configured, e.g. in the log4j configuration properties in the Confluent Platform.

SAP® JCo includes a logger called com.sap.conn.jco which can only be used with log4j. In addition to setting the logging level for the JCo logger, one can use configuration property jco.trace_level to fine tune the level of logging.

The connector provides additional log location information ohja.location using MDC (mapped diagnostic context). The log location contains the name of the nearest enclosing definition of val, class, trait, object or package and the line number.

Example Log4j 1.x appender:

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %X{connector.context}%m %X{ohja.location}%n

If using the Confluent Platform, the logs can be seen by running a CLI command:

confluent local services connect log

JMX Metrics

The ODP Source Connector supports all the connector and task metrics provided by Kafka Connect through Java Management Extensions (JMX). In addition, the ODP Source Connector provides extra JMX metrics for accessing state managed by the connector.

MBean: org.init.ohja.kafka.connect:type=odp-source-task-metrics,connector=([-.w]+),task=([d]+)

Metric	Explanation
task-phase	Phase in which the connector is currently operating. The connector can either be in fill cache, fetch, processing queue or waiting phase.
task-active-threads	Count of active threads that are in use by the connector for processing data.
thread-retries	Count of retries performed in a connector thread that is in retrying state.
thread-next-retry	Timestamp for next retry performed in a connector thread that is in retrying state.
jco-destination-pool-capacity	Maximum number of connections that will be held open for the JCo destination instance.
jco-destination-peak-limit	Maximum number of connections that can be used simultaneously with the JCo destination instance
jco-destination-pooled-connections	Count of connections that are currently held open for the JCo destination instance.
jco-destination-used-connections	Count of connections that are currently being used with the JCo destination instance.
${configGroup}-odp-context	ODP source context of ODP configured in the configuration group.
${configGroup}-odp-name	ODP source name of ODP configured in the configuration group.
${configGroup}-fetch-completed	Package ID of the latest successfully fetched package for ODP configured in the configuration group.
${configGroup}-prefetch-completed	Package ID of latest prefetched package for ODP configured in the configuration group.
${configGroup}-committed-record	Current committed Kafka offset for ODP configured in the configuration group.
${configGroup}-recent-request-update	Timestamp for last time of extraction for ODP configured in the configuration group.
${configGroup}-request-mode	The request mode for the ODP extraction in the configuration group. The request mode can either be rda, full or delta.

ODP Test & Monitoring in SAP®

ODPs use an Operational Data Queue (ODQ) system to manage delta changes. You can access the ODQ monitor using the SAP® transaction code ODQMON. This monitor offers details on all active ODP queues, including their subscribers, request history, and the option to inspect stored data.

For testing ODPs within SAP®, the RODPS_REPL_TEST report provides a comprehensive toolset.

To manually close requests (whether successful or failed) or clear out “zombie” requests in a running connector, use the RODPS_REPL_ODP_CLOSE function module.

If you’re working with Change Data Capture-enabled ABAP CDS Views, you can use transaction code DHCDCMON to access the SAP® CDC Monitor.