Monitoring

Logging

The connectors make use of SLF4J for logging integration. The loggers use the logger names org.init.ohja.kafka.connect.idoc.source and org.init.ohja.kafka.connect.idoc.sink. The logging destination and format might be configured via Kafka Connect configuration file.

The connector provides additional log location information ohja.location using MDC (mapped diagnostic context). The log location contains the name of the nearest enclosing definition of val, class, trait, object or package and the line number.

Example Log4j 1.x appender:

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %X{connector.context}%m %X{ohja.location}%n

If using the Confluent Platform, the logs can be seen by running a CLI command:

confluent local services connect log

JMX Metrics

The IDoc Connectors support all the connector and task metrics provided by Kafka Connect through Java Management Extensions (JMX).

In addition, the IDoc Connectors provide extra JMX metrics for accessing state managed by the connector.

Source Connector

MBean: org.init.ohja.kafka.connect:type=idoc-source-task-metrics,connector=([-.w]+),task=([d]+)

Metric	Explanation
thread-retries	Count of retries performed in a connector thread that is in retrying state.
thread-next-retry	Timestamp for next retry performed in a connector thread that is in retrying state.
jco-destination-pool-capacity	Maximum number of connections that will be held open for the JCo destination instance.
jco-destination-peak-limit	Maximum number of connections that can be used simultaneously with the JCo destination instance
jco-destination-pooled-connections	Count of connections that are currently held open for the JCo destination instance.
jco-destination-used-connections	Count of connections that are currently being used with the JCo destination instance.
${configGroup}-file-path	File path configured for IDoc configured in the configuration group.
${configGroup}-file-regexp	File name regular expression filter for IDoc configured in the configuration group.
${configGroup}-files-queue	Files currently in processing queue for IDoc configured in the configuration group.
${configGroup}-committed-record	Current committed Kafka offset for IDoc configured in the configuration group.
${configGroup}-recent-poll	Timestamp for last time of extraction for IDoc configured in the configuration group.
${configGroup}-files-remaining	Remaining files in poll cycle for IDoc configured in the configuration group.
${configGroup}-files-poll-size	Sum of remaining file sizes in MB to be processed in poll cycle for IDoc configured in the configuration group.

Sink Connector

MBean: org.init.ohja.kafka.connect:type=idoc-sink-task-metrics,connector=([-.w]+),task=([d]+)

Metric	Explanation
thread-retries	Count of retries performed in a connector thread that is in retrying state.
thread-next-retry	Timestamp for next retry performed in a connector thread that is in retrying state.
jco-destination-pool-capacity	Maximum number of connections that will be held open for the JCo destination instance.
jco-destination-peak-limit	Maximum number of connections that can be used simultaneously with the JCo destination instance
jco-destination-pooled-connections	Count of connections that are currently held open for the JCo destination instance.
jco-destination-used-connections	Count of connections that are currently being used with the JCo destination instance.
${configGroup}-topic	Name of source topic for IDoc configured in the configuration group.
${configGroup}-sap-tid	SAP® transaction ID of the last package of IDocs sent to SAP® for IDoc configured in the configuration group.
${configGroup}-pack-size	Package size/Amount of IDocs as part of the last package for IDoc configured in the configuration group.