Monitoring and Logging

Logging

The connector makes use of SLF4J for logging integration. The logger uses the logger name org.init.ohja.kafka.connect.webds.sink and can be configured e.g., in the log4j configuration properties in the Confluent Platform.

SAP JCo includes a logger called com.sap.conn.jco which can only be used with log4j. In addition to setting the logging level for the JCo logger, one can use configuration property jco.trace_level to fine tune the level of logging.

The connector provides additional log location information ohja.location using MDC (mapped diagnostic context). The log location contains the name of the nearest enclosing definition of val, class, trait, object or package and the line number.

Example Log4j 1.x appender:

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %X{connector.context}%m %X{ohja.location}%n

If using the Confluent Platform, the logs can be seen by running a CLI command:

confluent local services connect log
Note

For additional details check out the official tutorial.

JMX Metrics

The Webservice Data Source Sink Connector supports all the connector and task metrics provided by Kafka Connect through Java Management Extensions (JMX). In addition, the Webservice Data Source Sink Connector provides extra JMX metrics for accessing state managed by the connector.

MBean: org.init.ohja.kafka.connect:type=webds-sink-task-metrics,connector=([-.w]+),task=([d]+)

Metric Explanation
retries Count of retries performed in the connector task that is in retrying state.
${configGroup}-webds-system Webservice data source identifier “logical system” configured in the configuration group.
${configGroup}-webds-name Webservice data source identifier “data source name” configured in the configuration group.

Data Source Monitoring in SAP

SAP BI related transaction RSA1 can be used for monitoring and administration purposes. In the Modeling - Data Sources perspective you fill be able to see the open and closed data load requests, and you will have the possibility to inspect the data persisted in the so-called Persisting Staging Area.

WebDS
WebDS