Monitoring and Logging
Logging
The connector makes use of SLF4J for logging integration. The logger uses the logger name org.init.ohja.kafka.connect.webds.sink and can be configured e.g., in the log4j configuration properties in the Confluent Platform.
SAP JCo includes a logger called com.sap.conn.jco which can only be used with log4j. In addition to setting the logging level for the JCo logger, one can use configuration property jco.trace_level to fine tune the level of logging.
The connector provides additional log location information ohja.location using MDC (mapped diagnostic context). The log location contains the name of the nearest enclosing definition of val, class, trait, object or package and the line number.
Example Log4j 1.x appender:
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %X{connector.context}%m %X{ohja.location}%n
If using the Confluent Platform, the logs can be seen by running a CLI command:
confluent local services connect log
For additional details check out the official tutorial.
JMX Metrics
The Webservice Data Source Sink Connector supports all the connector and task metrics provided by Kafka Connect through Java Management Extensions (JMX). In addition, the Webservice Data Source Sink Connector provides extra JMX metrics for accessing state managed by the connector.
MBean: org.init.ohja.kafka.connect:type=webds-sink-task-metrics,connector=([-.w]+),task=([d]+)
| Metric | Explanation |
|---|---|
| retries | Count of retries performed in the connector task that is in retrying state. |
| ${configGroup}-webds-system | Webservice data source identifier “logical system” configured in the configuration group. |
| ${configGroup}-webds-name | Webservice data source identifier “data source name” configured in the configuration group. |
Data Source Monitoring in SAP
SAP BI related transaction RSA1 can be used for monitoring and administration purposes. In the Modeling - Data Sources perspective you fill be able to see the open and closed data load requests, and you will have the possibility to inspect the data persisted in the so-called Persisting Staging Area.
