integrate-with-sap-cloud-logging-eac5771.md

Integrate With SAP Cloud Logging

Configure the Telemetry module to send logs, metrics, and traces from your cluster to an SAP Cloud Logging instance. By centralizing this data in your SAP Cloud Logging instance, you can store, visualize, and analyze the observability of your applications.

Prerequisites

• Kyma as the target deployment environment, with the following modules added (see Adding and Deleting a Kyma Module):

  • Telemetry module

  • To collect data from your Istio service mesh: Istio module (default module)

  • SAP BTP Operator module (default module)

• An instance of SAP Cloud Logging with OpenTelemetry ingestion enabled. For details, see Ingest via OpenTelemetry API Endpoint.

  Tip:

  Create the SAP Cloud Logging instance with the SAP BTP service operator (see Create an SAP Cloud Logging Instance through SAP BTP Service Operator), because it takes care of creation and rotation of the required Secret. However, you can choose any other method of creating the instance and the Secret, as long as the parameter for OTLP ingestion is enabled in the instance. For details, see Configuration Parameters.

• A Secret in the respective namespace in your Kyma cluster, holding the credentials and endpoints for the instance. It’s recommended that you rotate your Secret (see SAP BTP Security Recommendation BTP-CLS-0003). In the following example, the Secret is named “sap-cloud-logging” and the namespace “sap-cloud-logging-integration”, as illustrated in the secret-example.yaml.

• Kubernetes CLI (kubectl) (see Install the Kubernetes Command Line Tool).

• UNIX shell or Windows Subsystem for Linux (WSL) to execute commands.

Context

The Telemetry module supports shipping logs and ingesting distributed traces as well as metrics from applications and the Istio service mesh to SAP Cloud Logging.

First, set up the Telemetry module to ship the logs, traces, and metrics to your backend by deploying the pipelines and other required resources. Then, you configure Kyma dashboard integration. Finally, set up SAP Cloud Logging alerts and dashboards.

SAP Cloud Logging is an instance-based and environment-agnostic observability service to store, visualize, and analyze logs, metrics, and traces.

Ship Logs To SAP Cloud Logging

You can set up ingestion of logs from applications and the Istio service mesh to the OTLP endpoint of the SAP Cloud Logging service instance.

Procedure

Set Up Application Logs

1. Deploy a LogPipeline for application logs:

   kubectl apply -f - <<EOF
   apiVersion: telemetry.kyma-project.io/v1beta1
   kind: LogPipeline
   metadata:
     name: sap-cloud-logging
   spec:
     input:
       runtime:
         enabled: true
     output:
       otlp:
         endpoint:
           valueFrom:
             secretKeyRef:
               name: sap-cloud-logging
               namespace: sap-cloud-logging-integration
               key: ingest-otlp-endpoint
         tls:
           cert:
             valueFrom:
               secretKeyRef:
                 name: sap-cloud-logging
                 namespace: sap-cloud-logging-integration
                 key: ingest-otlp-cert
           key:
             valueFrom:
               secretKeyRef:
                 name: sap-cloud-logging
                 namespace: sap-cloud-logging-integration
                 key: ingest-otlp-key
   EOF
   

2. Verify that the LogPipeline is running:

   kubectl get logpipelines
   

Set Up Istio Access Logs

By default, Istio sidecar injection and Istio access logs are disabled in Kyma. To analyze them, you must enable them:

3. Enable Istio sidecar injection for your workload (see Enabling Istio Sidecar Proxy Injection).

4. Configure the Istio Telemetry resource (see Configure Istio Access Logs) with the kyma-logs extension provider.

Ship Traces To SAP Cloud Logging

You can set up ingestion of distributed traces from applications and the Istio service mesh to the OTLP endpoint of the SAP Cloud Logging service instance.

Procedure

Set Up Traces

1. Deploy a TracePipeline:

   kubectl apply -f - <<EOF
   apiVersion: telemetry.kyma-project.io/v1beta1
   kind: TracePipeline
   metadata:
     name: sap-cloud-logging
   spec:
     output:
       otlp:
         endpoint:
           valueFrom:
             secretKeyRef:
               name: sap-cloud-logging
               namespace: sap-cloud-logging-integration
               key: ingest-otlp-endpoint
         tls:
           cert:
             valueFrom:
               secretKeyRef:
                 name: sap-cloud-logging
                 namespace: sap-cloud-logging-integration
                 key: ingest-otlp-cert
           key:
             valueFrom:
               secretKeyRef:
                 name: sap-cloud-logging
                 namespace: sap-cloud-logging-integration
                 key: ingest-otlp-key
   EOF
   

2. Verify that the TracePipeline is running:

   kubectl get tracepipelines
   

Set Up Istio Tracing

By default, Istio sidecar injection and Istio tracing are disabled in Kyma. To analyze them, you must enable them:

3. Enable Istio sidecar injection for your workload (see Enabling Istio Sidecar Proxy Injection).

4. Configure the Istio Telemetry resource to use the kyma-traces extension provider based on OTLP (see Configure Istio Tracing).

Ship Metrics To SAP Cloud Logging

You can set up ingestion of metrics from applications and the Istio service mesh to the OTLP endpoint of the SAP Cloud Logging service instance.

Procedure

1. Deploy a MetricPipeline:

   kubectl apply -f - <<EOF
   apiVersion: telemetry.kyma-project.io/v1beta1
   kind: MetricPipeline
   metadata:
     name: sap-cloud-logging
   spec:
     input:
       prometheus:
         enabled: false
       istio:
         enabled: false
       runtime:
         enabled: false
     output:
       otlp:
         endpoint:
           valueFrom:
             secretKeyRef:
               name: sap-cloud-logging
               namespace: sap-cloud-logging-integration
               key: ingest-otlp-endpoint
         tls:
           cert:
             valueFrom:
               secretKeyRef:
                 name: sap-cloud-logging
                 namespace: sap-cloud-logging-integration
                 key: ingest-otlp-cert
           key:
             valueFrom:
               secretKeyRef:
                 name: sap-cloud-logging
                 namespace: sap-cloud-logging-integration
                 key: ingest-otlp-key
   EOF
   

   The default configuration creates a gateway to receive OTLP metrics from your applications.

2. Optional: To collect additional metrics, such as those from the runtime or Istio, configure the presets in the input section of the MetricPipeline. For the available options, see Collecting Metrics.

3. Verify that the MetricPipeline is running:

   kubectl get metricpipelines
   

Set Up Kyma Dashboard Integration

You can add direct links from Kyma dashboard to SAP Cloud Logging.

Procedure

1. If your Secret has a name or namespace different from the example, download the file and edit the dataSources section before you apply it.

2. Apply the ConfigMap:

   kubectl apply -f https://raw.githubusercontent.com/kyma-project/telemetry-manager/main/docs/user/integration/sap-cloud-logging/kyma-dashboard-configmap.yaml
   
   

Use SAP Cloud Logging Alerts

You can import predefined alerts for SAP Cloud Logging to monitor the health of your telemetry integration.

Procedure

1. In the SAP Cloud Logging dashboard, define a “notification channel” to receive alert notifications.

2. To import a monitor, use the development tools of the SAP Cloud Logging dashboard.

3. Execute POST _plugins/_alerting/monitors, followed by the contents of the respective JSON file.

4. Depending on the pipelines you are using, enable some or all of the following alerts:

   The alerts are based on JSON documents defining a Monitor for the alerting plugin.

   Monitored Component	File	Description
   SAP Cloud Logging	alert-health.json	Monitors the health of the underlying OpenSearch cluster in SAP Cloud Logging using the cluster health API. Triggers if the cluster status becomes red.
   SAP Cloud Logging	alert-rejection-in-progress.json	Monitors the cls-rejected-* index for new data. Triggers if new rejected data is observed.
   Kyma Telemetry Integration	alert-telemetry-status.json	Monitors the status of the Telemetry module. Triggers if the module reports a non-ready state.
   Kyma Telemetry Integration	alert-log-ingestion.json	Monitors the LogPipeline. Triggers if log data stops flowing.
   Kyma Telemetry Integration	alert-trace-ingestion.json	Monitors the TracePipeline. Triggers if trace data stops flowing to SAP Cloud Logging.
   Kyma Telemetry Integration	alert-metric-ingestion.json	Monitors the MetricPipeline. Triggers if metric data stops flowing to SAP Cloud Logging.

5. After importing, edit the monitor to attach your notification channel or destination and adjust thresholds as needed.

6. Verify that the new monitor definition is listed among the SAP Cloud Logging alerts.

Use SAP Cloud Logging Dashboards

You can view logs, traces, and metrics in SAP Cloud Logging dashboards. Several dashboards come with SAP Cloud Logging, and you can import additional dashboards as needed.

Procedure

• For the status of the SAP Cloud Logging integration with the Telemetry module, import the file dashboard-status.ndjson.

• For application logs and Istio access logs: Use the preconfigured dashboard Overview.

• For traces, use the OpenSearch plugin “Observability”.

• For runtime metrics, import the file dashboard-runtime.ndjson.

• For Istio Pod metrics, import the file dashboard-istio.ndjson.