Skip to content

integrate-with-your-otlp-backend-e726417.md

Latest commit

 

History

History
213 lines (164 loc) · 6.93 KB

integrate-with-your-otlp-backend-e726417.md

File metadata and controls

213 lines (164 loc) · 6.93 KB

Integrate With Your OTLP Backend

Define the backend destination for your telemetry data by configuring the OTLP output, including the protocol and authentication method.

Overview

In every pipeline, you must configure an output section, which defines the destination for your telemetry data using OTLP. This is where you specify your chosen observability backend (see OpenTelemetry: Vendors).

Note:

Each pipeline resource supports exactly one backend. However, you can send specific inputs to different backends by setting up designated pipelines. For details, see Route Specific Inputs to Different Backends.

Specify the OTLP Endpoint

For the minimal pipeline configuration, you only specify an OTLP endpoint; though it's recommended that you use authentication.

...
  output:
    otlp:
      endpoint:
        value: https://backend.example.com:4317

Choose a Protocol

The default protocol for shipping the data to a backend is gRPC. If your backend requires the HTTP protocol instead, set the protocol attribute to http. Based on this setting, the gateway chooses the exporter: otlp_grpc for gRPC (the default) or otlp_http for HTTP.

Ensure the port in your endpoint URL is correct for the chosen protocol.

...
 output:
    otlp:
      protocol: http
      endpoint:
        value: https://backend.example.com:4318

Configure Compression

By default, the Telemetry gateway uses gzip compression when exporting data. To use a different compression algorithm, set the compression attribute to one of the supported values: gzip, snappy, zstd, or none (to disable compression explicitly).

...
  output:
    otlp:
      compression: none
      endpoint:
        value: https://backend.example.com:4317

Set Up Authentication

For each pipeline, add authentication details (like user names, passwords, certificates, or tokens) to connect securely to your observability backend. You can use mutual TLS (mTLS), custom headers, OAuth2, or Basic Authentication.

While you can choose to add your authentication details from plain text, it’s recommended to store these sensitive details in a Kubernetes Secret and reference the Secret's keys in your pipeline configuration. When you rotate the Secret and update its values, Telemetry Manager detects the changes and applies the new Secret to your setup.

Tip:

If you use a Secret owned by the SAP BTP Service Operator, you can configure an automated rotation policy with a specific rotation frequency and don’t have to intervene manually.

  • To use client certificates for mTLS, configure the tls section with your public certificate and private key.

      ...
  output:
    otlp:
      endpoint:
        value: https://backend.example.com/otlp:4317
      tls:
        cert:
          valueFrom:
            secretKeyRef:
              name: backend
              namespace: default
              key: cert
        key:
          valueFrom:
            secretKeyRef:
              name: backend
              namespace: default
              key: key

  • To send an authentication token (such as a bearer token) in an HTTP header, configure the headers section.

      ...
  output:
    otlp:
      endpoint:
        value: https://backend.example.com:4317
      headers:
      - name: Authorization
        prefix: Bearer
        valueFrom:
          secretKeyRef:
              name: backend
              namespace: default
              key: token

  • To use OAuth2 for authentication, configure the authentication.oauth2 section.

      ...
  output:
    otlp:
      endpoint:
        valueFrom:
          secretKeyRef:
              name: backend
              namespace: default
              key: endpoint
      authentication:
        oauth2:
          clientId:
            valueFrom:
              secretKeyRef:
                name: backend
                namespace: default
                key: clientId
          clientSecret:
            valueFrom:
              secretKeyRef:
                name: backend
                namespace: default
                key: clientSecret
          tokenUrl:
            valueFrom:
              secretKeyRef:
                name: backend
                namespace: default
                key: tokenUrl

    Note:

    If you want to use OAuth2 with gRPC, you must configure TLS for your backend connection as well.

  • To use a username and password for authentication, configure the authentication.basic section.

      ...
  output:
    otlp:
      endpoint:
        valueFrom:
          secretKeyRef:
              name: backend
              namespace: default
              key: endpoint
      authentication:
        basic:
          user:
            valueFrom:
              secretKeyRef:
                name: backend
                namespace: default
                key: user
          password:
            valueFrom:
              secretKeyRef:
                name: backend
                namespace: default
                key: password

  • If you want to configure authentication details from plain text, use the following pattern. The example shows mTLS, but you can also use Basic Authentication or custom headers:

      ...
  output:
    otlp:
      endpoint:
        value: https://backend.example.com/otlp:4317
      tls:
        cert:
          value: |
            -----BEGIN CERTIFICATE-----
            ...
        key:
          value: |
            -----BEGIN RSA PRIVATE KEY-----
            ...

Note:

If your backend is running inside the cluster and is part of the Istio service mesh, the gateways automatically secure the connection and you don't have to configure the authentication block. For details, see Sending Data to In-Cluster Backends.