fix: Observability stack now works (#398)

* fix(stack/observability): Change tempo port: 3100 -> 3200

Changed in chart 1.21.1:

> Breaking Change * In order to be consistent with other projects and documentations, the default port has been changed from 3100 to 3200.

* fix(stack/observability): Change jaeger endpoint jaeger-collector -> jaeger

It changed somewhere between chart 3.4.1 and 4.5.0 (massive restructure)

* chore(stack/observability): Use FQDNs

This allows otelcollector configs to be used by pods in any namespace and make it easier to use Secret Operator generated TLS certs so grafana can connect to datasources securely.

* fix: This didn't get bumped at release branch time

The script doesn't look for that pattern, so it was missed.

We should consider using renovate instead of a script with special (though fragile) rules.

See: [slack thread](https://stackable-workspace.slack.com/archives/C0AUHQDF1NE/p1776949735762449)

Author: NickLarsenNZ

stacks/observability/grafana.yaml

@@ -22,14 +22,14 @@ options:
       datasources:
         - name: Loki
           type: loki
-          url: http://loki:3100
+          url: http://loki.{{ NAMESPACE }}.svc.cluster.local:3100
           access: proxy
           isDefault: false
           jsonData:
             tlsAuthWithCACert: false
         - name: Tempo
           type: tempo
-          url: http://tempo:3100
+          url: http://tempo.{{ NAMESPACE }}.svc.cluster.local:3200
           access: proxy
           isDefault: false
           jsonData:

stacks/observability/opentelemetry-collector-deployment.yaml

@@ -38,15 +38,15 @@ spec:
       debug: {}
         # verbosity: detailed
       otlp/jaeger:
-        endpoint: jaeger-collector:4317
+        endpoint: jaeger.{{ NAMESPACE }}.svc.cluster.local:4317
         tls:
           insecure: true
       otlp/tempo:
-        endpoint: tempo:4317
+        endpoint: tempo.{{ NAMESPACE }}.svc.cluster.local:4317
         tls:
           insecure: true
       otlphttp/loki:
-        endpoint: http://loki:3100/otlp
+        endpoint: http://loki.{{ NAMESPACE }}.svc.cluster.local:3100/otlp
         tls:
           insecure: true
       #     auth:

stacks/observability/opentelemetry-collector-sidecar.yaml

@@ -36,15 +36,15 @@ spec:
       debug: {}
         # verbosity: detailed
       otlp/jaeger:
-        endpoint: jaeger-collector:4317
+        endpoint: jaeger.{{ NAMESPACE }}.svc.cluster.local:4317
         tls:
           insecure: true
       otlp/tempo:
-        endpoint: tempo:4317
+        endpoint: tempo.{{ NAMESPACE }}.svc.cluster.local:4317
         tls:
           insecure: true
       otlphttp/loki:
-        endpoint: http://loki:3100/otlp
+        endpoint: http://loki.{{ NAMESPACE }}.svc.cluster.local:3100/otlp
         tls:
           insecure: true
       #     auth:

stacks/stacks-v2.yaml

@@ -104,7 +104,8 @@ stacks:
         default: 3.4.0
       - name: stackableReleaseVersion
         description: The Stackable release to be used for the OpenSearch Dashboards image tag
-        default: 0.0.0-dev
+        # TODO (@NickLarsenNZ): Consider using renovate to bump the SDP versions at release branch time.
+        default: 26.3.0
   observability:
     description: >-
       An observability stack with auto-injection of the opentelemetry-collector sidecar to receive traces/logs/metrics via OTLP, and send them to Jaeger/Tempo/Loki.
@@ -688,7 +689,8 @@ stacks:
         default: 3.4.0
       - name: stackableReleaseVersion
         description: The Stackable release to be used for the OpenSearch Dashboards image tag
-        default: 0.0.0-dev
+        # TODO (@NickLarsenNZ): Consider using renovate to bump the SDP versions at release branch time.
+        default: 26.3.0
       - name: jupyterLabToken
         description: Token for JupyterLab UI access
         default: adminadmin