fix: Observability stack now works (#399)

* fix(stack/observability): Change tempo port: 3100 -> 3200

Changed in chart 1.21.1:

> Breaking Change * In order to be consistent with other projects and documentations, the default port has been changed from 3100 to 3200.

* fix(stack/observability): Change jaeger endpoint jaeger-collector -> jaeger

It changed somewhere between chart 3.4.1 and 4.5.0 (massive restructure)

* chore(stack/observability): Use FQDNs

This allows otelcollector configs to be used by pods in any namespace and make it easier to use Secret Operator generated TLS certs so grafana can connect to datasources securely.

Author: NickLarsenNZ

stacks/observability/grafana.yaml

@@ -30,14 +30,14 @@ options:
       datasources:
         - name: Loki
           type: loki
-          url: http://loki:3100
+          url: http://loki.{{ NAMESPACE }}.svc.cluster.local:3100
           access: proxy
           isDefault: false
           jsonData:
             tlsAuthWithCACert: false
         - name: Tempo
           type: tempo
-          url: http://tempo:3100
+          url: http://tempo.{{ NAMESPACE }}.svc.cluster.local:3200
           access: proxy
           isDefault: false
           jsonData:

stacks/observability/opentelemetry-collector-deployment.yaml

@@ -38,15 +38,15 @@ spec:
       debug: {}
         # verbosity: detailed
       otlp/jaeger:
-        endpoint: jaeger-collector:4317
+        endpoint: jaeger.{{ NAMESPACE }}.svc.cluster.local:4317
         tls:
           insecure: true
       otlp/tempo:
-        endpoint: tempo:4317
+        endpoint: tempo.{{ NAMESPACE }}.svc.cluster.local:4317
         tls:
           insecure: true
       otlphttp/loki:
-        endpoint: http://loki:3100/otlp
+        endpoint: http://loki.{{ NAMESPACE }}.svc.cluster.local:3100/otlp
         tls:
           insecure: true
       #     auth:

stacks/observability/opentelemetry-collector-sidecar.yaml

@@ -36,15 +36,15 @@ spec:
       debug: {}
         # verbosity: detailed
       otlp/jaeger:
-        endpoint: jaeger-collector:4317
+        endpoint: jaeger.{{ NAMESPACE }}.svc.cluster.local:4317
         tls:
           insecure: true
       otlp/tempo:
-        endpoint: tempo:4317
+        endpoint: tempo.{{ NAMESPACE }}.svc.cluster.local:4317
         tls:
           insecure: true
       otlphttp/loki:
-        endpoint: http://loki:3100/otlp
+        endpoint: http://loki.{{ NAMESPACE }}.svc.cluster.local:3100/otlp
         tls:
           insecure: true
       #     auth: