Wire up self telemetry OTLP export for TargetAllocator

[pirgeo]

Description:
Allow the TA to export it's selfmonitoring data via OTLP.

The meat of the changes is in the TA binary at cmd/otel-allocator/main.go, everything else is wiring through configuration. Configuration follows the same config structure as the OTel collector selfmonitoring.

Link to tracking Issue(s):

• Resolves: [TargetAllocator] Support exporting self-telemetry metrics via OTLP #5047

Testing:
Added tests in internal/manifests/targetallocator/configmap_test.go to ensure config is correct.

Documentation:
Added docs in docs/api/opentelemetrycollectors.md.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: odubajDT / name: odubajDT (20275b9, 46ea73b)
• ✅ login: pirgeo / name: Georg P (4b3bdc7, 539d950, 585b1aa, 62ffb32, 8783e50, 9859730, a31dba7, de99e62, f87a4c0)

[codecov]

Codecov Report

❌ Patch coverage is 78.03030% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.29%. Comparing base (d9fc6e9) to head (87b320d).
⚠️ Report is 18 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
cmd/otel-allocator/main.go	72.72%	21 Missing and 3 partials ⚠️
cmd/otel-allocator/internal/server/server.go	50.00%	4 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5068      +/-   ##
==========================================
+ Coverage   63.15%   63.29%   +0.13%     
==========================================
  Files         217      217              
  Lines       15311    15440     +129     
==========================================
+ Hits         9670     9773     +103     
- Misses       4962     4984      +22     
- Partials      679      683       +4     

Flag	Coverage Δ
integration	69.06% <74.33%> (+0.32%)	⬆️
unittests	63.29% <78.03%> (+0.13%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[odubajDT]

Rebased this onto main (it had drifted and was conflicting) and pushed a few changes on top. Most importantly, I took a run at the open question about the Prometheus service-discovery metrics.

Prometheus SD/registry metrics now covered

@swiatekm - your concern was that the TA's SD metrics (and the Go/process collectors) are registered directly on the Prometheus registry, bypassing the OTel SDK, so an OTLP exporter wired only to the SDK would expose a different metric set than /metrics.

This is now handled with the Prometheus bridge (go.opentelemetry.io/contrib/bridges/prometheus), without maintaining two divergent metric sets:

• The SDK's own metrics are exported to Prometheus through a dedicated registry instead of the default one.
• The OTLP periodic reader gets a prometheusbridge.NewMetricProducer() over the default registry (SD internals + Go/process collectors). Since the SDK metrics no longer live on the default registry, the bridge does not double-count them - the reader collects SDK metrics natively and the Prometheus-only metrics via the bridge.
• /metrics now serves the union of both registries, so its output is unchanged.

Net effect: OTLP export and /metrics expose the same set. I'd still be happy to move the SD metrics onto the SDK natively in a follow-up if you prefer that direction long-term, but this makes the feature complete now rather than shipping a partial metric set.

On config shape/otelconf

I kept the telemetry.metrics.otlp block from the issue proposal for now. You'd raised possibly standardizing on otelconf in the CRDs - happy to go that way if the approvers prefer it; it's a larger surface change, so I left it out of this PR unless there's consensus to pursue it.

Verification

Deployed the tiered Prometheus-large-scale setup on a local kind cluster with the TA pushing OTLP to a self-monitoring collector. Confirmed the TA's own metrics, the bridged SD/Go/process metrics, and the collector self-telemetry all arrive and that the customer scrape path is unaffected.

PTAL when you get a chance. Thank you!

[swiatekm]

Thank you for working on this!

I think it's going to be much easier to review and merge these changes if we decouple the target allocator feature from the CRD additions.

In plain target allocator configuration, we should be able to just embed https://pkg.go.dev/go.opentelemetry.io/contrib/otelconf into the config file - the fact that parts of the spec may be experimental is not an obstruction here.

On the other hand, in the CRD, we have to be more careful, as we're not allowed to make any breaking changes without bumping the version. The problem of embedding the declarative sdk config in the CRD also came up in @pavolloffay's work on the Instrumentation CRD, so we should align that bit.

[swiatekm]

We should move all the telemetry related setup to a separate telemetry.go file.

[odubajDT]

Thanks @swiatekm, that makes sense. I've split the work along the lines you suggested:

1. The target-allocator feature and the CRD additions are now two independent changes int two separate PRs feat(target allocator): export self-telemetry via OTLP #5294 feat(target allocator): add OTLP self-telemetry API to CRDs #5293

2. otelconf in the plain TA config - Agreed in principle, and I want to flag one wrinkle before I wire it up. The TA also has metrics registered directly on the Prometheus registry that bypass the OTel SDK. To avoid exposing a different metric set over OTLP than over /metrics, I bridge those into the OTLP export via contrib/bridges/prometheus. otelconf's SDK builder doesn't provide a way to attach a sdkmetric.Producer to the readers it constructs, so if I hand reader construction entirely to otelconf I'd lose that bridge and end up with the two divergent metric sets you mentioned wanting to avoid. My plan is therefore to use the otelconf config types for the config-file schema but keep a thin custom meter-provider build so the Prometheus bridge stays. Does that split sound right to you, or would you rather see full otelconf.NewSDK with the SD-metrics-into-SDK move handled as the separate follow-up we discussed?

3. For the CRD I kept narrow typed fields (endpoint/protocol/temporality/headers/tls/interval/timeout), which mirrors how the Instrumentation CRD models its exporter config rather than embedding a declarative-config blob. The additions are purely additive and only on the latest version of each CRD (the v1beta1 OpenTelemetryCollector storage version, and the v1alpha1 standalone TargetAllocator, which has no other version), so there's no breaking change / version bump. Happy to align with whatever comes out of @pavolloffay's Instrumentation work if the direction there is to embed the declarative SDK config instead - just point me at the pattern you'd like to standardize on.