Skip to content

Standalone mode for OpAMP k8s bridge#4986

Merged
jaronoff97 merged 3 commits into
open-telemetry:mainfrom
owais:standalone-mode
Jun 19, 2026
Merged

Standalone mode for OpAMP k8s bridge#4986
jaronoff97 merged 3 commits into
open-telemetry:mainfrom
owais:standalone-mode

Conversation

@owais

@owais owais commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

Description:
This PR implements a basic standlone mode for the OpAMP bridge that works directly on collector configmap(s) instead of operator CRDs. It does not depend on the operator. The ConfigApplier interface almost already satisfies everything we need with some minor tweaks. The change is mostly a new implementation of the ConfigApplier interface that works directly with configmaps.

It watches configmaps with a specific label and reports them to the OpAMP server. Users can then remotely manage that config from the server.

Link to tracking Issue(s):

How to review:

  • internal/agent & internal/operator packages moves things around a bit to allow the new standalone mode to implement the existing Config Applier interface. No new features or changes to existing features in these packages.
  • internal/standalone contains all of the new feature code. It implements a standalone client and an implementation of the ConfigApplier interface.
  • config/standalone-bridge contains the k8s manifests used to deploy the bridge in standlone mode.

Testing:

  • Tests added for the new implementation & updated existing tests where needed
  • Manually tested in a local kind cluster

@owais owais force-pushed the standalone-mode branch 2 times, most recently from 50b5f7b to 3464209 Compare April 22, 2026 19:12
@github-actions

github-actions Bot commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

E2E Test Results

 37 files  235 suites   2h 20m 57s ⏱️
102 tests 100 ✅ 1 💤 1 ❌
257 runs  251 ✅ 2 💤 4 ❌

Results for commit 704ea45.

♻️ This comment has been updated with latest results.

@owais owais force-pushed the standalone-mode branch 7 times, most recently from 3259be0 to 57e3079 Compare April 27, 2026 19:26
@owais

owais commented Apr 27, 2026

Copy link
Copy Markdown
Contributor Author

The failing VULN does not affect us. It only affect plugin installs when running docker as a daemon which we do not. We only use it indirectly as an API.

more details: GHSA-pxq6-2prw-chj9

Patching is not as simple as docker has deprecated the github.com/docker/docker package and replaced it with multiple smaller packages in github.com/moby/moby/*; and we don't use the docker package directly. The call path is:

go: github.com/open-telemetry/opentelemetry-operator/apis/v1beta1 imports
	go.opentelemetry.io/contrib/otelconf/v0.3.0 imports
	go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc imports
	google.golang.org/grpc imports
	google.golang.org/grpc/balancer/roundrobin imports
	google.golang.org/grpc/balancer/endpointsharding tested by
	google.golang.org/grpc/balancer/endpointsharding.test imports
	google.golang.org/grpc/internal/testutils/roundrobin imports
	
go: github.com/open-telemetry/opentelemetry-operator/cmd/otel-allocator/internal/watcher imports
	github.com/prometheus-operator/prometheus-operator/pkg/operator imports
	github.com/prometheus/prometheus/model/rulefmt imports
	github.com/prometheus/prometheus/promql tested by
	github.com/prometheus/prometheus/promql.test imports
	github.com/prometheus/prometheus/tsdb imports
	github.com/prometheus/prometheus/tsdb/index imports
	github.com/bboreham/go-loser: github.com/docker/docker@v29.3.1+incompatible: reading github.com/docker/docker/go.mod at revision v29.3.1: unknown revision v29.3.1
	
go: github.com/open-telemetry/opentelemetry-operator/cmd/otel-allocator/internal/config imports
	github.com/prometheus/prometheus/discovery/install imports
	github.com/prometheus/prometheus/discovery/scaleway imports
	github.com/scaleway/scaleway-sdk-go/api/instance/v1 tested by
	github.com/scaleway/scaleway-sdk-go/api/instance/v1.test imports
	github.com/scaleway/scaleway-sdk-go/internal/testhelpers/httprecorder imports
	github.com/scaleway/scaleway-sdk-go/vcr imports
	gopkg.in/dnaeon/go-vcr.v4/pkg/cassette: github.com/docker/docker@v29.3.1+incompatible: reading github.com/docker/docker/go.mod at revision v29.3.1: unknown revision v29.3.1
	
	
go: github.com/open-telemetry/opentelemetry-operator/cmd/otel-allocator/internal/config imports
	github.com/prometheus/prometheus/discovery/install imports
	github.com/prometheus/prometheus/discovery/gce imports
	google.golang.org/api/compute/v1 imports
	google.golang.org/api/internal/gensupport imports
	github.com/googleapis/gax-go/v2/callctx tested by
	github.com/googleapis/gax-go/v2/callctx.test imports
	google.golang.org/genproto/googleapis/type/color: github.com/docker/docker@v29.3.1+incompatible: reading github.com/docker/docker/go.mod at revision v29.3.1: unknown revision v29.3.1

So each one of those packages will need to migrate to moby to pick up the security patch. I suggest we ignore this vuln as it is essentially a false positive for us and upgrading to a fixed moby version means waiting on close to a dozen OSS packages to do it.

migration details: https://github.com/moby/moby/releases/tag/docker-v29.0.0

@owais owais force-pushed the standalone-mode branch 2 times, most recently from 2e90995 to f2c5727 Compare April 27, 2026 20:23
@owais owais marked this pull request as ready for review April 27, 2026 20:23
@owais owais requested a review from a team as a code owner April 27, 2026 20:23
@owais owais force-pushed the standalone-mode branch from f2c5727 to a4c54d2 Compare May 4, 2026 18:21
@jaronoff97

Copy link
Copy Markdown
Contributor

this is top on my review list, thank you very much for the contribution here and apologies for the delay, i was on PTO.

@jaronoff97 jaronoff97 self-requested a review May 5, 2026 16:48
@owais owais force-pushed the standalone-mode branch 3 times, most recently from 9cbc35f to 054483e Compare May 5, 2026 19:50

@jaronoff97 jaronoff97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some requested changes, but overall i would like to see some better test coverage as well as some more thought around the code boundaries. definitely in favor of this, but there's a few open questions I'd like to hear more about.

Comment thread cmd/operator-opamp-bridge/internal/agent/agent.go
Comment thread cmd/operator-opamp-bridge/internal/agent/kube_resource_key.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/agent/kube_resource_key.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/agent/kube_resource_key.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/config/config.go
Comment thread cmd/operator-opamp-bridge/internal/standalone/client.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/standalone/client.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/standalone/client.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/standalone/schema.go Outdated
Comment thread cmd/operator-opamp-bridge/manifests/standalone/configmap.yaml
@owais owais force-pushed the standalone-mode branch from 054483e to e2ede27 Compare May 6, 2026 22:03
@owais

owais commented May 6, 2026

Copy link
Copy Markdown
Contributor Author

Addressed some things. Will add more e2e tests.

@owais owais force-pushed the standalone-mode branch 5 times, most recently from edc4894 to fec3816 Compare May 7, 2026 06:01
@owais owais force-pushed the standalone-mode branch 4 times, most recently from aec863c to b95651f Compare June 2, 2026 18:16
Comment thread cmd/operator-opamp-bridge/main.go Outdated

@jaronoff97 jaronoff97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking much better, a few comments and questions but the shape of this is much better.

Comment thread cmd/operator-opamp-bridge/internal/config/config.go
Comment thread cmd/operator-opamp-bridge/internal/config/config.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/manager/manager.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/operator/client.go
Comment thread cmd/operator-opamp-bridge/internal/standalone/manager_opts.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/operatorbridge/manager_opts.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/standalone/validate.go
Comment thread cmd/operator-opamp-bridge/main.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/config/config.go Outdated
@owais

owais commented Jun 14, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @jaronoff97 made the requested changes. PTAL and let me know if anything else is needed.

@jaronoff97 jaronoff97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking much much better only a couple more conventional go things.

Comment thread cmd/operator-opamp-bridge/internal/standalone/manager_opts.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/config/config.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/manager/manager.go
Comment thread cmd/operator-opamp-bridge/internal/standalone/validate.go
@owais

owais commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @jaronoff97. Appreciate the quick review. PTAL when you can.

Comment thread cmd/operator-opamp-bridge/internal/config/config.go Outdated

@jaronoff97 jaronoff97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only a single minor comment, but otherwise this is good. thank you very much for enduring the long back and forth here – i'm happy with where we've ended up here. I'm going to ask for one more maintainer to review before we merge.

@codecov

codecov Bot commented Jun 15, 2026

Copy link
Copy Markdown

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
104 2 102 2
View the top 2 failed test(s) by shortest run time
::privilege-escalation-delta
Stack Traces | 0.882s run time
($error != null): Invalid value: true: Expected value: false
::opampbridge-standalone
Stack Traces | 45.5s run time
signal: killed

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@owais

owais commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

@jaronoff97 Thanks for all the time you invested and for the thorough review. I have a few improvements/simplifications lined up that I think you'll like.

@swiatekm swiatekm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic and content of this change makes sense to me. I'd like to try to give it a more narrow scope, so it's not +4k lines mixing different concerns.

  • Could we move the CRD modifications out into a follow-up? It's not even clear to me if those are valuable, considering the whole point is to decouple from the operator.
  • We're not running the E2E tests in CI. Either they should be run, or they should be removed from this PR and added in a follow-up.
  • I'd rather move the manifests in config/ out as well. These are something like a public API in the sense that users might just kubectl -k them, and so should require more careful review.
  • On a different note, I think this PR could use more docstrings for the functions it adds. It would make it easier to review, if nothing else.

Also @dmitryax, you're the author of the issue, could you have a look and see if this does what you'd envisioned?

Comment thread tests/e2e-opampbridge/standalone/00-assert.yaml
Comment thread cmd/operator-opamp-bridge/internal/agent/agent.go
@owais

owais commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

I'd rather move the manifests in config/ out as well. These are something like a public API in the sense that users might just kubectl -k them, and so should require more careful review.

@swiatekm They're very useful during development so I've moved them to a new dev/ dir for now. I can circle back with another PR after this is merged to add public version of these to config like we have for other things.

@owais

owais commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

Could we move the CRD modifications out into a follow-up? It's not even clear to me if those are valuable, considering the whole point is to decouple from the operator.

I've removed them for now. You're right they're not very useful right now. I think we might need them in the future. I'll add them back in another PR if needed/requested.

@owais

owais commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

@swiatekm PTAL when you can. Thanks.

@swiatekm

Copy link
Copy Markdown
Contributor

I'd rather move the manifests in config/ out as well. These are something like a public API in the sense that users might just kubectl -k them, and so should require more careful review.

@swiatekm They're very useful during development so I've moved them to a new dev/ dir for now. I can circle back with another PR after this is merged to add public version of these to config like we have for other things.

How about moving them to cmd/operator-opamp-bridge/manifests/standalone and adding a small README.md saying that there are dev manifests not intended for production usage?

@owais

owais commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

@swiatekm moved to cmd and added readme

@swiatekm swiatekm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the copious use of context.Background, but this is something the codebase unfortunately already does, so I'm not going to ask for it to be fixed. I highlighted two spots where it should be fixed in this PR, but we should do a follow-up where we thread context through all the methods that need it @jaronoff97.

Comment thread cmd/operator-opamp-bridge/internal/healthcheck/server_test.go Outdated
Comment thread cmd/operator-opamp-bridge/internal/manager/manager.go Outdated
@owais

owais commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

@swiatekm updated the ctx usage. thanks.

@swiatekm

Copy link
Copy Markdown
Contributor

@owais I'll be happy to approve after you fix the conflict.

owais and others added 2 commits June 18, 2026 20:41
Introduces a new standalone mode that allows managing collector config from a remote server without requiring Operator CRDs.
Adds the standalone client, a plain collector instance type, Kubernetes RBAC and deployment manifests, and a --mode flag to select between operator (default) and standalone at startup.
Co-authored-by: Mikołaj Świątek <mail@mikolajswiatek.com>
@owais

owais commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

@swiatekm resolved.

@dmitryax dmitryax left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @owais!

@dmitryax

dmitryax commented Jun 18, 2026

Copy link
Copy Markdown
Member

The PR description needs to be updated. Looks like we don't support app.kubernetes.io/managed-by: opamp-bridge-standalone in this PR (and probably don't need to)

@swiatekm swiatekm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@swiatekm

Copy link
Copy Markdown
Contributor

@jaronoff97 you did the majority of the review for this PR and are the opamp bridge expert, so I'm leaving it to you to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Standalone OpAMP-to-Kubernetes binary for updating Collector configs without the operator

5 participants