Resolve env-var placeholders in Flink config.yaml#369
Open
velo wants to merge 2 commits into
Open
Conversation
Signed-off-by: Marvin Froeder <marvin@datasqrl.com>
velo
marked this pull request as ready for review
July 21, 2026 16:52
ferenc-csaky
requested changes
Jul 21, 2026
ferenc-csaky
left a comment
ferenc-csaky
left a comment
Collaborator
There was a problem hiding this comment.
Added 2 comments, but the core logic looks good.
Comment on lines
+106
to
+112
| /** | ||
| * Resolves {@code ${VAR}} placeholders in Flink configuration values from the environment, the | ||
| * same mechanism already applied to the SQL script and compiled plan. Lets secrets (e.g. the | ||
| * Iceberg maintenance lock JDBC credentials) be injected via env vars instead of landing in the | ||
| * plaintext config.yaml ConfigMap. {@link Configuration#toMap()} returns a copy, so mutating | ||
| * {@code conf} while iterating it is safe. | ||
| */ |
Collaborator
There was a problem hiding this comment.
Delete this, it's just noise, the method is trivial.
| * plaintext config.yaml ConfigMap. {@link Configuration#toMap()} returns a copy, so mutating | ||
| * {@code conf} while iterating it is safe. | ||
| */ | ||
| private void resolveEnvVars(Configuration conf) { |
Collaborator
There was a problem hiding this comment.
I'd prefer creating a new instance instead of mutating the given one. Configuration has a copy ctor.
Signed-off-by: Marvin Froeder <marvin@datasqrl.com>
ferenc-csaky
approved these changes
Jul 21, 2026
ferenc-csaky
enabled auto-merge (squash)
July 21, 2026 20:10
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
BaseRunner.initConfiguration()now runs the loadedconfig.yamlvalues through the existingEnvVarResolver(the same${VAR}mechanism already applied to the SQL script and compiled plan). This lets secrets be injected into Flink config via env vars instead of landing in the plaintextconfig.yamlConfigMap.Motivating use: Iceberg auto-maintenance wires a JDBC lock store whose credentials must not sit in a ConfigMap. cloud-compilation emits
flink-maintenance.lock.jdbc.user: ${ICEBERG_LOCK_USER}/...password: ${ICEBERG_LOCK_PASSWORD}placeholders; the flink pod mounts them from theiceberg-maintainance-locktablesecret, and this change resolves them at runtime.Configuration.toMap()returns a copy, so mutatingconfwhile iterating is safe. Values without${...}are untouched.Test
CliRunnerTest.initConfiguration_shouldResolveEnvVarPlaceholdersInConfig— asserts a${VAR}config value resolves from env and literals pass through. FullCliRunnerTestgreen (14/14, under Java 17).Related
feat/iceberg-auto-maintenance) which emits the config.🤖 Generated with Claude Code