Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
8c9288b
feat(charts): add content digest for the embedded talm library
lexfrei Jun 5, 2026
dbc2a57
feat(commands): detect vendored charts/talm drift by content
lexfrei Jun 5, 2026
45cafe5
feat(version): warn when vendored charts drift from the talm binary
lexfrei Jun 5, 2026
ad56f0e
docs: document chart vendoring and drift detection
lexfrei Jun 5, 2026
5e9dfc2
feat(version): detect preset-template drift via .talm-preset.lock
lexfrei Jun 5, 2026
6af1a7b
fix(charts): anchor Chart.yaml metadata normalizer to top-level keys
lexfrei Jun 12, 2026
7d564d6
fix(cli): align --strict-charts help with the actual re-sync command
lexfrei Jun 12, 2026
9669a55
docs(manual-test-plan): clear simulated drift with the project's own …
lexfrei Jun 12, 2026
957c873
test(commands): pin malformed .talm-preset.lock failure modes
lexfrei Jun 12, 2026
31b6fdb
fix(init): validate inferred preset before --update writes anything
lexfrei Jun 12, 2026
e28d34e
fix(commands): treat CRLF line endings as checkout artifacts, not drift
lexfrei Jun 12, 2026
3dc254a
fix(cli): make strict mode block on unverifiable or missing drift bas…
lexfrei Jun 12, 2026
6031d46
fix(init,commands): make init --update an exact re-sync; name drifted…
lexfrei Jun 12, 2026
a44d621
fix(build): classify describe-suffix and dirty builds as non-release
lexfrei Jun 12, 2026
edb7f84
test(cli): pin the Chart.yaml strictCharts channel and the strict-sou…
lexfrei Jun 12, 2026
a49ca47
docs(readme): show .talm-preset.lock in the project tree
lexfrei Jun 12, 2026
170cdbe
docs(manual-test-plan): note that --force bypasses the edit-preservin…
lexfrei Jun 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
VERSION=$(shell git describe --tags)
# Full describe provenance for display: a clean exact tag yields the release
# version; any other state carries the describe suffix and/or -dirty marker,
# which releaseVersion in main.go classifies as a non-release build — so
# release-only behavior (the chart-drift checks) stays off for WIP builds,
# whose embedded charts are a moving target, while `talm --version` still
# identifies the build in bug reports.
VERSION=$(shell git describe --tags --dirty 2>/dev/null || echo dev)
TALOS_VERSION=$(shell go list -m github.com/siderolabs/talos | awk '{sub(/^v/, "", $$NF); print $$NF}')

build:
Expand Down
53 changes: 53 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,59 @@ talm template -f nodes/node1.yaml -I
>
> `talm template -f node.yaml` (with or without `-I`) does **not** apply the same overlay: its output is the rendered template plus the modeline and the auto-generated warning, byte-identical to what the template alone would produce. Routing it through the patcher would drop every YAML comment (including the modeline) and re-sort keys, breaking downstream commands that read the file back. Use `apply --dry-run` if you want to preview the exact bytes that will be sent to the node.

## Keeping charts in sync after a binary upgrade

`talm init` **vendors** its preset and library charts into the project directory — the preset templates plus a copy of the talm library chart under `charts/talm/`:

```text
mycluster/
├── Chart.yaml
├── values.yaml
├── .talm-preset.lock # pinned preset baseline — commit it (see "Preset drift")
├── templates/ # preset templates — you own and edit these
└── charts/
└── talm/ # talm library chart — vendored from the binary
├── Chart.yaml
└── templates/_helpers.tpl
```

Render commands (`template`, `apply`, `upgrade`) read this **local** copy, never the binary's built-in charts. That makes a project self-contained and reproducible — but it also means upgrading the `talm` binary does not touch `charts/talm/`. The vendored library stays frozen at whatever version last ran `init`, so a binary upgrade can leave you rendering with stale chart logic.

Re-sync the vendored library with `talm init --update`:

```bash
talm init --update --preset <your-preset>
```

This refreshes `charts/talm/` (always) and offers to update the preset templates (interactively, since you may have edited them). Your `values.yaml`, secrets, and node files are left untouched.

To catch drift automatically, a release build compares the vendored `charts/talm/` against its own built-in copy on every config-loading command. The comparison is by **content**, not version number — re-vendoring after a binary bump that did not change the library is a no-op and raises no warning. When the content genuinely differs, talm prints a non-fatal warning to stderr (stdout and the exit code are unchanged):

```text
WARN: project's vendored charts/talm/ library differs from the copy built into talm <version> (modified: templates/_helpers.tpl); run `talm init --update --preset <preset>` to re-sync (or ignore if this is intentional)
```

The remediation needs the preset name because `talm init --update` resolves the preset from `Chart.yaml`, which an init'd project does not record — pass `--preset <your-preset>` (the one you ran `talm init` with) explicitly.

Teams that want this enforced can turn the warning into a hard error (exit 1): set `strictCharts: true` in `Chart.yaml` so the whole team and CI inherit it, or pass `--strict-charts` for a single run. Strict mode applies to every config-loading command, including read-only ones such as `talm get` — run `talm init --update --preset <preset>`, or drop the flag / unset `strictCharts`, to unblock. Strict mode also escalates a check that cannot run at all — an unreadable `charts/talm/` or a corrupted `.talm-preset.lock` — into the same hard error, where the default behaviour degrades it to a `WARN: could not check drift` line: an unverifiable baseline passing silently would defeat the enforcement. A *missing* baseline (no `charts/talm/`, no `.talm-preset.lock`) blocks under strict for the same reason — deleting the baseline must not be a quieter bypass than corrupting it — while staying silent without strict, so projects generated before baseline pinning are not nagged. The check stays silent for `dev`/source builds, whose embedded charts are a moving target the developer controls.

### Preset drift

The vendored library (`charts/talm/`) is not the whole story. `talm init` also copies the **preset** — the `templates/` that render your machine config (sysctls, etcd args, the cozystack opinions) — into the project, and those you are *expected* to edit. That makes content comparison the wrong tool: it would flag every legitimate customization. So the preset is tracked differently. At `init` (and `init --update`) time talm pins the hash of the preset **as shipped** into `.talm-preset.lock`:

```yaml
preset: cozystack
presetHash: <hash of the preset built into the binary at init time>
```

A release build then compares the binary's *current* preset hash against that pinned baseline — never against your edited `templates/` — so operator customizations are never reported as drift. When a newer binary ships changed preset defaults, the baseline no longer matches and talm warns:

```text
WARN: project's cozystack preset differs from the copy built into talm <version>; run `talm init --update --preset cozystack` to pull the new preset defaults (your templates/ edits are preserved via the interactive diff)
```

`talm init --update --preset <preset>` shows you an interactive diff of the new preset against your `templates/`, lets you merge what you want, and advances the baseline — which clears the warning even if you decline individual diffs to keep your customizations. `--strict-charts` / `strictCharts: true` escalate this to a hard error exactly as for the library. Projects with no `.talm-preset.lock` (generated before preset pinning) stay silent — there is no baseline to compare — unless strict mode is on, which treats a missing baseline as a blocker. Commit `.talm-preset.lock` so the baseline is shared across your team.

## Apply with side-patches

`talm apply -f` accepts a chain of files. The FIRST `-f` is the **anchor** — it must carry a `# talm: nodes=[…], templates=[…]` modeline and live under a `talm init`'d project (Chart.yaml + secrets.yaml). Any subsequent `-f` files are **side-patches**: they are merged in order on top of the anchor's rendered config, and a single `ApplyConfiguration` is issued per node carrying the composed result.
Expand Down
106 changes: 99 additions & 7 deletions charts/charts.go
Original file line number Diff line number Diff line change
@@ -1,25 +1,60 @@
package charts

import (
"crypto/sha256"
"embed"
"encoding/hex"
"fmt"
"io/fs"
"path"
"regexp"
"sort"
"strings"

"github.com/cockroachdb/errors"
)

const presetGenericName = "generic"

// talmLibraryName is the directory of the talm library chart inside the
// embedded tree (and the name it is vendored under at charts/talm/ in a
// project). It is talm-owned — unlike the preset templates, which the
// operator edits — so it is the only tree the drift check compares.
const talmLibraryName = "talm"

// chartYamlName is the conventional Helm chart metadata filename.
const chartYamlName = "Chart.yaml"

//go:embed all:cozystack all:generic all:talm
var embeddedCharts embed.FS

// chartMetaRegex matches the TOP-LEVEL `name:`/`version:` metadata lines of
// a Chart.yaml. Both are rewritten to a %s placeholder so the init flow can
// substitute the real project name/version, and so the drift check can
// compare two chart trees without a version stamp counting as content. The
// `(?m)^` anchor keeps nested occurrences (dependencies[].name/version,
// maintainers[].name) intact: rewriting those would mask real dependency
// drift and inject extra verbs into the two-argument fmt template the init
// flow writes Chart.yaml with.
var chartMetaRegex = regexp.MustCompile(`(?m)^(name|version): \S+`)

// NormalizeChartMeta rewrites the name/version lines of a Chart.yaml to %s
// placeholders. Files other than Chart.yaml pass through unchanged. base is
// the file's base name (e.g. path.Base(p)). Keeping a single normalizer
// means the init-time substitution and the content-drift comparison treat
// chart metadata identically.
func NormalizeChartMeta(base, content string) string {
if base != chartYamlName {
return content
}

return chartMetaRegex.ReplaceAllString(content, "$1: %s")
}

// PresetFiles returns a map of file paths to their contents.
// For Chart.yaml files, name and version are replaced with %s placeholders.
func PresetFiles() (map[string]string, error) {
filesMap := make(map[string]string)
regex := regexp.MustCompile(`(name|version): \S+`)

err := fs.WalkDir(embeddedCharts, ".", func(filePath string, entry fs.DirEntry, err error) error {
if err != nil {
Expand Down Expand Up @@ -47,12 +82,8 @@ func PresetFiles() (map[string]string, error) {
return errors.Wrapf(err, "reading embedded chart file %q", filePath)
}

content := string(data)

// For Chart.yaml files, replace name and version with %s
if path.Base(filePath) == "Chart.yaml" {
content = regex.ReplaceAllString(content, "$1: %s")
}
// For Chart.yaml files, replace name and version with %s.
content := NormalizeChartMeta(path.Base(filePath), string(data))

// Use the file path as-is (relative to charts directory)
filesMap[filePath] = content
Expand All @@ -66,6 +97,67 @@ func PresetFiles() (map[string]string, error) {
return filesMap, nil
}

// TalmLibraryFiles returns the embedded talm library chart keyed by path
// relative to the talm/ root (e.g. "Chart.yaml", "templates/_helpers.tpl"),
// so the keys line up with a project's vendored charts/talm/ tree. Chart.yaml
// metadata is normalized via NormalizeChartMeta, so a version stamp is not
// treated as content. It is the embedded counterpart compared against the
// vendored copy to surface chart drift after a binary upgrade.
func TalmLibraryFiles() (map[string]string, error) {
filesMap := make(map[string]string)

err := fs.WalkDir(embeddedCharts, talmLibraryName, func(filePath string, entry fs.DirEntry, err error) error {
if err != nil {
return errors.Wrapf(err, "walking embedded talm library at %q", filePath)
}

if entry.IsDir() {
return nil
}

data, err := embeddedCharts.ReadFile(filePath)
if err != nil {
return errors.Wrapf(err, "reading embedded talm file %q", filePath)
}

// Strip the talm/ prefix so keys match a vendored charts/talm/ tree.
rel := strings.TrimPrefix(filePath, talmLibraryName+"/")
filesMap[rel] = NormalizeChartMeta(path.Base(filePath), string(data))

return nil
})
if err != nil {
return nil, errors.Wrap(err, "collecting embedded talm library")
}

return filesMap, nil
}

// HashChartFiles returns a deterministic digest of a chart tree described as
// a path→content map. The digest folds in the sorted set of (path, content)
// pairs and is independent of map iteration order. Each path and content is
// length-prefixed so distinct trees cannot collide by a fortunate alignment
// of concatenated bytes. Two trees hash equal iff they carry the same files
// with the same bytes — the signal the drift check relies on.
func HashChartFiles(files map[string]string) string {
paths := make([]string, 0, len(files))
for p := range files {
paths = append(paths, p)
}

sort.Strings(paths)

hasher := sha256.New()
for _, p := range paths {
// Length-prefix both the path and its content so the boundary
// between them (and between successive entries) is unambiguous.
fmt.Fprintf(hasher, "%d:%s%d:", len(p), p, len(files[p]))
hasher.Write([]byte(files[p]))
}

return hex.EncodeToString(hasher.Sum(nil))
}

// AvailablePresets returns a list of available preset chart names.
// The presetGenericName preset is always first if it exists.
func AvailablePresets() ([]string, error) {
Expand Down
Loading
Loading