Redesign checksum config and add migration tool

Author: kindermax

docs/docs/advanced_usage.md

@@ -91,17 +91,17 @@ But we have one downside - run `npm install` may take some time and we do not wa
 
 Checksums allow you to know when some of the files have changed and made a decision based on that.
 
-When you add `checksum` directive to a command - `lets` will calculate checksum from all of the files listed in `checksum` and put `LETS_CHECKSUM` env variable to command env.
+When you add `checksum.files` directive to a command - `lets` will calculate checksum from all of the listed files and put `LETS_CHECKSUM` env variable to command env.
 
 `LETS_CHECKSUM` will have a checksum value.
 
 We then can store this checksum somewhere in the file and check that stored checksum with a checksum from env.
 
-Fortunately, `lets` have an option for that - `persist_checksum`.
+Fortunately, `lets` have an option for that - `checksum.persist`.
 
-If `persist_cheksum` used with `checksum` `lets` will store new checksum to `.lets` dir and each time you run a command `lets` will check if stored checksum changed from the one from env.
+If `checksum.persist` is used with `checksum.files`, `lets` will store new checksum to `.lets` dir and each time you run a command `lets` will check if stored checksum changed from the one from env.
 
-While using `persist_checksum`, `lets` will add new env variable to command env - `LETS_CHECKUM_CHANGED`.
+While using `checksum.persist`, `lets` will add new env variable to command env - `LETS_CHECKSUM_CHANGED`.
 
 You can learn more about checksum in [Checksum section](config.md#checksum)
 
@@ -112,8 +112,9 @@ commands:
   build-deps:
     description: Install project dependencies
     checksum:
-      - package.json
-    persist_checksum: true
+      files:
+        - package.json
+      persist: true
     cmd: |
       if [[ ${LETS_CHECKSUM_CHANGED} == true ]]; then
         npm install
@@ -142,8 +143,9 @@ commands:
   build-deps:
     description: Install project dependencies
     checksum:
-      - package.json
-    persist_checksum: true
+      files:
+        - package.json
+      persist: true
     cmd: |
       if [[ ${LETS_CHECKSUM_CHANGED} == true ]]; then
         npm install

docs/docs/changelog.md

@@ -5,6 +5,8 @@ title: Changelog
 
 ## [Unreleased](https://github.com/lets-cli/lets/releases/tag/v0.0.X)
 
+* `[Added]` Add `checksum.files`, `checksum.sh`, and `checksum.persist` command checksum syntax while keeping the old checksum format compatible.
+* `[Added]` Add `lets self fix` config migration command with `--dry-run` preview output for deprecated checksum syntax.
 * `[Refactoring]` Use Go 1.26 `errors.AsType` for type-safe error unwrapping.
 
 ## [0.0.61](https://github.com/lets-cli/lets/releases/tag/v0.0.61)

docs/docs/cli.md

@@ -19,3 +19,5 @@ title: CLI options
 |`-v, --version`|||version for lets|
 
 Upgrade the lets binary with `lets self upgrade`.
+
+Migrate deprecated config syntax with `lets self fix`. Use `lets self fix --dry-run` to print migrated config content before writing files.

docs/docs/config.md

@@ -789,11 +789,11 @@ commands:
 
 `key: checksum`
 
-`type: array of string | mapping string => array of string`
+`type: object`
 
 Checksum used for computing file hashes. It is useful when you depend on some files content changes.
 
-In `checksum` you can specify:
+In `checksum.files` you can specify:
 
 - a list of file names
 - a list of file regexp patterns (parsed via go `path/filepath.Glob`)
@@ -814,10 +814,11 @@ If checksum is a mapping, e.g:
 commands:
   build:
     checksum:
-      deps:
-        - package.json
-      doc:
-        - Readme.md
+      files:
+        deps:
+          - package.json
+        doc:
+          - Readme.md
 ```
 
 Resulting env will be:
@@ -830,29 +831,42 @@ Checksum is calculated with `sha1`.
 
 If you specify patterns, `lets` will try to find all matches and will calculate checksum of that files.
 
+You can use `checksum.sh` instead of `checksum.files` when you need to provide the checksum value from a shell command:
+
+```yaml
+commands:
+  build:
+    checksum:
+      sh: git rev-parse HEAD
+    cmd: docker build -t myrepo/app:${LETS_CHECKSUM} .
+```
+
+`checksum.files` and `checksum.sh` are mutually exclusive.
+
 Example:
 
 ```yaml
 shell: bash
 commands:
   app-build:
     checksum:
-      - requirements-*.txt
+      files:
+        - requirements-*.txt
     cmd: |
       docker pull myrepo/app:${LETS_CHECKSUM}
       docker run --rm myrepo/app${LETS_CHECKSUM} python -m app
 ```
 
 
-### `persist_checksum`
+### `checksum.persist`
 
-`key: persist_checksum`
+`key: checksum.persist`
 
 `type: bool`
 
 This feature is useful when you want to know that something has changed between two executions of a command.
 
-`persist_checksum` can be used only if `checksum` declared for command.
+`checksum.persist` can be used only if `checksum.files` or `checksum.sh` declared for command.
 
 If set to `true`, each run all calculated checksums will be stored to disk.
 
@@ -869,12 +883,13 @@ Example:
 ```yaml
 commands:
   build:
-    persist_checksum: true
     checksum:
-      deps:
-        - package.json
-      doc:
-        - Readme.md
+      persist: true
+      files:
+        deps:
+          - package.json
+        doc:
+          - Readme.md
 ```
 
 Resulting env will be:
@@ -887,6 +902,8 @@ Resulting env will be:
 * `LETS_CHECKSUM_DOC_CHANGED` - is checksum of doc files changed
 * `LETS_CHECKSUM_CHANGED` - is checksum of all checksums (deps and doc) changed
 
+`checksum` as a direct list/map and command-level `persist_checksum` are deprecated compatibility syntax. Use `lets self fix` to migrate local configs to `checksum.files` and `checksum.persist`.
+
 ### `ref`
 
 `key: ref`

docs/static/schema.json

@@ -132,10 +132,7 @@
           "type": "boolean"
         },
         "checksum": {
-          "type": "array",
-          "items": {
-            "type": "string"
-          }
+          "$ref": "#/definitions/command_checksum"
         },
         "env": {
           "$ref": "#/definitions/env"
@@ -237,6 +234,61 @@
           }
         }
       ]
+    },
+    "checksum_files": {
+      "oneOf": [
+        {
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        },
+        {
+          "type": "object",
+          "additionalProperties": {
+            "type": "array",
+            "items": {
+              "type": "string"
+            }
+          }
+        }
+      ]
+    },
+    "command_checksum": {
+      "oneOf": [
+        {
+          "$ref": "#/definitions/checksum_files"
+        },
+        {
+          "type": "object",
+          "properties": {
+            "files": {
+              "$ref": "#/definitions/checksum_files"
+            },
+            "sh": {
+              "type": "string",
+              "description": "Shell command that prints checksum value."
+            },
+            "persist": {
+              "type": "boolean",
+              "description": "Persist checksum and expose LETS_CHECKSUM_CHANGED variables."
+            }
+          },
+          "oneOf": [
+            {
+              "required": [
+                "files"
+              ]
+            },
+            {
+              "required": [
+                "sh"
+              ]
+            }
+          ],
+          "additionalProperties": false
+        }
+      ]
     }
   },
   "additionalProperties": false

internal/checksum/checksum.go

@@ -6,8 +6,10 @@ import (
 	"encoding/hex"
 	"fmt"
 	"os"
+	"os/exec"
 	"path/filepath"
 	"sort"
+	"strings"
 
 	"github.com/lets-cli/lets/internal/set"
 	"github.com/lets-cli/lets/internal/util"
@@ -147,6 +149,44 @@ func CalculateChecksumFromSources(workDir string, checksumSources map[string][]s
 	return checksumMap, nil
 }
 
+func CalculateChecksumFromConfig(
+	workDir string,
+	checksumSources map[string][]string,
+	shell string,
+	script string,
+	env map[string]string,
+) (map[string]string, error) {
+	if script != "" {
+		result, err := CalculateChecksumFromScript(workDir, shell, script, env)
+		if err != nil {
+			return nil, err
+		}
+
+		return map[string]string{DefaultChecksumKey: result}, nil
+	}
+
+	return CalculateChecksumFromSources(workDir, checksumSources)
+}
+
+func CalculateChecksumFromScript(workDir string, shell string, script string, env map[string]string) (string, error) {
+	cmd := exec.Command(shell, "-c", script)
+	cmd.Dir = workDir
+
+	envList := os.Environ()
+	for key, value := range env {
+		envList = append(envList, fmt.Sprintf("%s=%s", key, value))
+	}
+
+	cmd.Env = envList
+
+	out, err := cmd.Output()
+	if err != nil {
+		return "", fmt.Errorf("can not get output from checksum.sh script: %s: %w", script, err)
+	}
+
+	return strings.TrimSpace(string(out)), nil
+}
+
 func ReadChecksumFromDisk(checksumsDir, cmdName, checksumName string) (string, error) {
 	_, checksumFilePath := getChecksumPath(checksumsDir, cmdName, checksumName)
 

internal/checksum/checksum_test.go

@@ -136,3 +136,16 @@ func TestCalculateChecksumFromListOrMap(t *testing.T) {
 		)
 	}
 }
+
+func TestCalculateChecksumFromScript(t *testing.T) {
+	tempDir := t.TempDir()
+
+	got, err := CalculateChecksumFromScript(tempDir, "bash", `printf "checksum-value\n"`, map[string]string{})
+	if err != nil {
+		t.Fatalf("unexpected error: %s", err)
+	}
+
+	if got != "checksum-value" {
+		t.Fatalf("unexpected checksum: %s", got)
+	}
+}

internal/cmd/fix.go

@@ -0,0 +1,36 @@
+package cmd
+
+import (
+	"os"
+
+	"github.com/lets-cli/lets/internal/config/migrate"
+	"github.com/spf13/cobra"
+)
+
+func initFixCommand() *cobra.Command {
+	var dryRun bool
+
+	fixCmd := &cobra.Command{
+		Use:   "fix",
+		Short: "Apply lets config migrations",
+		Args:  cobra.NoArgs,
+		RunE: func(cmd *cobra.Command, args []string) error {
+			configName, err := cmd.Root().Flags().GetString("config")
+			if err != nil {
+				return err
+			}
+
+			if configName == "" {
+				configName = os.Getenv("LETS_CONFIG")
+			}
+
+			_, err = migrate.Fix(configName, os.Getenv("LETS_CONFIG_DIR"), dryRun, cmd.OutOrStdout())
+
+			return err
+		},
+	}
+
+	fixCmd.Flags().BoolVar(&dryRun, "dry-run", false, "print migrated config without writing files")
+
+	return fixCmd
+}

internal/cmd/self.go

@@ -25,6 +25,7 @@ func initSelfCmd(rootCmd *cobra.Command, version string, openURL func(string) er
 	rootCmd.AddCommand(selfCmd)
 
 	selfCmd.AddCommand(initDocCommand(openURL))
+	selfCmd.AddCommand(initFixCommand())
 	selfCmd.AddCommand(initLspCommand(version))
 	selfCmd.AddCommand(initUpgradeCommand(version))
 }