Revise precedence to defaults < env < config < CLI

Reworks the layering so a --config file (deliberate, per-invocation) ranks
above ambient environment variables, and makes --config repeatable.

- Environment layer: env-var names are now derived from the registered CLI11
  options (--strictness-level -> STRICTNESS_LEVEL) instead of CLI11's
  ->envname(), so there is no hand-maintained env table and env support is
  automatic for every option. A synthetic "--name=value" argv is built from
  the environment and parsed by CLI11, reusing all its coercion (comma lists,
  enum transformer, hex-seed callback). Flags accept --flag=1/0.
- --config: repeatable; each JSON file is merge_patch-ed in command-line order
  (later files win per key). Applied between the env and CLI layers, so it
  beats env and loses to explicit CLI options. (Inline JSON dropped for now to
  avoid command-line re-tokenisation hazards; file paths only.)
- The env layer reads through an injectable EnvProvider again, so the unit
  tests no longer mutate the real process environment (also removes the
  Windows _putenv_s fragility).
- Tests updated for the new precedence + a repeatable-config test.
- Help footer, docs and CLAUDE.md updated.

Verified end-to-end: env(6) -> 6; env(6)+config(2) -> 2; +CLI(9) -> 9.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: drowaudio

CLAUDE.md

@@ -194,24 +194,32 @@ VST2_SDK_DIR=/path/to/vst2sdk cmake -B Builds/Debug .
 ### CLI Settings Pipeline
 
 Command-line parsing centres on one plain settings struct (`PluginvalSettings`)
-that CLI11 binds to directly. The flow (in `SettingsParser`):
+that CLI11 binds to directly. A single instance is filled by successive layers,
+**lowest to highest precedence: defaults → environment → `--config` → CLI**
+(in `SettingsParser::parseTokens`):
 
 1. **preprocess** the raw command line — rewrite the deprecated `strictnessLevel`,
    strip the macOS `-NSDocumentRevisionsDebugMode YES` flag, and insert an
    implicit `--validate` when the last argument is a bare plugin path.
-2. If `--config <file.json>` is present, **seed** the struct from that JSON first.
-3. **CLI11** binds each `add_option`/`add_flag` straight to a `PluginvalSettings`
-   member, with the environment variable on the same line via `->envname()`.
-   Because CLI11 only overwrites a member when its flag/env was actually provided,
-   precedence is **defaults < config < env < CLI** (CLI wins) with no manual
-   layering. Comma lists use `->delimiter(',')`, the enum uses a
-   `CheckedTransformer`, and the hex/int seed is a small callback.
-4. `PluginvalSettings::toPluginTestOptions()` converts to the JUCE-flavoured
-   `PluginTests::Options` at the boundary.
+2. **Environment layer.** Env-var names are *derived* from the registered
+   options (`--strictness-level` → `STRICTNESS_LEVEL`), so there is no separate
+   env table. A synthetic `--name=value` argv is built from the environment and
+   parsed by CLI11, reusing all its coercion.
+3. **`--config` layer.** Repeatable; each JSON file is `merge_patch`-ed in
+   command-line order (later files win per key). Beats the environment.
+4. **CLI layer.** The real arguments are parsed last and beat everything;
+   CLI11 only overwrites a member when its option was actually provided.
+
+`configureApp()` registers every option (bound to the struct) and is used for
+both the env pass and the CLI pass. Comma lists use `->delimiter(',')`, the enum
+uses a `CheckedTransformer`, and the hex/int seed is a small callback.
+`PluginvalSettings::toPluginTestOptions()` converts to the JUCE-flavoured
+`PluginTests::Options` at the boundary.
 
 Adding a new option is three edits: a struct member, an entry in the nlohmann
-macro list, and one `add_option(...)` line. `SettingsSerializer` handles JSON
-load/save plus the two remaining conversions (hex seed, disabled-tests file).
+macro list, and one `add_option(...)` line — its environment variable then works
+automatically. `SettingsSerializer` handles JSON load/save plus the two
+remaining conversions (hex seed, disabled-tests file).
 
 The child validation process receives a fully-resolved, **authoritative**
 settings set via a base64-encoded JSON argument (`--config-base64`), avoiding

Source/CommandLineTests.cpp

@@ -12,7 +12,7 @@
 
  ==============================================================================*/
 
-#include <cstdlib>
+#include <map>
 
 struct CommandLineTests : public juce::UnitTest
 {
@@ -21,46 +21,32 @@ struct CommandLineTests : public juce::UnitTest
     {
     }
 
-    static constexpr const char* knownEnvVars[] = {
-        "STRICTNESS_LEVEL", "RANDOM_SEED", "TIMEOUT_MS", "VERBOSE", "REPEAT",
-        "RANDOMISE", "SKIP_GUI_TESTS", "DATA_FILE", "OUTPUT_DIR", "OUTPUT_FILENAME",
-        "SAMPLE_RATES", "BLOCK_SIZES", "RTCHECK"
-    };
-
-    static void setEnv (const char* name, const char* value)
+    static settings_parser::EnvProvider emptyEnv()
     {
-       #if JUCE_WINDOWS
-        _putenv_s (name, value);
-       #else
-        ::setenv (name, value, 1);
-       #endif
+        return [] (const juce::String&) { return juce::String(); };
     }
 
-    static void unsetEnv (const char* name)
+    static settings_parser::EnvProvider envFrom (std::map<juce::String, juce::String> vars)
     {
-       #if JUCE_WINDOWS
-        _putenv_s (name, "");
-       #else
-        ::unsetenv (name);
-       #endif
+        return [vars = std::move (vars)] (const juce::String& name)
+        {
+            const auto it = vars.find (name);
+            return it != vars.end() ? it->second : juce::String();
+        };
     }
 
-    static void clearKnownEnv()
+    static PluginvalSettings parse (const juce::String& cmd, settings_parser::EnvProvider env)
     {
-        for (auto* n : knownEnvVars)
-            unsetEnv (n);
+        return settings_parser::parse (cmd, env);
     }
 
     static PluginvalSettings parse (const juce::String& cmd)
     {
-        return settings_parser::parse (cmd);
+        return settings_parser::parse (cmd, emptyEnv());
     }
 
     void runTest() override
     {
-        // Start from a clean environment so host env vars don't affect the deterministic tests.
-        clearKnownEnv();
-
         beginTest ("Command line defaults");
         {
             const auto opts = parse ("").toPluginTestOptions();
@@ -203,79 +189,93 @@ struct CommandLineTests : public juce::UnitTest
 
         beginTest ("Environment variables");
         {
-            setEnv ("STRICTNESS_LEVEL", "7");
-            setEnv ("RANDOM_SEED", "1234");
-            setEnv ("TIMEOUT_MS", "20000");
-            setEnv ("VERBOSE", "1");
-            setEnv ("REPEAT", "11");
-            setEnv ("RANDOMISE", "1");
-            setEnv ("SKIP_GUI_TESTS", "1");
-            setEnv ("DATA_FILE", "/path/to/file");
-            setEnv ("OUTPUT_DIR", "/path/to/dir");
-            setEnv ("SAMPLE_RATES", "22050,44100");
-            setEnv ("BLOCK_SIZES", "32,64");
-            setEnv ("RTCHECK", "relaxed");
-
-            const auto opts = parse ("--validate x").toPluginTestOptions();
-
-            clearKnownEnv();
-
+            const auto env = envFrom ({
+                { "STRICTNESS_LEVEL", "7" },
+                { "RANDOM_SEED", "1234" },
+                { "TIMEOUT_MS", "20000" },
+                { "VERBOSE", "1" },
+                { "REPEAT", "11" },
+                { "RANDOMISE", "1" },
+                { "SKIP_GUI_TESTS", "1" },
+                { "DATA_FILE", "/path/to/file" },
+                { "OUTPUT_DIR", "/path/to/dir" },
+                { "SAMPLE_RATES", "22050,44100" },
+                { "BLOCK_SIZES", "32,64" },
+                { "RTCHECK", "relaxed" },
+            });
+
+            const auto settings = parse ("--validate x", env);
+            const auto opts = settings.toPluginTestOptions();
             expectEquals (opts.strictnessLevel, 7);
             expectEquals (opts.randomSeed, (juce::int64) 1234);
             expectEquals (opts.timeoutMs, (juce::int64) 20000);
             expect (opts.verbose);
             expectEquals (opts.numRepeats, 11);
             expect (opts.randomiseTestOrder);
             expect (opts.withGUI == false);
+            expectEquals (juce::String (settings.dataFile), juce::String ("/path/to/file"));
             expect (opts.sampleRates == std::vector<double> ({ 22050.0, 44100.0 }));
             expect (opts.blockSizes == std::vector<int> ({ 32, 64 }));
             expect (opts.realtimeCheck == RealtimeCheck::relaxed);
         }
 
         beginTest ("Command line overrides environment variables");
         {
-            setEnv ("STRICTNESS_LEVEL", "3");
-            const auto envOnly = parse ("--validate x").strictnessLevel;
-            const auto cliWins = parse ("--strictness-level 9 --validate x").strictnessLevel;
-            clearKnownEnv();
-
-            expectEquals (envOnly, 3);   // env only
-            expectEquals (cliWins, 9);   // CLI wins
+            const auto env = envFrom ({ { "STRICTNESS_LEVEL", "3" } });
+            expectEquals (parse ("--validate x", env).strictnessLevel, 3);                       // env only
+            expectEquals (parse ("--strictness-level 9 --validate x", env).strictnessLevel, 9);  // CLI wins
         }
 
-        beginTest ("Config file and precedence (CLI > env > config > defaults)");
+        beginTest ("Precedence: CLI > --config > env > defaults");
         {
             juce::TemporaryFile configFile (".json");
-            configFile.getFile().replaceWithText (R"({ "strictnessLevel": 2, "timeoutMs": 12345, "numRepeats": 4 })");
+            configFile.getFile().replaceWithText (R"({ "strictnessLevel": 2, "timeoutMs": 12345 })");
             const auto cfg = "--config " + configFile.getFile().getFullPathName().quoted();
 
+            const auto env6 = envFrom ({ { "STRICTNESS_LEVEL", "6" } });
+
+            // env beats defaults
+            expectEquals (parse ("--validate x", env6).strictnessLevel, 6);
+
             // config alone
             {
                 const auto s = parse (cfg + " --validate x");
                 expectEquals (s.strictnessLevel, 2);
                 expectEquals ((juce::int64) s.timeoutMs, (juce::int64) 12345);
-                expectEquals (s.numRepeats, 4);
             }
 
-            // env overrides config
+            // config beats env
             {
-                setEnv ("STRICTNESS_LEVEL", "6");
-                const auto s = parse (cfg + " --validate x");
-                clearKnownEnv();
-                expectEquals (s.strictnessLevel, 6);                 // env beats config
-                expectEquals ((juce::int64) s.timeoutMs, (juce::int64) 12345);     // still from config
+                const auto s = parse (cfg + " --validate x", env6);
+                expectEquals (s.strictnessLevel, 2);                 // config wins over env
+                expectEquals ((juce::int64) s.timeoutMs, (juce::int64) 12345);
             }
 
-            // CLI overrides env and config
+            // CLI beats config and env
             {
-                setEnv ("STRICTNESS_LEVEL", "6");
-                const auto s = parse (cfg + " --strictness-level 9 --validate x");
-                clearKnownEnv();
+                const auto s = parse (cfg + " --strictness-level 9 --validate x", env6);
                 expectEquals (s.strictnessLevel, 9);
                 expectEquals ((juce::int64) s.timeoutMs, (juce::int64) 12345);
             }
         }
 
+        beginTest ("Repeatable --config merges per key, last wins");
+        {
+            juce::TemporaryFile baseFile (".json");
+            baseFile.getFile().replaceWithText (R"({ "strictnessLevel": 2, "timeoutMs": 11111 })");
+
+            juce::TemporaryFile overrideFile (".json");
+            overrideFile.getFile().replaceWithText (R"({ "strictnessLevel": 8 })");
+
+            const auto cmd = "--config " + baseFile.getFile().getFullPathName().quoted()
+                           + " --config " + overrideFile.getFile().getFullPathName().quoted()
+                           + " --validate x";
+
+            const auto s = parse (cmd);
+            expectEquals (s.strictnessLevel, 8);                       // overridden by the second file
+            expectEquals ((juce::int64) s.timeoutMs, (juce::int64) 11111); // untouched, kept from the first
+        }
+
         beginTest ("Child-process command line round-trip");
         {
             PluginTests::Options opts;