Acceptance testing: Added playhead support

Author: drowaudio

CLAUDE.md

@@ -114,7 +114,8 @@ pluginval/
 │   ├── AddPluginvalTests.cmake  # CMake module for CTest integration
 │   ├── test_plugins/         # Test plugin files
 │   │   ├── tone_generator/   # Deterministic dogfood generator (PLUGINVAL_BUILD_TEST_PLUGINS)
-│   │   └── gain/             # Deterministic dogfood gain effect (for the input.audio path)
+│   │   ├── gain/             # Deterministic dogfood gain effect (for the input.audio path)
+│   │   └── playhead_probe/   # Writes the host transport to output (for the playhead path)
 │   ├── acceptance/           # Acceptance self-tests: configs (*.json.in), inputs/, checked-in refs/ WAVs
 │   ├── mac_tests/            # macOS-specific tests
 │   └── windows_tests.bat     # Windows test scripts
@@ -286,25 +287,27 @@ spec: `tests/acceptance/Acceptance testing design.md`; end-user guide:
   layering: the test config is a positional argument loaded standalone.
 - **Flow** (`AcceptanceTest.cpp`): load plugin → apply `state.file` then
   `state.parameters` (normalised, matched by index / case-insensitive name or
-  paramID) → feed `input.audio`/`input.midi` or silence → render a fixed
-  duration block-by-block (reusing the `AudioProcessingTest` shape + the
-  VST3-safe helpers in `TestUtilities.h`). If no reference exists it **records**
-  one (32-bit float WAV + `<name>.wav.json` sidecar manifest); otherwise it
-  **compares** and writes a diff WAV on failure. Exit `0`/`1`.
+  paramID) → feed `input.audio`/`input.midi` or silence → if a `playhead` is
+  configured, point a fixed-tempo transport (`FixedPlayHead`, position advances
+  per block) at the plugin → render a fixed duration block-by-block (reusing the
+  `AudioProcessingTest` shape + the VST3-safe helpers in `TestUtilities.h`). If
+  no reference exists it **records** one (32-bit float WAV + `<name>.wav.json`
+  sidecar manifest); otherwise it **compares** and writes a diff WAV on failure.
+  Exit `0`/`1`.
 - **Comparators** (`ReferenceComparator.cpp/h`): pluggable `Comparator` +
   `createComparator(name)` registry. v1 ships only `sample` (per-sample abs-diff
   tolerance, default one 16-bit LSB = `1/32768`; `0` = bit-exact). Adding
   `spectrum`/`crosscorr`/etc. is one registry entry, no config/runner changes.
-- **Dogfood + self-tests**: two minimal deterministic `juce_add_plugin` targets
+- **Dogfood + self-tests**: three minimal deterministic `juce_add_plugin` targets
   behind `PLUGINVAL_BUILD_TEST_PLUGINS` — `tests/test_plugins/tone_generator/`
-  (closed-form sine/square generator, phase resets on `prepareToPlay`) and
-  `tests/test_plugins/gain/` (a gain effect, used to dogfood the `input.audio`
-  path: it gains a checked-in full-height sine and is compared bit-exact).
-  `tests/acceptance/` holds checked-in configs (`*.json.in`, the plugin paths +
-  input dir substituted at configure time), `inputs/` and reference WAVs, run via
-  CTest (`pluginval.acceptance.*`: sine-440, square-220, square-state, gain-half).
-  Phase 2 items (config-array multiplexing,
-  automation, playhead, extra comparators, child-process isolation) are notes
+  (closed-form sine/square generator, phase resets on `prepareToPlay`),
+  `tests/test_plugins/gain/` (a gain effect, dogfoods the `input.audio` path) and
+  `tests/test_plugins/playhead_probe/` (writes the host transport to its output,
+  dogfoods the `playhead` path). `tests/acceptance/` holds checked-in configs
+  (`*.json.in`, the plugin paths + input dir substituted at configure time),
+  `inputs/` and reference WAVs, run via CTest (`pluginval.acceptance.*`: sine-440,
+  square-220, square-state, gain-half, playhead-120). Phase 2 items (config-array
+  multiplexing, automation, extra comparators, child-process isolation) are notes
   only.
 
 ### Test Framework

CMakeLists.txt

@@ -274,5 +274,6 @@ if (PLUGINVAL_BUILD_TEST_PLUGINS)
     enable_testing()
     add_subdirectory(tests/test_plugins/tone_generator)
     add_subdirectory(tests/test_plugins/gain)
+    add_subdirectory(tests/test_plugins/playhead_probe)
     add_subdirectory(tests/acceptance)
 endif()

docs/Acceptance testing.md

@@ -48,6 +48,7 @@ JSON keys are `snake_case`. The most useful fields:
 | `state.file` | A binary `getStateInformation` blob to restore first (e.g. a captured preset). Applied before `state.parameters`. |
 | `reference` | The golden `.wav`. Defaults to `<name>.wav` next to the config. |
 | `render_duration` | Seconds to render. If omitted, the input audio's length is used. |
+| `playhead` | A fixed transport for tempo-dependent plugins: `{ "bpm": 120, "time_signature": { "numerator": 4, "denominator": 4 } }`. Omit it and the plugin gets no playhead. The position advances with the render. |
 | `comparison` | How to compare. `{ "sample": <tolerance> }` is a per-sample absolute-difference tolerance (`0` = bit-exact); the default is one 16-bit LSB. |
 
 ### Determinism matters

source/CommandLineTests.cpp

@@ -506,6 +506,55 @@ struct CommandLineTests : public juce::UnitTest
             expect (! config.renderDuration.has_value());
             expectEquals (config.getComparison()["sample"].get<double>(), 0.0);
         }
+
+        beginTest ("Acceptance TestConfig playhead (object time signature)");
+        {
+            // Absent playhead -> unset (no transport supplied to the plugin).
+            {
+                const auto config = nlohmann::json::parse (R"({ "plugin": "P.vst3" })").get<acceptance::TestConfig>();
+                expect (! config.playhead.has_value());
+            }
+
+            // Present playhead -> parsed, with time_signature as an object.
+            {
+                const auto json = R"({
+                    "plugin": "P.vst3",
+                    "playhead": {
+                        "bpm": 90,
+                        "time_signature": { "numerator": 6, "denominator": 8 },
+                        "start_ppq": 4.0
+                    }
+                })";
+
+                const auto config = nlohmann::json::parse (json).get<acceptance::TestConfig>();
+                expect (config.playhead.has_value());
+                expectEquals (config.playhead->bpm, 90.0);
+                expectEquals (config.playhead->timeSigNumerator, 6);
+                expectEquals (config.playhead->timeSigDenominator, 8);
+                expectEquals (config.playhead->startPpq, 4.0);
+            }
+
+            // Time signature defaults to 4/4 when omitted.
+            {
+                const auto config = nlohmann::json::parse (R"({ "plugin": "P.vst3", "playhead": { "bpm": 100 } })")
+                                        .get<acceptance::TestConfig>();
+                expect (config.playhead.has_value());
+                expectEquals (config.playhead->timeSigNumerator, 4);
+                expectEquals (config.playhead->timeSigDenominator, 4);
+            }
+
+            // An invalid (zero) denominator is rejected.
+            {
+                bool threw = false;
+                try
+                {
+                    nlohmann::json::parse (R"({ "plugin": "P.vst3", "playhead": { "bpm": 120, "time_signature": { "numerator": 4, "denominator": 0 } } })")
+                        .get<acceptance::TestConfig>();
+                }
+                catch (const std::exception&) { threw = true; }
+                expect (threw, "expected a zero denominator to be rejected");
+            }
+        }
     }
 };
 

source/acceptance/AcceptanceTest.cpp

@@ -38,6 +38,45 @@ namespace
        #endif
     }
 
+    //==============================================================================
+    /** A fixed-tempo transport whose position advances with the render. Constructed
+        from the config's playhead block and pointed at the plugin for the duration
+        of the render; setSamplePosition() is called before each processBlock. */
+    class FixedPlayHead : public juce::AudioPlayHead
+    {
+    public:
+        FixedPlayHead (const TestConfig::PlayheadConfig& config, double sr)
+            : cfg (config), sampleRate (sr)
+        {
+        }
+
+        void setSamplePosition (juce::int64 sample) noexcept   { currentSample = sample; }
+
+        juce::Optional<PositionInfo> getPosition() const override
+        {
+            const double seconds = (double) currentSample / sampleRate;
+
+            // PPQ is measured in quarter notes; a bar is numerator * (4/denominator) of them.
+            const double quarterNotesPerBar = cfg.timeSigNumerator * 4.0 / cfg.timeSigDenominator;
+            const double ppq = cfg.startPpq + seconds * (cfg.bpm / 60.0);
+
+            PositionInfo info;
+            info.setBpm (cfg.bpm);
+            info.setTimeSignature (TimeSignature { cfg.timeSigNumerator, cfg.timeSigDenominator });
+            info.setTimeInSamples (currentSample);
+            info.setTimeInSeconds (seconds);
+            info.setPpqPosition (ppq);
+            info.setPpqPositionOfLastBarStart (std::floor (ppq / quarterNotesPerBar) * quarterNotesPerBar);
+            info.setIsPlaying (true);
+            return info;
+        }
+
+    private:
+        const TestConfig::PlayheadConfig cfg;
+        const double sampleRate;
+        juce::int64 currentSample = 0;
+    };
+
     //==============================================================================
     std::unique_ptr<juce::AudioPluginInstance> loadPlugin (juce::AudioPluginFormatManager& formatManager,
                                                            const juce::String& pathOrID,
@@ -254,7 +293,15 @@ RenderedAudio renderPlugin (const TestConfig& config)
     if (numSamples <= 0)
         throw std::runtime_error ("render_duration is required when there is no input.audio");
 
-    // 4. Prepare and render block by block.
+    // 4. Optional fixed transport for time-dependent plugins.
+    std::unique_ptr<FixedPlayHead> playHead;
+    if (config.playhead)
+    {
+        playHead = std::make_unique<FixedPlayHead> (*config.playhead, sampleRate);
+        instance->setPlayHead (playHead.get());
+    }
+
+    // 5. Prepare and render block by block.
     callPrepareToPlayOnMessageThreadIfVST3 (*instance, sampleRate, blockSize);
 
     const int numInputChannels = instance->getTotalNumInputChannels();
@@ -270,6 +317,9 @@ RenderedAudio renderPlugin (const TestConfig& config)
     {
         const int thisBlock = juce::jmin (blockSize, numSamples - pos);
 
+        if (playHead != nullptr)
+            playHead->setSamplePosition (pos);
+
         block.clear();
 
         // Copy the input audio slice into the block (silence past its end).
@@ -290,6 +340,7 @@ RenderedAudio renderPlugin (const TestConfig& config)
             output.copyFrom (c, pos, proc, c, 0, thisBlock);
     }
 
+    instance->setPlayHead (nullptr);   // playHead is about to be destroyed
     callReleaseResourcesOnMessageThreadIfVST3 (*instance);
     instance.reset();
 

source/acceptance/TestConfig.cpp

@@ -73,6 +73,16 @@ void to_json (nlohmann::json& j, const TestConfig& c)
 
     if (! c.comparison.is_null())
         j["comparison"] = c.comparison;
+
+    if (c.playhead)
+    {
+        j["playhead"] = {
+            { "bpm",            c.playhead->bpm },
+            { "time_signature", { { "numerator",   c.playhead->timeSigNumerator },
+                                  { "denominator", c.playhead->timeSigDenominator } } },
+            { "start_ppq",      c.playhead->startPpq }
+        };
+    }
 }
 
 void from_json (const nlohmann::json& j, TestConfig& c)
@@ -98,6 +108,24 @@ void from_json (const nlohmann::json& j, TestConfig& c)
 
     if (auto comp = j.find ("comparison"); comp != j.end() && ! comp->is_null())
         c.comparison = *comp;
+
+    if (auto ph = j.find ("playhead"); ph != j.end() && ph->is_object())
+    {
+        TestConfig::PlayheadConfig p;
+        p.bpm      = ph->value ("bpm", 120.0);
+        p.startPpq = ph->value ("start_ppq", 0.0);
+
+        if (auto ts = ph->find ("time_signature"); ts != ph->end() && ts->is_object())
+        {
+            p.timeSigNumerator   = ts->value ("numerator", 4);
+            p.timeSigDenominator = ts->value ("denominator", 4);
+        }
+
+        if (p.timeSigNumerator <= 0 || p.timeSigDenominator <= 0)
+            throw std::runtime_error ("playhead.time_signature numerator and denominator must be positive");
+
+        c.playhead = p;
+    }
 }
 
 //==============================================================================

source/acceptance/TestConfig.h

@@ -40,6 +40,17 @@ namespace acceptance
 */
 struct TestConfig
 {
+    /** A fixed transport supplied to the plugin during the render, for
+        time-dependent plugins (tempo-synced LFOs, arpeggiators, ...). The tempo
+        and time signature are constant; the position advances with the render. */
+    struct PlayheadConfig
+    {
+        double bpm = 120.0;
+        int    timeSigNumerator = 4;
+        int    timeSigDenominator = 4;
+        double startPpq = 0.0;       /**< Transport position (in quarter notes) at sample 0. */
+    };
+
     std::string name;                              /**< Labels results; derives the default reference path. */
     std::string plugin;                            /**< Plugin path or AU id. Required. */
     std::string inputAudio;                        /**< input.audio: path to an input audio file, or empty for silence. */
@@ -51,6 +62,7 @@ struct TestConfig
     int blockSize = 512;
     std::optional<double> renderDuration;          /**< Seconds. Unset -> derive from the input length. */
     nlohmann::json comparison;                     /**< Map of comparator name -> sub-config. Empty -> default. */
+    std::optional<PlayheadConfig> playhead;        /**< Fixed transport. Unset -> no playhead supplied to the plugin. */
 
     //==============================================================================
     /** The directory the config was loaded from. Relative reference / input /

tests/acceptance/Acceptance testing design.md

@@ -122,7 +122,7 @@ member names verbatim). Missing keys still fall back to the defaults.
   "block_size": 512,
   "render_duration": 2.0,                      // seconds; omitted -> use input length
   "comparison": { "sample": 1e-6 },            // omitted -> default 1/32768 (16-bit LSB)
-  "playhead": { "bpm": 120 },                  // future
+  "playhead": { "bpm": 120, "time_signature": { "numerator": 4, "denominator": 4 } },
   "automation": [ /* phase 2 */ ]
 }
 ```
@@ -142,7 +142,7 @@ member names verbatim). Missing keys still fall back to the defaults.
 | `block_size` | number | 512 | Single value. |
 | `render_duration` | number | input length | Seconds. `num_samples = round(duration * sample_rate)`. Input shorter than the duration is padded with silence; longer is truncated. |
 | `comparison` | object | `{ "sample": 0.0000305 }` | Map of comparator name -> its sub-config. See §5. |
-| `playhead` | object | none | **Future.** Fixed tempo / time signature for time-dependent plugins. |
+| `playhead` | object | none | Fixed transport for time-dependent plugins. `{ "bpm": <number>, "time_signature": { "numerator": N, "denominator": D }, "start_ppq": <number> }`. `time_signature` defaults to 4/4, `start_ppq` to 0. Omitted -> no playhead is set (the plugin sees `getPlayHead() == nullptr`). The tempo / time signature are constant; the position advances with the render. |
 | `automation` | array | none | **Phase 2.** Parameter changes scheduled at sample positions. |
 
 ### State precedence
@@ -323,6 +323,7 @@ tests/acceptance/
   square-220.json.in      // tone gen, waveform=square, state.parameters, bit-exact
   square-state.json.in    // tone gen, state.file blob + a gain parameter override
   gain-half.json.in       // gain effect, input.audio = a full-height sine, bit-exact
+  playhead-120.json.in    // playhead probe, fixed transport (bpm 120, 4/4), bit-exact
   inputs/sine-full.wav    // checked-in input for gain-half (±1.0 sine)
   refs/<case>.wav (+ .wav.json)   // checked-in references + sidecar manifests
   refs/square-state.state         // checked-in getStateInformation blob
@@ -337,7 +338,7 @@ stable enough to commit — a first smoke test of cross-platform portability for
 the `sample` comparator and the baseline for future comparators (`spectrum`,
 `crosscorr`, …).
 
-The four cases cover the distinct render paths:
+The five cases cover the distinct render paths:
 
 - **`sine-440` / `square-220`** — the `state.parameters` (name/index → normalised
   value) path. `square-220` compares bit-exact (`"sample": 0`).
@@ -350,6 +351,11 @@ The four cases cover the distinct render paths:
   full-height (±1.0) sine by 0.5 and is compared bit-exact (0.5 is exact in
   float). It also omits `render_duration`, so the render length is derived from
   the input file.
+- **`playhead-120`** — the **`playhead`** path: a third dogfood plugin
+  (`tests/test_plugins/playhead_probe/`) writes the host transport into its
+  output (channel 0 = `ppqPosition`, channel 1 = tempo), so the recorded
+  reference is a direct check that the fixed transport reached the plugin. If the
+  playhead regressed, the probe would output silence and the compare would fail.
 
 ## 9. Execution flow
 
@@ -358,11 +364,13 @@ The four cases cover the distinct render paths:
    `sample_rate` / `block_size`.
 3. Apply `state.file` then `state.parameters`.
 4. Load `input.audio` and/or `input.midi`; otherwise use silence.
-5. `prepareToPlay`, render `render_duration` worth of blocks, accumulating the
+5. If a `playhead` is configured, point a fixed-tempo transport at the plugin
+   (its position advances each block).
+6. `prepareToPlay`, render `render_duration` worth of blocks, accumulating the
    output into a single buffer.
-6. **No reference exists** -> write the float WAV + manifest (record mode);
+7. **No reference exists** -> write the float WAV + manifest (record mode);
    report "reference created".
-7. **Reference exists** -> run each configured comparator; report each verdict
+8. **Reference exists** -> run each configured comparator; report each verdict
    and the overall pass/fail; on failure write a diff WAV; exit `0` / `1`.
 
 ## 10. Phasing
@@ -377,14 +385,14 @@ The four cases cover the distinct render paths:
 - Record-or-compare with float-WAV + JSON-sidecar references.
 - `sample` comparator only (default tolerance = one 16-bit LSB).
 - Text + JSON result reporting; diff WAV on failure.
-- **Dogfood tone-generator plugin** (§8) plus a CTest self-test that runs the
-  full record/compare path against checked-in references.
+- Fixed `playhead` (tempo / time signature) for time-dependent plugins.
+- **Dogfood test plugins** (§8) plus CTest self-tests that run the full
+  record/compare path against checked-in references.
 
 **Phase 2 and beyond**
 
 - Multiplexed execution of config arrays.
 - Parameter `automation` timelines.
-- Fixed `playhead` (tempo / time signature) for time-dependent plugins.
 - Additional comparators: `peakrms`, `spectrum`, `crosscorr`, `fingerprint`.
 - Synthesised inputs (`input.generator`: noise / sine, with seed).
 - Optional child-process isolation mirroring the validate handoff.