Add spec-level hint gate, actionable hints, and concise runner output

Driven by analysis of /tmp/liquid-spec-results.jsonl (6.2M run lines, 8523 specs):
find specs that consistently fail at low complexity without actionable guidance.

Hints + complexity (#1, #2):
- Add specific spec-level hints to 19 consistently-failing specs: the
  to_liquid-in-conditions/lookups cluster (VariableTest#test_*_calls_to_liquid_value_*,
  19 specs at c=220, ~89% fail), the nil-comparison quirk
  (StatementsTest#test_zero_lq_or_equal_one_involving_nil, c=140, 89% fail), and the
  nested {% liquid %} spec (LiquidTagTest#test_nested_liquid_tag, c=130, 50% fail).
- Raise ForTagTest#test_iterate_with_each_when_no_limit from c=70 to c=200: it uses
  instantiate:LoaderDrop (drop iteration) and was misplaced in the near-beginner band.

Spec-quality gate (#3):
- New test_specs_through_220_have_spec_level_hints: requires a spec-LEVEL hint
  (spec.hint, not suite/file effective_hint) for c<=220, with a frozen per-spec
  baseline (test/spec_hint_baseline.txt). Bidirectional: fails on new no-hint specs
  (regressions) AND stale baseline entries (hints added or specs removed), so the
  grandfather list shrinks instead of accreting dead weight. Threshold stays at 220
  per the stated policy; generated/bulk specs are exempted exactly, not by file.
- scripts/generate_spec_hint_baseline.rb regenerates the baseline.

Filter-matrix hints (#4):
- docs/filter_matrix_quirks.md: category-level guide for the 716 c=160 generated
  filter specs (date rendering, float formatting, type coercion, nil/empty, drops).
- tasks/helpers.rb#format_and_write_specs accepts metadata: and writes the
  _metadata wrapper, so rake generate:standard_filters preserves the file-level
  hint instead of erasing it. standard_filters.yml metadata hint now references
  the doc and lists the five failure categories.

Runner output (user request):
- Default plain-mode output is now a single summary line then the lowest-complexity
  failures: "Complexity bar cleared: NN, NNNN passes, NNN failures. Next best specs
  to work on:" + N failure rows (default 5, --max-failures). Each failure row shows
  [c=N] complexity. Preamble, per-suite progress, and skipped-suite output are
  verbose-only (-v); --json is unchanged. --add-specs output also gated to verbose.
- Updated test/runner_diagnostics_test.rb: new test_plain_output_starts_with_...
  asserts stdout starts with the summary and omits preamble; prioritized test
  asserts no "Prioritized Specs" before the header.

Author: tobi

CLAUDE.md

@@ -38,6 +38,22 @@ liquid-spec my_adapter.rb -l
 liquid-spec my_adapter.rb --list-suites
 ```
 
+### Default runner output
+
+The standard (`liquid-spec adapter.rb`) run prints a single summary line followed by the
+lowest-complexity failures — the "next best specs to work on" (capped at `--max-failures`,
+default 5). Preamble, per-suite progress, and skipped-suite lines are verbose-only (`-v`):
+
+```
+Complexity bar cleared: 70, 1234 passes, 56 failures. Next best specs to work on:
+
+1) [c=70] ForTagTest#test_iterate_with_each...
+   ...
+```
+
+`--json` output is unchanged (machine-readable). `--list-passed` appends passing specs
+after the failures.
+
 
 
 ### Spec Quality Gates
@@ -51,8 +67,15 @@ ruby -Ilib -I$LIQUID -Itest -e 'require File.expand_path("test/spec_quality_test
 It currently enforces:
 - complexity scores must be <= 1000
 - every spec with effective complexity <= 220 must have an effective hint
-
-If it fails, add a useful spec/source-level hint or move the spec later in the ramp.
+- every spec with complexity <= 220 must have a spec-LEVEL hint (`spec.hint`), not just
+  suite/file boilerplate. Existing generated/bulk specs without one are grandfathered in
+  `test/spec_hint_baseline.txt`; any NEW spec at c<=220 lacking a spec-level hint fails
+  the gate, and stale baseline entries (specs that gained a hint or were removed) also
+  fail. After intentionally adding hints, shrink the baseline:
+
+  ```bash
+  ruby -Ilib scripts/generate_spec_hint_baseline.rb
+  ```
 
 ### `verify_*` scripts
 

docs/filter_matrix_quirks.md

@@ -0,0 +1,75 @@
+# Generated filter-matrix quirks
+
+`specs/liquid_ruby/standard_filters.yml` is generated by `rake generate:standard_filters`
+from Shopify/liquid's `StandardFilterTest` / `StandardFiltersTest`. Each filter is exercised
+across many input types (strings, integers, floats, booleans, nil, arrays, hashes, drops,
+Dates). The specs are scored at complexity 160 — past the curated beginner ramp in
+`specs/basics/` — because the interesting failures are rarely about the filter itself and
+almost always about one of the following cross-cutting behaviors. If a single filter spec
+fails, check which category the mismatch falls into before assuming the filter is wrong.
+
+## 1. Date / Time rendering
+
+Filters that receive a `Date` or `Time` (e.g. `escape`, `capitalize`, `append`) must first
+render the date to its Liquid string form before applying. Liquid renders `Date` as
+`YYYY-MM-DD` (e.g. `2001-02-03`) and `Time` as `YYYY-MM-DD HH:MM:SS %z` (UTC under the
+frozen test time `2024-01-01 00:01:58 UTC`).
+
+- `{{ d | escape }}` with `d = 2001-02-03` → `2001-02-03` (escape of a date string is a
+  no-op; the work is in the date→string step).
+- A filter that stringifies its input (e.g. `append`) must use the date string, not the
+  raw object.
+
+**Check:** does your implementation render `Date`/`Time` to the correct string *before*
+the filter sees it?
+
+## 2. Float formatting and precision
+
+Liquid outputs floats using Ruby's `Float#to_s`, which trims trailing zeros and uses the
+shortest round-trip representation.
+
+- `0.0` renders as `0.0` (not `0` and not `0.00`).
+- `0.1 + 0.5` (e.g. via `plus`) renders as `0.6`.
+- Integer-valued float results keep the `.0` (`4.0` → `4.0`, not `4`).
+
+**Check:** does your number rendering preserve the float/integer distinction and match
+Ruby's `to_s` trimming?
+
+## 3. Type coercion of inputs
+
+Generated inputs deliberately mix types. A filter's behavior depends on the *coerced*
+input type, not the surface type:
+
+- `abs` of `true` → `0` (booleans coerce to 0/1, then abs).
+- `first` / `last` on a coerced string or array.
+- `contains` / `has` with type-mismatched operands → `false` (no implicit conversion).
+
+**Check:** are you coercing operands the way Liquid does (boolean→0/1, string→chars for
+`first`/`last`, strict `contains` equality) before applying the filter?
+
+## 4. nil / empty handling
+
+- Most filters pass `nil` through as `""` (empty string), not `"nil"`.
+- `nil | size` → `0`. `nil | first` → `""`.
+- Empty array/string inputs produce empty or zero outputs, not errors.
+
+**Check:** does every filter treat `nil` as empty-string (or the filter-specific empty
+value) rather than erroring or stringifying `nil`?
+
+## 5. Drops and Ruby objects (ruby_drops / ruby_types)
+
+Inputs may be drops (`instantiate:ToSDrop`, `instantiate:IntegerDrop`, …) or Ruby-typed
+hashes (integer keys, symbol keys). These specs are auto-tagged `ruby_drops` / `ruby_types`
+by the generator. JSON-RPC adapters that opt out of those features skip them.
+
+- A drop input must have `to_liquid` / `to_s` called before the filter applies.
+- Hash-inspect outputs (`{"k"=>"v"}`) are Ruby-interop — see
+  `docs/ruby_hash_inspect_format.md`.
+
+## When to add a spec-level hint
+
+The file-level `_metadata.hint` (above) points here. A *spec-level* `hint:` is only needed
+when a single spec exercises a quirk not covered by the categories above — add it directly
+to the spec in `standard_filters.yml`; regeneration preserves existing metadata but
+overwrites spec entries, so re-add the spec-level hint in the generator
+(`lib/liquid/spec/deps/standard_filter_patch.rb`) if it must survive regen.

lib/liquid/spec/cli/runner.rb

@@ -291,7 +291,7 @@ def run_specs_frozen(config, options, run_id)
 
             missing_features = config.missing_features
             json_mode = options[:json]
-            puts "Missing features: #{missing_features.join(", ")}" unless json_mode
+            puts "Missing features: #{missing_features.join(", ")}" if config.verbose && !json_mode
 
             # Count available specs from all sources
             has_prioritized = options[:add_specs] && !options[:add_specs].empty?
@@ -303,7 +303,7 @@ def run_specs_frozen(config, options, run_id)
             local_specs = discover_local_specs
             local_specs = filter_specs(local_specs, config.filter) if config.filter && local_specs.any?
             local_specs = filter_by_missing(local_specs, missing_features) if local_specs.any?
-            if local_specs.any? && !json_mode
+            if local_specs.any? && config.verbose && !json_mode
               puts "Auto-including: #{local_specs.size} specs from ./specs/*.yml"
             end
 
@@ -328,10 +328,9 @@ def run_specs_frozen(config, options, run_id)
             known_failures.load_file(options[:known_failures_file])
             known_failures.load_patterns(config.known_failures)
 
-            unless known_failures.empty? || json_mode
+            if !known_failures.empty? && config.verbose && !json_mode
               puts "Known failures: #{known_failures.size} patterns loaded"
             end
-            puts "" unless json_mode
 
             # Check if --bench flag is provided and any suite has timings enabled
             if options[:bench]
@@ -361,7 +360,7 @@ def run_specs_frozen(config, options, run_id)
             # Run explicitly added specs FIRST (--add-specs), before suites
             if prioritized_specs.any?
               add_passed, add_failed, add_errors, add_known_failed, add_known_fixed =
-                run_prioritized_specs(prioritized_specs, known_failures, all_failures, all_known_failures, all_known_fixed, all_passed, results_by_complexity, log_file, run_id, config, quiet: json_mode)
+                run_prioritized_specs(prioritized_specs, known_failures, all_failures, all_known_failures, all_known_fixed, all_passed, results_by_complexity, log_file, run_id, config, quiet: json_mode || !config.verbose)
               total_passed += add_passed
               total_failed += add_failed
               total_errors += add_errors
@@ -387,11 +386,10 @@ def run_specs_frozen(config, options, run_id)
               next if suite_specs.empty?
 
               suite_name_padded = "#{suite.name} ".ljust(40, ".")
-              unless json_mode
+              if config.verbose && !json_mode
                 print("#{suite_name_padded} ")
                 $stdout.flush
               end
-
               passed = 0
               failed = 0
               errors = 0
@@ -446,14 +444,13 @@ def run_specs_frozen(config, options, run_id)
                 end
               end
 
-              unless json_mode
+              if config.verbose && !json_mode
                 parts = ["#{passed}/#{suite_specs.size} passed"]
                 parts << "#{failed} failed" if failed > 0
                 parts << "#{errors} errors" if errors > 0
                 parts << "#{known_failed} known" if known_failed > 0
                 puts parts.join(", ")
               end
-
               total_passed += passed
               total_failed += failed
               total_errors += errors
@@ -465,21 +462,7 @@ def run_specs_frozen(config, options, run_id)
 
             # Show skipped suites
             skipped_suites = skipped_suites_for(config, missing_features)
-            show_skipped_suites(config, missing_features) unless json_mode
-
-            unless json_mode
-              # Print unexpected failures
-              print_failures(all_failures, max_failures)
-
-              # Print known failures (expected)
-              print_known_failures(all_known_failures) if all_known_failures.any?
-
-              # Print passing specs when requested (diagnostic for ramp audits)
-              print_passed_specs(all_passed) if options[:list_passed]
-
-              # Print warnings about known failures that now pass
-              print_known_fixed(all_known_fixed) if all_known_fixed.any?
-            end
+            show_skipped_suites(config, missing_features) if config.verbose && !json_mode
 
             # Calculate max complexity reached (highest level where all specs pass)
             sorted_complexities = results_by_complexity.keys.sort
@@ -515,13 +498,17 @@ def run_specs_frozen(config, options, run_id)
                 skipped_suites: skipped_suites,
               )
             else
-              puts ""
-              parts = ["#{total_passed} passed"]
-              parts << "#{total_failed} failed" if total_failed > 0
-              parts << "#{total_errors} errors" if total_errors > 0
-              parts << "#{total_known_failed} known failures" if total_known_failed > 0
-              parts << "#{total_known_fixed} known now passing" if total_known_fixed > 0
-              puts "Total: #{parts.join(", ")}. Max complexity reached: #{max_complexity_reached}/#{max_possible}"
+              print_run_summary(
+                total_passed: total_passed,
+                total_failed: total_failed,
+                total_errors: total_errors,
+                max_complexity_reached: max_complexity_reached,
+                failures: all_failures,
+                known_fixed: all_known_fixed,
+                passed_specs: all_passed,
+                list_passed: options[:list_passed],
+                max_failures: max_failures
+              )
             end
 
             # Determine exit code
@@ -1417,13 +1404,22 @@ def sort_result_entries(entries)
             end
           end
 
+          # Default plain-mode output: a single summary line followed by the
+          # lowest-complexity failures (the "next best specs to work on").
+          # All preamble/progress/skipped output is verbose-only so stdout
+          # starts with the "Complexity bar cleared" line.
+          def print_run_summary(total_passed:, total_failed:, total_errors:, max_complexity_reached:, failures:, known_fixed:, passed_specs:, list_passed:, max_failures:)
+            failures_count = total_failed + total_errors
+            puts "Complexity bar cleared: #{max_complexity_reached}, #{total_passed} passes, #{failures_count} failures. Next best specs to work on:"
+            puts ""
+            print_failures(failures, max_failures)
+            print_known_fixed(known_fixed) if known_fixed.any?
+            print_passed_specs(passed_specs) if list_passed
+          end
+
           def print_failures(failures, max_failures = nil)
             return if failures.empty?
 
-            puts ""
-            puts "Failures:"
-            puts ""
-
             shown_hints = Set.new
             sorted_failures = sort_result_entries(failures)
 
@@ -1438,8 +1434,8 @@ def print_failures(failures, max_failures = nil)
               location = spec.source_file
               location = "#{location}:#{spec.line_number}" if spec.line_number
               puts "\e[2m#{location}\e[0m"
-
-              puts "#{i + 1}) #{spec.name}"
+              complexity = spec.complexity || 1000
+              puts "#{i + 1}) [c=#{complexity}] #{spec.name}"
               print_labeled_value("Template", spec.template)
 
               # Show environment if present

scripts/generate_spec_hint_baseline.rb

@@ -0,0 +1,34 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Regenerate test/spec_hint_baseline.txt: the frozen list of specs at
+# complexity <= 220 that have no spec-level hint (spec.hint). These are existing
+# generated/bulk specs grandfathered out of the spec-level-hint gate. Any NEW
+# spec at c<=220 without a spec-level hint is caught by the gate (not in this
+# baseline). Re-run after intentionally adding spec-level hints to baseline
+# entries (it shrinks the baseline).
+#
+# Usage: ruby -Ilib scripts/generate_spec_hint_baseline.rb
+
+require "liquid"
+require "liquid/spec"
+require "liquid/spec/suite"
+require "liquid/spec/spec_loader"
+require "liquid/spec/deps/liquid_ruby"
+
+root = File.expand_path("..", __dir__)
+baseline_path = File.join(root, "test", "spec_hint_baseline.txt")
+
+entries = []
+Liquid::Spec::Suite.all.each do |suite|
+  Liquid::Spec::SpecLoader.load_suite(suite).each do |spec|
+    c = spec.complexity || suite.minimum_complexity || 1000
+    next unless c <= 220
+    next unless spec.hint.to_s.strip.empty?
+    rel = spec.source_file.to_s.sub(root + "/", "")
+    entries << "#{rel}\t#{spec.name}"
+  end
+end
+entries.sort!
+File.write(baseline_path, entries.join("\n") + "\n")
+puts "wrote #{entries.size} entries to #{baseline_path}"