Skip to content

Tracking: Search-oriented Nous — find deployable winners, not just confirm/refute (4 children) #166

@sriumcp

Description

@sriumcp

Search-oriented Nous: campaigns must produce a deployable winning artifact, not just CONFIRMED/REFUTED stamps. This tracker covers four additions that together reframe the campaign loop from hypothesis-test (stop on refute) to outcome-search (always emit best-found).

Each is opt-in / schema-additive and adds zero LLM tokens at the engine seam (per CLAUDE.md test policy).

Why now

  • 29-campaign audit (inference-sim/.nous, May 2026): campaigns like mech-design-kvtime, fp-delay-frontier, sgsf-unification REFUTED at iter-1 and walked away producing nothing — even though the simulator runs that produced the refutation contained signal about which configs were closer to deployable.
  • User push-back (2026-05-25, trading-strategy campaign): "the objective here is not merely confirming and refuting hypothesis for the sake of it. The objective here is, actually find a really good structure that we can profit from."
  • User-endorsed addition (same conversation): cheap deterministic pre-work before iter-1 to inform iteration structure — "30 minutes of exploration could give the campaign a much better starting point than throwing it at the LLM cold."

Children

Single-PR landing

One branch off upstream/reflective, four feat commits (one per child, each Closes #<child>), tracking PR title Tracking #<this>: Search-oriented Nous (4 children). Mirrors #153/#161/#162.

Ordering rationale (A → B → C → D):

  • A (pre-work) is independent and lowest risk — pure additive phase.
  • B (best_found) needs the same findings.json shape A doesn't change.
  • C (engine) needs B's best_found.json to redirect search toward.
  • D (deployment recommendation) reads B's best_found.json + C's expanded ledger.

Relationship to #162

Independent. #162 makes existing methodology rigorous (power, posteriors, sweeps); this tracker makes the methodology outcome-oriented. No file/schema overlap; can land in either order. Recommend #162 first (already drafted, lower risk) then this.

/goal predicate (for the tracking issue)

A PR exists with base upstream/reflective and head sriumcp:<branch> AND the working tree of that PR satisfies ALL of:

  • orchestrator/pre_work.py exists and exports run_pre_work accepting runner=
  • schemas/state.schema.json accepts PRE_WORK as a phase value
  • orchestrator/composite_score.py exists and exports compute_score
  • schemas/campaign.schema.yaml accepts an optional objective block
  • schemas/best_found.schema.json exists
  • engine.py no longer routes REFUTE to DONE (covered by a fixture replay test)
  • schemas/meta_findings.schema.json includes a required deployment_recommendation field
  • tests/test_pre_work.py, tests/test_composite_score.py, tests/test_engine_search_continuation.py, tests/test_deployment_recommendation.py all exist
  • pytest -q exit code is 0
  • the PR body references Closes #167, Closes #168, Closes #169, Closes #170 and Refs #<this>

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions