diff --git a/.claude/commands/suggest-next.md b/.claude/commands/suggest-next.md new file mode 100644 index 0000000..e22a595 --- /dev/null +++ b/.claude/commands/suggest-next.md @@ -0,0 +1,344 @@ +Given a user's research intent, retrieve prior knowledge from the cross-campaign registry and recommend how to frame a new campaign. + +## Usage + +`/suggest-next ` + +Examples: +- `/suggest-next /path/to/inference-sim "improve admission control fairness across priority bands"` +- `/suggest-next inference-sim "reduce tail latency under burst workloads"` +- `/suggest-next` (no arguments — list available projects and ask) + +## Argument Parsing + +- If `$ARGUMENTS` is empty, read `~/.nous/wiki/registry.json`, list all projects (by name and path), and ask the user which project and what their research intent is. +- If `$ARGUMENTS` starts with a path (contains `/`) or matches a project name in the registry, use it as the project filter. Everything after it is the intent. +- If `$ARGUMENTS` doesn't match any project, treat the entire argument as the intent and ask the user which project to use. + +## Algorithm + +The algorithm has five phases: **A: Retrieval** (script-driven, deterministic), **B: Synthesis** (LLM reasoning over the retrieved context), **C: Output** (write markdown), **D: Format** (file structure), and **E: Campaign Generation** (interactive YAML creation). The LLM selects what to retrieve; the script does the mechanical graph traversal and filtering. + +--- + +### Phase A: Retrieval + +#### A1. Load Registry and Match Project + +Read `~/.nous/wiki/registry.json`. Find the project entry matching the user's repo path or project name: +- Try exact path match against `projects` keys +- Try substring match (user might give just the repo name, match against the end of each key) +- Try fuzzy match against project `name` fields + +If not found, report: "No prior knowledge for this system. Available projects:" and list them. **STOP.** + +#### A2. Select Campaigns and Entities (LLM judgment) + +From the matched project's registry entry, select: + +- **Exactly 3 campaign names** (or all campaigns if fewer than 3 exist) — rank by relevance to the user's intent using `research_question`, `concepts[].name`, and `frontiers[].title` +- **Exactly 6 entity names** (or all entities if fewer than 6 exist) — from the project-level `entities` array, pick those whose `name` or `aliases` relate to the user's intent. Also include entities that appear in the selected campaigns if their role is relevant. + +#### A3. Run Retrieval Script + +Call the retrieval script with the selected campaigns and entities: + +```bash +python scripts/retrieve_wiki_context.py \ + -c ... \ + -e "" "" ... \ + -i "" +``` + +The script: +1. Builds a knowledge graph from each campaign's `concepts.json` (nodes = entities/concepts/parameters, edges = shared principles) +2. Extracts the subgraph reachable from the specified entities (1-hop via principle overlap) +3. Loads principles from `principles.json` — only those referenced by the subgraph +4. Loads all dead-ends from `dead-ends.json` +5. Loads frontiers and interactions filtered by the scoped principle IDs +6. Outputs a structured context block to stdout + +#### A4. Read Script Output + +Capture the script's stdout. This is the **Retrieved Context** block that feeds Phase B. + +--- + +### Phase B: Synthesis + +Using the assembled context block from Phase A, generate **top 3 recommended campaign framings**. For each recommendation: + +1. **Score it** on five dimensions: + - **Novelty (weight 0.25):** How far is this from known dead-ends? Does it explore genuinely new territory? + - **Foundation (weight 0.20):** How many scoped principles does it build upon? Stronger foundation = higher confidence. + - **Impact (weight 0.25):** Based on related results, what's the estimated effect size? Prioritize high-impact experiments. + - **Testability (weight 0.15):** Can this be validated in a single campaign run? Concrete, bounded experiments score higher. + - **Efficiency (weight 0.15):** How cost-effective is this experiment predicted to be? Score based on: + - Predicted cost relative to predicted impact (low cost + high impact = high efficiency) + - Whether the experiment can reuse cached context from prior runs (cache reads reduce cost) + - Whether a cheaper model configuration could work (e.g., Sonnet-only for refinement campaigns vs Opus+Sonnet for exploratory) + - Fewer predicted iterations = higher efficiency + +2. **For each recommendation, provide:** + - A suggested `research_question` (1-2 sentences, phrased as a testable question) + - Which entities/concepts from the context block it builds on (with brief context) + - Which frontiers it addresses (by ID and title) + - Which interactions it could test (by ID and title) + - Which dead-ends to explicitly avoid (by ID and brief reason) + - Score breakdown (Novelty/Foundation/Impact/Testability/Efficiency + weighted total) + - Predicted cost (iterations × cost/iter, with basis for estimate) + - Suggested model configuration (which models for design/execute phases, with rationale) + +### Phase C: Output File + +Write the full recommendation to a markdown file at: + +``` +~/.nous/wiki/suggestions/-.md +``` + +- Create the `~/.nous/wiki/suggestions/` directory if it doesn't exist. +- Slugify the intent: lowercase, replace spaces with `-`, strip non-alphanumeric characters, truncate to 50 chars. +- If the file already exists (same date + intent), append a numeric suffix (`-2`, `-3`, etc.). + +After writing the file, print a short summary to the terminal: + +``` +Wrote: ~/.nous/wiki/suggestions/.md + +Top recommendations: + 1. — score: <total>/1.0 + 2. <title> — score: <total>/1.0 + 3. <title> — score: <total>/1.0 +``` + +### Phase D: File Format + +The markdown file should follow this structure. The scoring table is **required** for every recommendation — it is the primary decision-making artifact. + +```markdown +# Suggest-Next: <project name> + +**Date:** <YYYY-MM-DD> +**Research intent:** "<user's intent>" +**Prior campaigns:** <count> +**Total confirmed principles:** <count> +**Campaigns consulted:** <comma-separated names> +**Entities scoped:** <comma-separated names> + +--- + +## Scoring Summary + +| # | Recommendation | Novelty | Foundation | Impact | Testability | Efficiency | **Total** | +|---|---------------|---------|-----------|--------|-------------|------------|-----------| +| 1 | <short title> | X.XX | X.XX | X.XX | X.XX | X.XX | **X.XX** | +| 2 | <short title> | X.XX | X.XX | X.XX | X.XX | X.XX | **X.XX** | +| 3 | <short title> | X.XX | X.XX | X.XX | X.XX | X.XX | **X.XX** | + +*Weights: Novelty 0.25, Foundation 0.20, Impact 0.25, Testability 0.15, Efficiency 0.15* + +--- + +## Recommendation 1: <short title> + +**Suggested research question:** +> <1-2 sentence testable question> + +### Score Breakdown + +**Weighted total: X.XX/1.0** + +| Dimension | Weight | Score | Rationale | +|-------------|--------|-------|-----------| +| Novelty | 0.25 | X.XX | <brief — what makes this novel or not> | +| Foundation | 0.20 | X.XX | <brief — which principles it builds on> | +| Impact | 0.25 | X.XX | <brief — expected effect size and why> | +| Testability | 0.15 | X.XX | <brief — how bounded/measurable it is> | +| Efficiency | 0.15 | X.XX | <brief — cost/impact ratio reasoning> | + +### Builds on +- <Entity/Concept name> — <how it's relevant> +- ... + +### Addresses frontiers +- F-N: <title> — <how this experiment would push the boundary> +- ... + +### Tests interactions +- I-N: <title> — <what combining these would reveal> +- ... + +### Avoid (dead-ends) +- DE-N: <title> — <why this failed before> +- ... + +### Predicted cost + +| Metric | Estimate | Basis | +|--------|----------|-------| +| Iterations | N-M | <reasoning: refinement/exploratory, builds on N principles, etc.> | +| Cost/iter | ~$X.XX | Project historical average (adjusted if applicable) | +| Total | $XX-YY | iterations × cost/iter | +| Duration | ~Xh | Based on avg duration/iter from similar campaigns | + +### Model configuration +- Design phase: <model> (<rationale>) +- Execute phase: <model> (<rationale>) +- Alternative: <cheaper/costlier option with savings estimate> + +**Efficiency note:** <1 sentence on why this cost is justified relative to expected impact> + +--- + +## Recommendation 2: <short title> + +<same structure as Recommendation 1> + +--- + +## Recommendation 3: <short title> + +<same structure as Recommendation 1> + +--- + +## Next Steps + +To start a campaign from these recommendations, use the interactive generator below or manually: +1. Select recommendations to generate `campaign.yaml` files (Phase E prompt follows) +2. Review and adjust the generated config if needed +3. Run: `nous run <path-to-campaign.yaml>` +4. After completion, run `/post-campaign` to feed results back into the registry +``` + +### Phase E: Interactive Campaign Generation + +After printing the terminal summary (end of Phase C), offer to generate executable `campaign.yaml` files from the recommendations. + +#### E1. Ask the User + +Use AskUserQuestion to present choices: + +**Question:** "Which recommendations would you like to generate campaign.yaml files for?" + +**Options:** +- "1" — Generate for recommendation 1 only +- "2" — Generate for recommendation 2 only +- "3" — Generate for recommendation 3 only +- "All" — Generate for all recommendations +- "None" — Skip campaign generation + +Allow multi-select (the user can pick e.g. "1" and "3"). + +If the user selects "None", print `No campaigns generated.` and **STOP**. + +#### E2. Generate campaign.yaml for Each Selected Recommendation + +For each selected recommendation, produce a YAML document with these field mappings: + +| campaign.yaml field | Source | +|---|---| +| `research_question` | Recommendation's suggested research question (verbatim from the `> <question>` block) | +| `run_id` | Slugified recommendation title (lowercase, hyphens, ≤50 chars) | +| `max_iterations` | Upper bound from the "Iterations" row in the Predicted cost table (e.g., "6-8" → 8) | +| `target_system.name` | From registry `projects[key].name` | +| `target_system.description` | Synthesized from registry project description + recommendation context | +| `target_system.repo_path` | The project key (path) from the registry | +| `target_system.observable_metrics` | Inferred from recommendation's Impact rationale (omit field entirely if not confidently inferable) | +| `target_system.controllable_knobs` | Parameter names from "Builds on" section (omit field entirely if not confidently inferable) | +| `prompts.methodology_layer` | `"prompts/methodology"` (standard default) | +| `prompts.domain_adapter_layer` | `null` | +| `models.design` | From recommendation's "Model configuration → Design phase" model name | +| `models.execute_analyze` | From recommendation's "Model configuration → Execute phase" model name | +| `metadata` | Traceability block (see E3) | + +**Schema compliance rules:** +- Do NOT include any fields not in `orchestrator/schemas/campaign.schema.yaml` +- Root object: only `research_question`, `run_id`, `max_iterations`, `target_system`, `prompts`, `models`, `metadata` +- `target_system`: only `name`, `description`, `repo_path`, `observable_metrics`, `controllable_knobs`, `live_target` +- `prompts`: only `methodology_layer`, `domain_adapter_layer` +- `models`: only `design`, `execute_analyze`, `report` +- Omit optional fields rather than including empty values +- Model values default: `claude-opus-4-6` (design), `claude-sonnet-4-6` (execute_analyze) + +#### E3. Metadata Traceability Block + +Include a `metadata` section for provenance tracking: + +```yaml +metadata: + source_suggestion: "<YYYY-MM-DD>-<slug>.md" + recommendation_rank: <1|2|3> + research_intent: "<user's original intent verbatim>" + builds_on_frontiers: ["F-1", "F-3"] + tests_interactions: ["I-2"] + avoids_dead_ends: ["DE-1", "DE-4"] + foundation_principles: ["RP-5", "RP-12"] + composite_score: 0.XX +``` + +- Use the actual IDs from the recommendation's "Addresses frontiers", "Tests interactions", "Avoid (dead-ends)" sections +- `foundation_principles`: principle IDs referenced in the Foundation score rationale +- `composite_score`: the weighted total from the scoring table + +#### E4. Write Files + +Write each generated YAML to: + +``` +~/.nous/wiki/suggestions/campaigns/<YYYY-MM-DD>-<slugified-intent>-<N>.yaml +``` + +Where `<N>` is the recommendation number (1, 2, or 3). + +- The `<YYYY-MM-DD>-<slugified-intent>` prefix matches the suggestion markdown filename (without `.md`) +- If the file already exists, append a numeric suffix before `.yaml` (e.g., `-1-2.yaml`) +- Create `~/.nous/wiki/suggestions/campaigns/` if it doesn't exist + +#### E5. Print Execution Instructions + +After writing all campaign files, print: + +``` +Generated campaign files: + <N>. ~/.nous/wiki/suggestions/campaigns/<filename>.yaml + Run: nous run <full-path> + + ... +``` + +Example: +``` +Generated campaign files: + 1. ~/.nous/wiki/suggestions/campaigns/2026-06-03-improve-fairness-1.yaml + Run: nous run ~/.nous/wiki/suggestions/campaigns/2026-06-03-improve-fairness-1.yaml + 3. ~/.nous/wiki/suggestions/campaigns/2026-06-03-improve-fairness-3.yaml + Run: nous run ~/.nous/wiki/suggestions/campaigns/2026-06-03-improve-fairness-3.yaml +``` + +--- + +## Model Configuration Guidance + +When suggesting models for a recommendation, use the **Cost Context** section from the retrieved context and apply these heuristics: + +- **Opus design + Sonnet execute** (default): For campaigns exploring new territory, combining multiple approaches, or where the design phase needs to reason about complex interactions. Historical cost: ~$5.50-6.00/iter. +- **Sonnet design + Sonnet execute** (cheaper, ~45% savings): For campaigns that are narrow refinements of known-good configurations — the design space is well-constrained by prior principles. Historical cost: ~$3.00-3.50/iter estimate. +- **Opus both** (expensive, ~80% increase): Only for campaigns that need deep analysis in the execute phase (e.g., debugging subtle failures where Sonnet might miss root causes). Historical cost: ~$10-11/iter estimate. + +Iteration count heuristics: +- **Refinement** (builds on 3+ confirmed principles, narrow scope): 4-6 iterations +- **Exploratory** (new territory, tests interactions, <2 confirmed principles to build on): 8-12 iterations +- **Standard** (mix of known and new): 6-8 iterations + +## Important Rules + +- This skill **writes files only to `~/.nous/wiki/suggestions/`** — the suggestion markdown at the top level, and optionally campaign YAML files in the `campaigns/` subdirectory. It never modifies registry files, campaign data, or any other existing files. +- All reasoning happens in-context using the LLM's judgment — no external scripts beyond `retrieve_wiki_context.py`. +- If the registry is empty or the project has no campaigns, say so clearly and suggest the user run their first campaign manually. +- Always ground recommendations in specific prior data (principle IDs, frontier IDs, dead-end IDs). Never hallucinate IDs that don't exist in the loaded files. +- Keep recommendations actionable — each should be concrete enough to immediately write a `campaign.yaml` from. +- Prefer recommendations that combine insights from multiple campaigns over those that just extend a single campaign. +- Always use the Cost Context section to ground cost predictions in real data — never invent cost numbers without historical basis. +- **Scoring transparency is non-negotiable** — every recommendation must include its full score breakdown table with per-dimension rationale. The summary table at the top lets users compare at a glance. diff --git a/docs/nous-wiki.md b/docs/nous-wiki.md index 3667014..d313e27 100644 --- a/docs/nous-wiki.md +++ b/docs/nous-wiki.md @@ -120,6 +120,48 @@ Merges a single campaign's extracted knowledge into the cross-campaign registry. --- +### `/suggest-next` + +Retrieves prior knowledge from the cross-campaign registry and recommends how +to frame a new campaign. Optionally generates executable `campaign.yaml` files. + +**Usage:** + +``` +/suggest-next /path/to/repo "research intent" +/suggest-next inference-sim "reduce tail latency under burst workloads" +/suggest-next # lists available projects and asks +``` + +**Prerequisites:** The project must have at least one campaign indexed via +`/index-wiki` (i.e., it must exist in `registry.json`). + +**What it reads:** + +| Source | What it uses | +|--------|--------------| +| `~/.nous/wiki/registry.json` | Project matching, campaign selection, entity selection | +| Campaign wiki files (via `retrieve_wiki_context.py`) | Principles, dead-ends, frontiers, interactions, concepts | + +**What it writes:** + +| File | Contents | +|------|----------| +| `~/.nous/wiki/suggestions/<date>-<slug>.md` | Scored recommendations with research questions, cost predictions, model configs | +| `~/.nous/wiki/suggestions/campaigns/<date>-<slug>-<N>.yaml` | Nous-compatible campaign configs (optional, user-selected) | + +**Algorithm:** +1. **Phase A (Retrieval):** Matches project in registry, selects 3 campaigns and 6 entities, runs `retrieve_wiki_context.py` for subgraph extraction +2. **Phase B (Synthesis):** Scores 3 recommendations on Novelty, Foundation, Impact, Testability, Efficiency +3. **Phase C (Output):** Writes suggestion markdown +4. **Phase D (Format):** Structures the markdown with scoring tables and per-recommendation detail +5. **Phase E (Campaign Generation):** Asks which recommendations to turn into `campaign.yaml` files, writes schema-valid YAML to `suggestions/campaigns/` + +**What doesn't happen:** This skill never modifies registry files, campaign +wiki data, or any other existing files. + +--- + ## Output Data Model All output lives under `~/.nous/wiki/` — a user-level directory outside any @@ -138,6 +180,10 @@ repo. Each campaign gets its own subdirectory. │ ├── interactions.json │ ├── llm_metrics.jsonl │ └── summary.md +├── suggestions/ # Written by /suggest-next +│ ├── <date>-<slug>.md # Scored recommendation reports +│ └── campaigns/ # Generated campaign configs +│ └── <date>-<slug>-<N>.yaml └── viz/ └── <campaign-name>.html ``` diff --git a/scripts/retrieve_wiki_context.py b/scripts/retrieve_wiki_context.py new file mode 100644 index 0000000..ba2dbcb --- /dev/null +++ b/scripts/retrieve_wiki_context.py @@ -0,0 +1,387 @@ +#!/usr/bin/env python3 +"""Retrieve structured context from the Nous wiki for a given research intent. + +This script implements the deterministic retrieval pipeline for /suggest-next. +The LLM picks campaign names and entity names; this script does direct lookups +on explicit relationship fields in concepts.json to produce a structured context block. + +Usage: + python scripts/retrieve_wiki_context.py \ + --campaigns epp-ttft-slope-detector epp-saturation-detector-archive-20260519-115428 \ + --entities "UtilizationDetector" "GatewayQueue" \ + --intent "improve admission control fairness across priority bands" + +Output: A structured markdown context block printed to stdout. +""" + +import argparse +import json +import sys +from pathlib import Path + + +def load_json(path: Path) -> dict | list | None: + """Load a JSON file, returning None if it doesn't exist or can't be parsed.""" + if not path.exists(): + return None + try: + with open(path) as f: + return json.load(f) + except json.JSONDecodeError as e: + print(f"Warning: {path} exists but contains invalid JSON: {e}", file=sys.stderr) + return None + except OSError as e: + print(f"Warning: {path} exists but could not be read: {e}", file=sys.stderr) + return None + + +def load_cost_context(wiki_dir: Path, campaign_names: list[str]) -> dict: + """Compute cost stats from llm_metrics.jsonl for each campaign.""" + results = {} + for name in campaign_names: + metrics_path = wiki_dir / "campaigns" / name / "llm_metrics.jsonl" + if not metrics_path.exists(): + continue + entries = [] + with open(metrics_path) as f: + for line_num, line in enumerate(f, 1): + line = line.strip() + if not line: + continue + try: + entries.append(json.loads(line)) + except json.JSONDecodeError: + print( + f"Warning: {metrics_path}:{line_num} — skipping malformed JSON line", + file=sys.stderr, + ) + + # Only count real cost-bearing entries (planner:design + executor:execute-analyze) + design_entries = [e for e in entries if e.get("phase") == "design" and e.get("cost_usd")] + execute_entries = [e for e in entries if e.get("phase") == "execute-analyze" and e.get("cost_usd")] + + design_cost = sum(e.get("cost_usd", 0) for e in design_entries) + execute_cost = sum(e.get("cost_usd", 0) for e in execute_entries) + total_cost = design_cost + execute_cost + n_iters = max(len(design_entries), len(execute_entries)) + + results[name] = { + "total_cost_usd": round(total_cost, 2), + "iterations": n_iters, + "cost_per_iteration": round(total_cost / n_iters, 2) if n_iters else 0, + "design_cost_usd": round(design_cost, 2), + "execute_cost_usd": round(execute_cost, 2), + "design_model": design_entries[0].get("model") if design_entries else None, + "execute_model": execute_entries[0].get("model") if execute_entries else None, + "avg_design_turns": round(sum(e.get("num_turns", 0) for e in design_entries) / len(design_entries)) if design_entries else 0, + "avg_execute_turns": round(sum(e.get("num_turns", 0) for e in execute_entries) / len(execute_entries)) if execute_entries else 0, + "total_duration_hours": round(sum(e.get("duration_ms", 0) for e in design_entries + execute_entries) / 3_600_000, 1), + } + return results + + +def retrieve_context( + wiki_dir: Path, + campaign_names: list[str], + entity_names: list[str], + intent: str, +) -> str: + """Run the retrieval pipeline and return the structured context block. + + Uses explicit relationship fields in concepts.json for direct lookups: + 1. Match entities by name + 2. Find concepts whose operates_on includes matched entities + 3. Find parameters whose parent_concept matches found concepts + 4. Collect scoped principles from all matched items + 5. Filter frontiers/interactions by scoped principles + """ + entity_names_lower = {n.lower() for n in entity_names} + + all_campaigns_info = [] + all_entities = [] + all_concepts = [] + all_parameters = [] + all_principles = [] + all_dead_ends = [] + all_frontiers = [] + all_interactions = [] + + seen_entity_names = set() + seen_concept_names = set() + seen_param_names = set() + seen_principle_ids = set() + + loaded_count = 0 + for campaign_name in campaign_names: + campaign_dir = wiki_dir / "campaigns" / campaign_name + + if not campaign_dir.exists(): + print(f"Warning: campaign '{campaign_name}' not found at {campaign_dir}", file=sys.stderr) + continue + + concepts_data = load_json(campaign_dir / "concepts.json") + if not concepts_data: + print(f"Warning: campaign '{campaign_name}' has no usable concepts.json, skipping", file=sys.stderr) + continue + + loaded_count += 1 + + all_campaigns_info.append({ + "name": campaign_name, + "research_question": concepts_data.get("research_question", ""), + "date": concepts_data.get("date", ""), + }) + + # 1. Match entities by name + matched_entity_names = set() + for entity in concepts_data.get("entities", []): + name = entity.get("name", "") + if name.lower() in entity_names_lower: + matched_entity_names.add(name) + if name not in seen_entity_names: + seen_entity_names.add(name) + all_entities.append(entity) + + if not matched_entity_names: + continue + + # 2. Find concepts that operate on matched entities + matched_concept_names = set() + for concept in concepts_data.get("concepts", []): + concept_name = concept.get("name", "") + operates_on = concept.get("operates_on", []) + if any(e in matched_entity_names for e in operates_on): + matched_concept_names.add(concept_name) + if concept_name not in seen_concept_names: + seen_concept_names.add(concept_name) + all_concepts.append(concept) + # Also include entities referenced by operates_on (neighbors) + for e_name in operates_on: + if e_name not in matched_entity_names and e_name not in seen_entity_names: + # Find this entity in the data + for ent in concepts_data.get("entities", []): + if ent.get("name") == e_name: + seen_entity_names.add(e_name) + all_entities.append(ent) + break + + # 3. Find parameters belonging to matched concepts + for param in concepts_data.get("parameters", []): + param_name = param.get("name", "") + if param.get("parent_concept") in matched_concept_names: + if param_name not in seen_param_names: + seen_param_names.add(param_name) + all_parameters.append(param) + + # 4. Collect scoped principle IDs from all matched items + scoped_principle_ids = set() + for entity in concepts_data.get("entities", []): + if entity.get("name", "") in seen_entity_names: + scoped_principle_ids.update(entity.get("principles", [])) + for concept in concepts_data.get("concepts", []): + if concept.get("name", "") in matched_concept_names: + scoped_principle_ids.update(concept.get("principles", [])) + for param in concepts_data.get("parameters", []): + if param.get("name", "") in seen_param_names: + scoped_principle_ids.update(param.get("principles", [])) + + # Load principles.json, filter to scoped IDs + principles_data = load_json(campaign_dir / "principles.json") + if principles_data: + for p in principles_data.get("principles", []): + pid = p.get("id", "") + if pid in scoped_principle_ids and pid not in seen_principle_ids: + seen_principle_ids.add(pid) + all_principles.append(p) + + # Load dead-ends (all) + dead_ends = load_json(campaign_dir / "dead-ends.json") + if dead_ends and isinstance(dead_ends, list): + all_dead_ends.extend(dead_ends) + + # Load frontiers (filtered by scoped principles) + frontiers = load_json(campaign_dir / "frontiers.json") + if frontiers and isinstance(frontiers, list): + for f in frontiers: + related = set(f.get("related_principles", [])) + if related & scoped_principle_ids: + all_frontiers.append(f) + + # Load interactions (filtered by scoped principles) + interactions = load_json(campaign_dir / "interactions.json") + if interactions and isinstance(interactions, list): + for i in interactions: + related = set(i.get("related_principles", [])) + if related & scoped_principle_ids: + all_interactions.append(i) + + if loaded_count == 0: + print( + f"Error: none of the requested campaigns could be loaded: {campaign_names}", + file=sys.stderr, + ) + sys.exit(1) + + cost_context = load_cost_context(wiki_dir, campaign_names) + + return _format_context_block( + intent=intent, + campaigns=all_campaigns_info, + entities=all_entities, + concepts=all_concepts, + parameters=all_parameters, + principles=all_principles, + dead_ends=all_dead_ends, + frontiers=all_frontiers, + interactions=all_interactions, + cost_context=cost_context, + ) + + +def _format_context_block( + intent: str, + campaigns: list, + entities: list, + concepts: list, + parameters: list, + principles: list, + dead_ends: list, + frontiers: list, + interactions: list, + cost_context: dict | None = None, +) -> str: + """Format all retrieved data into the structured context block.""" + lines = [] + lines.append("## Retrieved Context\n") + + # Research Problem + lines.append("### Research Problem") + lines.append(intent) + lines.append("") + + # Selected Campaigns + lines.append(f"### Selected Campaigns ({len(campaigns)})") + for c in campaigns: + lines.append(f"- **{c['name']}**: {c['research_question']} ({c['date']})") + lines.append("") + + # Matched Entities + lines.append(f"### Matched Entities ({len(entities)})") + for e in entities: + principles_str = ", ".join(e.get("principles", [])) + lines.append(f"- **{e.get('name', '?')}** — {e.get('definition', '')} [principles: {principles_str}]") + lines.append("") + + # Related Concepts + lines.append(f"### Related Concepts ({len(concepts)})") + for c in concepts: + principles_str = ", ".join(c.get("principles", [])) + lines.append(f"- **{c.get('name', '?')}** — {c.get('definition', '')} [principles: {principles_str}]") + lines.append("") + + # Related Parameters + lines.append(f"### Related Parameters ({len(parameters)})") + for p in parameters: + principles_str = ", ".join(p.get("principles", [])) + evo_str = "" + if p.get("evolution"): + evo_parts = [f"{e.get('iter', '?')}={e.get('value', '?')} ({e.get('outcome', '?')})" for e in p["evolution"]] + evo_str = f" [evolution: {'; '.join(evo_parts)}]" + lines.append(f"- **{p.get('name', '?')}** — {p.get('definition', '')} [principles: {principles_str}]{evo_str}") + lines.append("") + + # Scoped Principles + lines.append(f"### Scoped Principles ({len(principles)})") + for p in principles: + confidence = p.get("confidence", "unknown") + regime = p.get("regime", "") + regime_str = f" | regime: {regime}" if regime else "" + lines.append(f"- **{p['id']}** ({confidence}): {p['statement']}{regime_str}") + lines.append("") + + # Dead-Ends + lines.append(f"### Dead-Ends ({len(dead_ends)})") + for d in dead_ends: + lines.append(f"- **{d.get('id', '?')}**: {d.get('title', '')} — tried: {d.get('what_was_tried', '')} | failed: {d.get('why_it_failed', '')} | avoid when: {d.get('avoid_when', '')}") + lines.append("") + + # Frontiers + lines.append(f"### Frontiers ({len(frontiers)})") + for f in frontiers: + lines.append(f"- **{f.get('id', '?')}**: {f.get('title', '')} — untried: {f.get('what_was_left_untried', '')} | try next: {f.get('what_to_try_next', '')}") + lines.append("") + + # Interactions + lines.append(f"### Interactions ({len(interactions)})") + for i in interactions: + lines.append(f"- **{i.get('id', '?')}**: {i.get('title', '')} — A: {i.get('approach_a', '')} | B: {i.get('approach_b', '')} | why: {i.get('why_combine', '')} | experiment: {i.get('experiment_to_run', '')}") + lines.append("") + + # Cost Context + if cost_context: + lines.append(f"### Cost Context ({len(cost_context)} campaigns with metrics)") + lines.append("| Campaign | Iters | Total Cost | $/Iter | Design Model | Execute Model |") + lines.append("|----------|-------|-----------|--------|--------------|---------------|") + total_cost_all = 0.0 + total_iters_all = 0 + total_design = 0.0 + total_execute = 0.0 + for name, stats in cost_context.items(): + lines.append( + f"| {name} | {stats['iterations']} | ${stats['total_cost_usd']:.2f} " + f"| ${stats['cost_per_iteration']:.2f} | {stats['design_model'] or '—'} " + f"| {stats['execute_model'] or '—'} |" + ) + total_cost_all += stats["total_cost_usd"] + total_iters_all += stats["iterations"] + total_design += stats["design_cost_usd"] + total_execute += stats["execute_cost_usd"] + lines.append("") + avg_per_iter = total_cost_all / total_iters_all if total_iters_all else 0 + design_pct = round(100 * total_design / total_cost_all) if total_cost_all else 0 + execute_pct = 100 - design_pct + lines.append(f"**Project averages:** ${avg_per_iter:.2f}/iter, {design_pct}% design / {execute_pct}% execute split") + lines.append(f"**Total project research investment:** ${total_cost_all:.2f} across {total_iters_all} iterations") + lines.append("") + + return "\n".join(lines) + + +def main(): + parser = argparse.ArgumentParser( + description="Retrieve structured context from Nous wiki for research planning" + ) + parser.add_argument( + "--campaigns", "-c", nargs="+", required=True, + help="Campaign names to retrieve from" + ) + parser.add_argument( + "--entities", "-e", nargs="+", required=True, + help="Entity names to seed the graph traversal" + ) + parser.add_argument( + "--intent", "-i", default="", + help="Research intent (included in context block header)" + ) + parser.add_argument( + "--wiki-dir", "-w", default=str(Path.home() / ".nous" / "wiki"), + help="Path to wiki directory (default: ~/.nous/wiki/)" + ) + args = parser.parse_args() + + wiki_dir = Path(args.wiki_dir) + if not wiki_dir.exists(): + print(f"Error: wiki directory not found: {wiki_dir}", file=sys.stderr) + sys.exit(1) + + context = retrieve_context( + wiki_dir=wiki_dir, + campaign_names=args.campaigns, + entity_names=args.entities, + intent=args.intent, + ) + print(context) + + +if __name__ == "__main__": + main()