Phase D: Conversations + Social Posts (#66)

* feat: Phase D - agent conversations and narrative prompts

- Add conversation system for agent-agent and agent-NPC exchanges
- Add --fidelity flag: low (no convs), medium (2 turns), high (3 turns)
- Add talk_to action type to reasoning schema
- Add relationship_weights to ScenarioSpec for conversation priority
- Add conversations table to study DB with helper methods
- Integrate conversation phase into engine timestep loop
- Add available_contacts to reasoning prompts
- Export conversation stats and conversations.json in results

* feat: add social_posts table and helper methods to StudyDB

* feat: record social posts from sharing agents during timestep execution

* feat: add social feed to reasoning context and prompts

- Add social_feed field to ReasoningContext
- Build feed from recent posts beyond agent's direct network
- Render 'What People Are Saying Online' section in prompts

* feat: export social_posts.json in results

* docs: add Phase D conversations and social posts to capabilities

* docs: mark DayPhases as omitted in v2 architecture

* fix: remove unnecessary async markers from fidelity tests

* fix: remove unused pytest import

Author: DeveshParagiri

docs/capabilities.md

@@ -234,6 +234,41 @@ After simulation runs, you have:
 
 ---
 
+## Agent Conversations
+
+Agents can talk to each other. When reasoning, an agent can choose to initiate a conversation with someone in their network.
+
+**Talk-to actions**: During reasoning, agents can request to talk to someone - a coworker, neighbor, partner, or family member. The request includes who they want to talk to and what topic they want to discuss.
+
+**Multi-turn exchanges**: Conversations are full back-and-forth dialogues. The initiator speaks, the target responds, they go back and forth for 2-3 turns depending on fidelity settings. Each turn is a complete LLM call with the speaker's persona and context.
+
+**State updates from conversations**: Conversations change minds. After talking, both participants get updated sentiment, conviction, and potentially position. These conversation-driven state changes override provisional reasoning state - a compelling conversation can shift someone more than passive exposure.
+
+**Partner and NPC conversations**: Agents can talk to their partner (whether another full agent or an NPC spouse) or household dependents. NPC conversations work the same way, but only the initiator's state updates - NPCs don't have persistent state.
+
+**Conflict resolution**: When multiple agents want to talk to the same target, priority determines who wins. Higher relationship weight wins - a partner request beats a coworker request. Deferred requests can execute in later timesteps.
+
+**Fidelity control**: The `--fidelity` flag controls conversation depth:
+- `low`: No conversations at all - just reasoning
+- `medium` (default): 2 turns (4 messages), top 1 conversation per agent
+- `high`: 3 turns (6 messages), up to 2 conversations per agent
+
+---
+
+## Social Feed and Public Discourse
+
+Agents don't just see their direct network neighbors. They perceive broader public discourse.
+
+**Public statements become posts**: When agents share (set `will_share=True`), their public statement gets recorded as a social post. These accumulate over timesteps, creating a timeline of what people are saying.
+
+**Social feed beyond network**: In subsequent timesteps, agents see a "What People Are Saying Online" section in their prompts. This shows recent posts from the broader population - not just direct neighbors. It's like seeing trending takes on social media from strangers.
+
+**Network vs. public**: Peer opinions come from direct network neighbors. The social feed comes from everyone else who's sharing. Agents experience both - personal network influence and ambient public discourse.
+
+**Configurable lookback**: The feed pulls from recent timesteps (default: 3 timesteps back, 5 posts shown). This models the recency bias of social media - you see what's trending now, not old takes.
+
+---
+
 ## Scenarios You Can Run Today
 
 To make it concrete, here are scenarios that work right now with no additional development:
@@ -249,12 +284,13 @@ To make it concrete, here are scenarios that work right now with no additional d
 - Professional networks processing industry disruption news
 - Religious communities responding to doctrinal changes
 - Parent networks reacting to school policy updates
+- Couples having conversations that shift their positions
+- Workplace discussions that change minds
+- Social media dynamics where public discourse influences individuals
 - Any population, any country, any event, any outcome structure
 
 The constraints are:
-- No agent-to-agent conversations (yet)
-- No agents creating public social posts (yet)
-- No runtime fidelity/cost tradeoffs beyond merged pass (yet)
+- No runtime fidelity/cost tradeoffs beyond merged pass and fidelity levels (yet)
 - No validation against historical ground truth (yet)
 
-Those are Phases D, E, F, and G. What's here now is Phases A, B, and C - the core simulation engine with households, networks, timelines, and reasoning.
+Those are Phases E and F. What's here now is Phases A through D - the core simulation engine with households, networks, timelines, conversations, and social posts.

docs/simulation-v2-architecture.md

@@ -339,55 +339,9 @@ Each timeline event:
 
 For scenarios with no evolution (Netflix password sharing), the timeline is just the single t=0 event. No extra configuration needed.
 
-#### Day Phases (New, Optional)
+#### ~~Day Phases~~ (OMITTED)
 
-Templates that structure the agent's timestep experience into life contexts. Optional — if omitted, the engine uses an improved flat prompt (still much better than current).
-
-```yaml
-day_phases:
-  defaults:
-    - phase: morning
-      condition: "true"
-      template: "I wake up and check my phone."
-      slots: [social_media_exposures, aggregate_mood]
-    - phase: work
-      condition: "employment_status in ['employed full-time', 'employed part-time', 'self-employed']"
-      template: "I head to work at my {occupation_sector} job."
-      slots: [workplace_exposures, coworker_opinions]
-    - phase: school
-      condition: "school_enrollment != 'not enrolled'"
-      template: "I go to class."
-      slots: [peer_opinions]
-    - phase: evening
-      condition: "true"
-      template: "I'm home for the evening."
-      slots: [family_context, partner_opinion, network_opinions, memory_trace]
-
-  # Override for weekly timesteps
-  weekly:
-    - phase: this_week_at_work
-      condition: "employment_status in ['employed full-time', 'employed part-time', 'self-employed']"
-      template: "This week at work..."
-      slots: [workplace_exposures, coworker_opinions]
-    - phase: this_week_at_home
-      condition: "true"
-      template: "At home this week..."
-      slots: [family_context, partner_opinion, aggregate_mood]
-    - phase: this_week_online
-      condition: "social_media_usage in ['heavy/multiple times daily', 'moderate/daily']"
-      template: "Online this week..."
-      slots: [social_media_exposures, public_discourse]
-```
-
-Phase selection adapts to `timestep_unit`:
-- Daily → morning/work/evening
-- Weekly → this_week_at_work/home/online
-- Monthly → this_month overview
-- Hourly → single phase per timestep
-
-Agent attributes determine which phases render. A retiree skips `work`. A student gets `school` instead. Night shift workers get a different `work` time slot. This is evaluated at prompt build time using the condition expressions.
-
-If no `day_phases` defined in scenario YAML → engine uses sensible defaults based on timestep_unit.
+**Decision:** Day phase templates are not implemented. The improved flat prompt structure (first-person voice, temporal awareness, named peers, local mood, social feed) provides sufficient narrative context without explicit morning/work/evening phase segmentation. The complexity of condition-based phase selection doesn't justify the marginal improvement over the current prompt design.
 
 #### Outcome Tracks (New)
 
@@ -492,20 +446,7 @@ class TimelineEvent(BaseModel):
     exposure_rules: list[ExposureRule] | None = None  # If None, reuse seed_exposure rules
     description: str | None = None  # Human-readable context for this development
 
-class DayPhase(BaseModel):
-    """A single phase in a day template."""
-    phase: str
-    condition: str
-    template: str
-    slots: list[str]
-
-class DayPhaseConfig(BaseModel):
-    """Day phase templates, optionally keyed by timestep unit."""
-    defaults: list[DayPhase] | None = None
-    hourly: list[DayPhase] | None = None
-    daily: list[DayPhase] | None = None
-    weekly: list[DayPhase] | None = None
-    monthly: list[DayPhase] | None = None
+# DayPhase and DayPhaseConfig — OMITTED (see "Day Phases" section above)
 
 class ChannelVariant(BaseModel):
     when: str
@@ -524,7 +465,7 @@ class ScenarioSpec(BaseModel):
     spread: SpreadConfig
     outcomes: OutcomeConfig
     simulation: SimulationConfig
-    day_phases: DayPhaseConfig | None = None      # NEW
+    # day_phases — OMITTED
     channel_experience: dict[str, ChannelExperience] | None = None  # NEW
     relationship_weights: dict[str, float] | None = None  # NEW
 ```

extropy/cli/commands/simulate.py

@@ -191,6 +191,12 @@ def simulate_command(
         "--merged-pass",
         help="Use single merged reasoning pass instead of two-pass (experimental)",
     ),
+    fidelity: str = typer.Option(
+        "medium",
+        "--fidelity",
+        "-f",
+        help="Fidelity level: low (no conversations), medium (2 turns), high (3 turns)",
+    ),
     quiet: bool = typer.Option(False, "--quiet", "-q", help="Suppress progress output"),
     verbose: bool = typer.Option(False, "--verbose", "-v", help="Show detailed logs"),
     debug: bool = typer.Option(
@@ -322,6 +328,7 @@ def on_progress(timestep: int, max_timesteps: int, status: str):
                 db_write_batch_size=db_write_batch_size,
                 resource_governor=governor,
                 merged_pass=merged_pass,
+                fidelity=fidelity,
             )
             simulation_error = None
         except Exception as e:
@@ -359,6 +366,7 @@ def do_simulation():
                     db_write_batch_size=db_write_batch_size,
                     resource_governor=governor,
                     merged_pass=merged_pass,
+                    fidelity=fidelity,
                 )
             except Exception as e:
                 simulation_error = e

extropy/core/models/scenario.py

@@ -281,6 +281,24 @@ class SimulationConfig(BaseModel):
     )
 
 
+# =============================================================================
+# Conversation Relationship Weights
+# =============================================================================
+
+# Default relationship weights used when scenario doesn't specify custom weights
+DEFAULT_RELATIONSHIP_WEIGHTS: dict[str, float] = {
+    "partner": 1.0,
+    "household": 0.9,
+    "close_friend": 0.7,
+    "coworker": 0.6,
+    "neighbor": 0.4,
+    "congregation": 0.4,
+    "school_parent": 0.35,
+    "acquaintance": 0.2,
+    "online_contact": 0.15,
+}
+
+
 # =============================================================================
 # Complete Scenario Spec
 # =============================================================================
@@ -318,6 +336,10 @@ class ScenarioSpec(BaseModel):
         default=None,
         description="Optional background context injected into reasoning prompts",
     )
+    relationship_weights: dict[str, float] | None = Field(
+        default=None,
+        description="Scenario-specific edge weights for conversation priority and peer ordering",
+    )
 
     def to_yaml(self, path: Path | str) -> None:
         """Save scenario spec to YAML file."""

extropy/core/models/simulation.py

@@ -17,7 +17,7 @@
 
 from datetime import datetime
 from enum import Enum
-from typing import Any
+from typing import Any, Literal
 
 from pydantic import BaseModel, Field
 
@@ -331,6 +331,15 @@ class ReasoningContext(BaseModel):
         default=None,
         description="Agent's conformity attribute (0-1)",
     )
+    # Phase D additions
+    available_contacts: list[dict[str, Any]] = Field(
+        default_factory=list,
+        description="People the agent can talk to (name, relationship, observable state)",
+    )
+    social_feed: list[dict[str, Any]] = Field(
+        default_factory=list,
+        description="Recent public statements from the broader population (beyond direct network)",
+    )
 
 
 # =============================================================================
@@ -366,6 +375,10 @@ class ReasoningResponse(BaseModel):
     outcomes: dict[str, Any] = Field(
         default_factory=dict, description="All structured outcomes (from Pass 2)"
     )
+    actions: list[dict[str, Any]] = Field(
+        default_factory=list,
+        description="Actions the agent intends to take (talk_to, etc.)",
+    )
 
     # Token usage tracking (populated by two-pass reasoning)
     pass1_input_tokens: int = Field(default=0, description="Pass 1 input tokens")
@@ -411,6 +424,10 @@ class SimulationRunConfig(BaseModel):
         default=False,
         description="Use single merged pass instead of two-pass reasoning (experimental)",
     )
+    fidelity: Literal["low", "medium", "high"] = Field(
+        default="medium",
+        description="Fidelity level controlling conversation depth and prompt richness",
+    )
 
     # Backward compat aliases
     @property

extropy/simulation/aggregation.py

@@ -1,7 +1,7 @@
 """Aggregate computation for simulation results.
 
 Computes population-level statistics including per-timestep summaries,
-segment breakdowns, and outcome distributions.
+segment breakdowns, outcome distributions, and conversation statistics.
 """
 
 from collections import defaultdict
@@ -13,6 +13,7 @@
     PopulationSpec,
     TimestepSummary,
 )
+from ..storage import StudyDB
 from .state import StateManager
 
 
@@ -296,3 +297,49 @@ def compute_timeline_aggregates(
         )
 
     return timeline
+
+
+def compute_conversation_stats(
+    study_db: StudyDB,
+    run_id: str,
+    max_timesteps: int,
+) -> dict[str, Any]:
+    """Compute conversation statistics for a simulation run.
+
+    Args:
+        study_db: Study database connection
+        run_id: Simulation run ID
+        max_timesteps: Maximum timestep for iteration
+
+    Returns:
+        Dict with conversation statistics
+    """
+    total_conversations = 0
+    conversations_by_timestep: dict[int, int] = {}
+    state_changes_from_conversations = 0
+    total_messages = 0
+
+    for timestep in range(max_timesteps):
+        convs = study_db.get_conversations_for_timestep(run_id, timestep)
+        count = len(convs)
+        total_conversations += count
+        if count > 0:
+            conversations_by_timestep[timestep] = count
+
+        for conv in convs:
+            messages = conv.get("messages", [])
+            total_messages += len(messages)
+            if conv.get("initiator_state_change"):
+                state_changes_from_conversations += 1
+            if conv.get("target_state_change"):
+                state_changes_from_conversations += 1
+
+    avg_turns = total_messages / total_conversations if total_conversations > 0 else 0
+
+    return {
+        "total_conversations": total_conversations,
+        "conversations_by_timestep": conversations_by_timestep,
+        "state_changes_from_conversations": state_changes_from_conversations,
+        "total_messages": total_messages,
+        "avg_turns": round(avg_turns, 2),
+    }