feat(agent): sub-agents, reasoning→agentic routing, layered context pipeline (tinyhumansai#474)

* Enhance agent architecture with sub-agent support and memory optimizations

- Refactored the `Agent` struct to use `Arc` for shared ownership of the provider, tools, and tool specifications, enabling efficient memory management and concurrent access.
- Introduced a `NullMemoryLoader` to optimize memory usage for sub-agents, allowing them to operate without incurring the cost of memory recall.
- Added new methods in the `Agent` implementation to facilitate sharing of the provider, tools, and tool specifications with sub-agents, enhancing their operational efficiency.
- Implemented a new `SystemPromptBuilder` method for constructing prompts specifically for sub-agents, ensuring they receive tailored context while minimizing unnecessary information.
- Established a framework for loading custom agent definitions from TOML files, allowing for dynamic agent configuration and specialization.
- Introduced a `ForkContext` to support efficient sub-agent execution in fork mode, leveraging shared resources for improved performance and reduced token usage.

* Enhance agent definition management and sub-agent functionality

- Introduced a global `AgentDefinitionRegistry` to manage built-in and custom agent definitions from TOML files, ensuring idempotent initialization.
- Added new RPC handlers for listing, fetching, and reloading agent definitions, improving the flexibility of agent management.
- Refactored the `Agent` struct to streamline sub-agent execution, including enhancements to the task execution flow and context handling.
- Updated the orchestrator configuration to support fork mode for sub-agents, optimizing resource usage and performance.
- Improved error handling and logging for agent definition loading and initialization processes, enhancing system reliability.

* Add end-to-end test for sub-agent spawning and response integration

- Implemented a new asynchronous test to validate the full path of a parent agent issuing a `spawn_subagent` tool call.
- The test ensures that the sub-agent's output is correctly folded into the parent's response, verifying the interaction between the parent agent and the sub-agent.
- Enhanced the `AgentDefinitionRegistry` to support global initialization of built-in agents, ensuring consistent behavior across tests.
- Updated the handling of tool calls and memory configuration to facilitate the new test scenario, improving overall test coverage for agent interactions.

* Enhance agent tool filtering with category support

- Introduced a new `category_filter` in `AgentDefinition` to restrict tool visibility based on their category (System or Skill).
- Updated the `from_archetype` function to apply the category filter for the `SkillsAgent` archetype.
- Modified `SubagentRunOptions` to include a `category_filter_override` for dynamic filtering during sub-agent execution.
- Enhanced the `filter_tool_indices` function to incorporate category filtering logic, ensuring tools are correctly filtered based on their defined categories.
- Updated relevant tests to validate the new category filtering functionality, improving overall test coverage for agent interactions.

* Implement layered context reduction pipeline for agent

- Introduced a new `context_pipeline` module to manage a layered context reduction strategy, enhancing memory efficiency during agent interactions.
- Added stages for tool-result budgeting, history trimming, microcompaction, autocompaction, and session memory extraction, each with specific triggers and cache implications.
- Updated the `Agent` struct to include a `context_pipeline` field, ensuring state persistence across turns.
- Enhanced the `AgentBuilder` to initialize the context pipeline by default.
- Implemented tests to validate the functionality and stability of the context pipeline, ensuring consistent behavior across agent sessions.
- Refactored relevant components to integrate the new context management features, improving overall agent performance and memory handling.

* Enhance agent context pipeline with tool result budgeting and microcompaction

- Renamed variable for clarity in tool execution result handling.
- Implemented a new stage in the context pipeline to apply a byte budget to tool results, ensuring efficient memory usage.
- Added logging for budget application, including details on original and final byte sizes.
- Integrated microcompaction stages before tool calls to manage history and reduce memory footprint, with appropriate logging for outcomes.
- Updated the agent's session memory management to track turn counts, facilitating better resource handling across iterations.

* Refactor agent context pipeline for session memory extraction and tool call management

- Simplified method calls in the `Agent` struct for clarity and efficiency.
- Enhanced session memory extraction logic to spawn a background archivist sub-agent when thresholds are met.
- Improved context pipeline handling for tool call recording and usage tracking.
- Updated documentation and comments for better understanding of session memory extraction process.
- Refactored microcompact function for cleaner code structure and readability.

* refactor(agent): split agent.rs into focused submodules

- Convert agent.rs (1988 lines) into agent/ folder with six files:
  * types.rs — Agent + AgentBuilder struct defs
  * builder.rs — AgentBuilder fluent API + Agent::from_config factory
  * turn.rs — turn lifecycle, tool dispatch, context pipeline wiring
  * runtime.rs — public accessors, run_single/run_interactive, helpers
  * tests.rs — integration tests with shared fakes
  * mod.rs — glue + top-level `run` convenience function

- Drop misc external inspiration references from doc comments in the
  context_pipeline module and fork_context — the files now stand on
  their own design language.

* fix(agent): address review comments on sub-agent + context pipeline PR

Inline comment fixes:
- definition.rs: YAML/TOML inconsistency — module doc, PromptSource, source
  bookkeeping, and load() all now uniformly document the TOML format.
- subagent_runner.rs: render_subagent_system_prompt previously only appended
  PromptSource::Inline bodies, silently dropping PromptSource::File content.
  Thread the preloaded archetype_body through as an explicit &str parameter
  so both source variants render. Drops the unused SystemPromptBuilder +
  tools_for_prompt wiring while we're here.
- prompt.rs: remove DateTimeSection from SystemPromptBuilder::for_subagent.
  Local::now() would make the sub-agent system prompt change per call and
  break KV-prefix cache stability. Document the invariant.

Nitpick fixes:
- agent/turn.rs: drop the no-op mark_extraction_started() call — the
  immediately-following mark_extraction_complete() clears the in-flight
  flag anyway.
- context_pipeline/pipeline.rs: call guard.record_compaction_success() on
  microcompact success so a prior streak of autocompaction failures
  doesn't leave the circuit breaker tripped after a successful reduction.
- context_pipeline/tool_result_budget.rs: remove the unnecessary out.clone()
  in the truncation return — capture final_bytes first, then move out.
- definition_loader.rs: replace brittle reg.len() == 10 with
  assert!(reg.len() > 1) plus the existing targeted .get() checks.
- executor.rs: tracing::warn! on unknown sandbox override values so typos
  surface during development; explicitly accept "none" and empty string
  as valid defaults.
- fork_context.rs: add parent_context_visible_inside_scope test mirroring
  fork_context_visible_inside_scope, with minimal stub Provider/Memory
  impls so the test stays self-contained.
- schemas.rs: drop redundant serde_json::to_value(serde_json::json!(...))
  wrapping in handle_list_definitions + handle_get_definition.
- event_bus/events.rs: add SubagentSpawned/SubagentCompleted/SubagentFailed
  cases to all_variants_have_correct_domain.
- tools/ops.rs: add all_tools_includes_spawn_subagent regression test.
- tools/spawn_subagent.rs: sort/dedup the known skill list in place
  instead of cloning into a second vec.

Tests: 2316 passed / 0 failed (up from 2314; two new tests added).
fmt + clippy clean on all touched files.

* udpate prompts

* Enhance AgentBuilder and runtime with event context and interactive CLI improvements

- Added `event_context` method to `AgentBuilder` for setting `session_id` and `channel` for `DomainEvent`s, improving event tagging and correlation.
- Updated `run_interactive` method in `Agent` to dispatch messages through `run_single`, ensuring consistent lifecycle event handling and error sanitization for interactive turns.

* Optimize configuration handling in AgentBuilder by lazily creating Arc for full config in reflection hook. This change reduces unnecessary cloning when learning is enabled, improving performance.

* Add fork-mode test for sub-agent spawning in agent

- Introduced a new test, `turn_dispatches_spawn_subagent_in_fork_mode`, to validate the behavior of the agent when spawning a sub-agent in fork mode.
- The test ensures that the parent agent correctly processes the sub-agent's output and maintains the expected response sequence.
- Enhanced the test setup with a mock provider and memory configuration to simulate the agent's environment effectively.

* Refactor sync RPC handling in skills to treat missing onSync as no-op

- Updated the `handle_sync` function to log a debug message and return a no-op response when a skill does not implement the `onSync` handler, preventing unnecessary RPC errors for skills that do not require periodic syncs.
- This change improves logging clarity and reduces error noise in logs and dashboards for skills that are not designed to handle sync operations.

Author: senamakel

src/core/jsonrpc.rs

@@ -836,6 +836,19 @@ pub async fn bootstrap_skill_runtime() {
     // from both jsonrpc and repl paths) cannot double-subscribe.
     register_domain_subscribers();
 
+    // --- Sub-agent definition registry bootstrap ---
+    // Loads built-in archetype definitions plus any custom TOML files
+    // under `<workspace>/agents/*.toml`. Idempotent — safe to call from
+    // both jsonrpc and repl paths.
+    if let Err(err) = crate::openhuman::agent::harness::AgentDefinitionRegistry::init_global(
+        &base_dir.join("workspace"),
+    ) {
+        log::warn!(
+            "[runtime] AgentDefinitionRegistry::init_global failed: {err} — \
+             spawn_subagent will be unavailable until restart"
+        );
+    }
+
     // --- Socket manager bootstrap ---
     let socket_mgr = Arc::new(SocketManager::new());
     set_global_socket_manager(socket_mgr.clone());