Update main from bpsa#57
Merged
Merged
Conversation
Implements a tag-based XML knowledge system that replaces the lossy summary-of-summaries merge with knowledge extraction: - AgentMemory gains a knowledge field (persistent tagged XML string) - Knowledge is injected into context just before the last message - merge_compressed() now extracts tagged XML facts via LLM and applies them to the knowledge store using merge_context() (deterministic: update existing tags, delete empty tags, append new tags) - Merged compressed steps are removed (info lives in knowledge) - New UpdateKnowledge tool lets the agent manage its own knowledge - Three new functions: merge_context(), list_xml_tag_names(), create_knowledge_extraction_prompt() Files changed: - memory.py: knowledge field + reset - agents.py: knowledge injection + UpdateKnowledge tool registration - bp_compression.py: merge_context, knowledge extraction prompt, modified merge_compressed to return (steps, knowledge) tuple - bp_tools.py: UpdateKnowledge tool class - tests/test_compression.py: updated + 13 new tests (67 pass) Model: claude-opus-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
- save_session and save_session_to_dict now persist agent.memory.knowledge - load_session and load_session_from_dict restore knowledge (backward compatible: missing key defaults to empty string) - /show-stats shows knowledge store size when non-empty - /show-compression-stats shows knowledge chars and section tag names - /show-memory-stats shows knowledge store chars - /session-save and /session-load messages include knowledge info - Added 3 session tests: roundtrip, empty roundtrip, backward compat All 87 tests pass (67 compression + 20 session). Model: claude-opus-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
Add boolean property to Tool base class (default False). When True, the tool full description is injected into the system prompt after the compact tool listing, giving the agent richer guidance for complex tools. - Tool.should_add_tool_description_into_system_prompt added (tools.py) - UpdateKnowledge sets flag to True with enhanced description - All 3 prompt templates updated (code_agent, structured_code_agent, toolcalling_agent) with Jinja loop for flagged tools - Removed temp commit_msg.txt from repo Model: claude-opus-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
…pt log - bp_cli.py: print_turn_summary now shows Knowledge chars when non-empty - agents.py: System prompt log line now includes Context and Knowledge chars Example output: [System prompt: 16,301 chars | Instructions: 11,547 chars | Tool descriptions: 4,754 chars | Context: 27,595 chars | Knowledge: 2,450 chars] Turn 3 | 32.0s | In: 22,882 | Out: 386 | Total: 23,268 | Memory: 17 steps | Context: 27,595 chars | Knowledge: 2,450 chars Model: claude-opus-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
Rewrote the document from an initial implementation plan to accurate documentation of the current two-phase compression pipeline: - Phase 1: Step compression into CompressedHistoryStep - Phase 2: Knowledge extraction into memory.knowledge (tagged XML) - All CompressionConfig fields with actual defaults - All functions and classes in bp_compression.py - Knowledge store mechanics (merge_context, update_knowledge tool) - BPSA CLI environment variables with defaults - Updated file list, usage examples, and design decisions Model: claude-opus-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
Phase 1 step compression now knows which topics are already in the persistent knowledge store. The compression prompt tells the LLM: "These topics are already captured in knowledge: <tags>. Do NOT repeat them. Focus on new findings, changes, or corrections." This avoids redundant information in compressed summaries without any extra LLM calls — the existing compression call simply gets a smarter prompt. Changes: - create_compression_prompt() accepts optional knowledge_tag_names - compress() accepts optional knowledge string, extracts tag names - compression_callback passes agent.memory.knowledge to compress() - Fixed pre-existing test (Planning step assertion) and added 3 new tests for knowledge-aware prompt generation Model: claude-opus-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
…ion 2) Phase 1 step compression now produces both a summary AND knowledge updates in a single LLM call. The compression prompt asks the LLM to output structured format: <summary>Concise summary...</summary> <knowledge_updates> <tag>content</tag> </knowledge_updates> When the LLM includes knowledge_updates, they are applied via the existing merge_context() function. If the LLM doesn't follow the structured format (no <summary> tags), the entire output is used as the summary — fully backwards compatible. This means knowledge starts accumulating from the FIRST compression cycle, not after Phase 2 triggers (which requires 32+ compressed steps). No extra LLM calls — same single compression call, smarter prompt. Changes to bp_compression.py: - create_compression_prompt() now takes full knowledge string (not tag names) and includes it as <current_knowledge> context in the prompt - New parse_compression_output() parses <summary> + <knowledge_updates> with graceful fallback for unstructured output - compress() returns tuple[list[MemoryStep], str] (steps, knowledge) - compress() applies knowledge_updates via merge_context() when present - compression_callback unpacks tuple return - parse_compression_output added to __all__ Changes to bp_cli.py: - /compress command handles tuple return from compress() Changes to tests/test_compression.py: - 6 new tests for parse_compression_output (structured, summary-only, fallback, no-summary-with-knowledge, empty, None) - 2 new tests for compress (knowledge extraction, fallback no tags) - Updated existing compress tests for tuple return type - Updated prompt tests for new knowledge parameter Model: claude-opus-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
Reflects the combined summary + knowledge update feature: - Phase 1 now extracts knowledge alongside summarization - Updated architecture diagram showing both phases - compress() signature updated: returns tuple[list, str] - New parse_compression_output() documented - create_compression_prompt() takes full knowledge string - Knowledge store section: 3 sources (Phase 1, Phase 2, manual) - New subsection: Phase 1 Knowledge Extraction with format details - Updated design decisions for combined approach - Updated test descriptions Model: claude-opus-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
The Phase 1 compression prompt now includes ALL existing compressed history steps alongside the knowledge store. This gives the LLM full context to avoid duplication: - **Compressed history** = chronological record of past events/changes. The LLM is told not to repeat any already-summarized events. - **Knowledge** = current beliefs/facts. The LLM can propose updates when the execution history reveals corrections or new information. The prompt clearly explains the distinction between the two stores: history captures what happened over time, knowledge captures current truth. The new summary goes into history; knowledge updates are extracted separately. Changes to bp_compression.py: - create_compression_prompt() now accepts existing_summaries parameter - Prompt includes <compressed_history> section when summaries exist - Deduplication instructions reference both history and knowledge - compress() collects existing CompressedHistoryStep instances and passes them to the prompt Changes to tests/test_compression.py: - New test: prompt with existing summaries - New test: prompt with both history and knowledge - Updated assertion for knowledge-aware prompt Model: claude-opus-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
…prompt Model: claude-opus-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
…ove_actionstep_to_memory, and move_actionstep_from_memory tools Set should_add_tool_description_into_system_prompt = True for these four tools so their full descriptions are included in the system prompt, matching the behavior already present for update_knowledge. Model: claude-opus-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
Add knowledge char count to the per-step summary line shown in the console. Changes: - monitoring.py: Monitor.__init__ now accepts optional memory parameter - monitoring.py: update_metrics appends knowledge char count when present - agents.py: passes self.memory to Monitor Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
The Knowledge store row in print_stats() was previously only shown when knowledge was non-empty. Now it always displays the character count (showing 0 when empty) so users can always see the knowledge store size in /show-stats. Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
Generated-by: claude-sonnet-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
Generated-by: claude-sonnet-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
Generated-by: claude-sonnet-4.6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
…d-steps commands Completes the compression CLI command set by adding runtime control over the two remaining parameters: keep_compressed_steps and max_compressed_steps. Changes: - cmd_compression_keep_compressed(): new function to change keep_compressed_steps - cmd_compression_max_compressed(): new function to change max_compressed_steps - Added both commands to SLASH_COMMANDS autocomplete list - Added both commands to /help table - Added both dispatch handlers in main REPL loop Model: claude-sonnet-4-6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
If the user types /session-load mysession and the file is not found, and no extension was given, automatically retry with mysession.json. If the .json fallback also fails, the error message says: File not found: mysession (also tried mysession.json) Model: claude-sonnet-4-6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
…low presets Three new commands that set all 4 compression parameters at once: /compression-set-high: keep_recent=10, max_uncompressed=13, keep_compressed=10, max_compressed=13 /compression-set-normal: keep_recent=40, max_uncompressed=50, keep_compressed=10, max_compressed=20 /compression-set-low: keep_recent=90, max_uncompressed=100, keep_compressed=20, max_compressed=40 Each command prints a confirmation table of all 4 applied values. Model: claude-sonnet-4-6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
…ons→/instructions-load - Rename /compression-set-normal to /compression-set-medium in bp_cli.py (autocomplete list, help table, command handler, description text) - Rename /load-instructions to /instructions-load in bp_cli.py (autocomplete list, help table, command handler) - Update CLI.md to reflect /instructions-load rename - Alphabetical order verified and maintained in all affected lists Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
- Add _parse_mcp_servers() helper: parses URL (HTTP) or shell command (stdio) strings - Add _shutdown_mcp() helper: safe teardown of MCPClient context manager - build_agent() gains mcp_servers=None param; creates MCPClient, extends tools, stores as agent._mcp_client - run_one_shot() and run_repl() gain mcp_servers=None param, passed through to build_agent() - _shutdown_mcp() called at all exit points: EOF, /exit, /clear (which also reconnects) - main() parses --mcp URL_OR_CMD (repeatable) and wires mcp_servers through Usage: bpsa --mcp http://localhost:8000/mcp bpsa --mcp 'npx @mcp/server-filesystem /tmp' bpsa --mcp URL1 --mcp CMD2 (multiple servers) Requires: pip install 'bpsa[mcp]' Generated by claude-sonnet-4-6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
Mirror the MCP integration from bp_cli.py into the autonomous loop: - run_loop() gains mcp_servers=None param, passed to build_agent() - _shutdown_mcp(agent) called in finally: block after each prompt task - main() gains --mcp URL_OR_CMD (repeatable) CLI argument - _parse_mcp_servers() imported from bp_cli to parse server strings - mcp_servers wired through main() -> run_loop() - print_banner() shows MCP server count in the startup panel Usage: ad-infinitum tasks/ --mcp http://localhost:8000/mcp ad-infinitum tasks/ --mcp 'npx @mcp/server-filesystem /tmp' Requires: pip install 'bpsa[mcp]' Generated by claude-sonnet-4-6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
…updates The compressor is now given the steps that occurred AFTER the batch being compressed (post_steps). This allows the compressor to see what is still current vs already superseded, preventing it from writing stale knowledge that was already overridden by more recent steps. Changes: - create_compression_prompt(): added post_steps param (list[MemoryStep] | None) - Builds <subsequent_steps> section in prompt (truncated to 500/300 chars per step) - compress(): computes post_steps as steps after max(to_compress_indices) (excluding TaskStep and CompressedHistoryStep) and passes to prompt builder Generated by claude-sonnet-4-6 Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
…stale knowledge Extends the post-batch context pattern (from d9d4b73) to Phase 2 compression: - _build_post_steps_section() extracted as shared standalone helper (used by both phases) - create_knowledge_extraction_prompt() gains post_steps param - merge_compressed() computes steps not being merged and passes as post_steps - LLM now sees subsequent steps before updating knowledge in both phases This prevents the compressor from writing stale knowledge entries that have already been superseded by later agent activity. Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
Documents 9 parallels between BPSA two-phase compression and the Standard Model of Memory Consolidation in neuroscience: - Working memory vs long-term memory (prefrontal cortex / hippocampus) - Sleep consolidation phases (SWS → Phase 1, REM → Phase 2) - Episodic vs semantic memory (CompressedHistoryStep vs knowledge store) - Reconstruction vs recording (Bartlett 1932) - Schemas / semantic networks → tagged XML knowledge - Attention weight / recency effect → knowledge placement - Forgetting curve → min_compression_chars threshold - Motivated forgetting → preserve_final_answer_steps - Metacognition → agent-driven update_knowledge Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
Updated context compression description to include biologically inspired summarization.
- Soften SWS/REM sleep analogy: clarify it is functional, not mechanistic - Add Multiple Trace Theory caveat to the Standard Consolidation Model diagram - Fix hippocampus/neocortex roles in Section 1 (neocortex holds compressed form) - Remove anthropomorphising language in Section 3 - Rename Section 5 from 'Metacognition' to 'Deliberate Belief Revision' - Add hypothesis caveat to convergence claim in intro paragraph - Update diagram label from 'deeper sleep' to 'later consolidation' Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
Co-Authored-By: bpsa2 <241537330+bpsa2@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.