[evolution] fix(#399): add SessionDB (state.db) scan path to introspection_extract#401
Closed
Da-Mikey wants to merge 1 commit into
Closed
[evolution] fix(#399): add SessionDB (state.db) scan path to introspection_extract#401Da-Mikey wants to merge 1 commit into
Da-Mikey wants to merge 1 commit into
Conversation
…on_extract The build_digest() function previously scanned only *.jsonl transcripts and request_dump_*.json snapshots from the sessions directory. On installations that persist conversation history via the SQLite SessionDB (state.db), the messages table holds >90% of real sessions, while on-disk files contain only error snapshots -- making the entire introspection pipeline blind and error-biased. Changes: 1. Added _sessions_from_db(db_path, cutoff) -- reads the messages table grouped by session_id, ordered by timestamp, casting each row to the same message-dict shape scan_messages() consumes (role, content, tool_calls parsed from JSON text, tool_call_id). 2. Added optional db_path parameter to build_digest() -- when provided and the file exists, each session from the DB is fed through scan_messages() and aggregated identically to the file-based paths. 3. Updated main() and evolution_trace_miner.py to auto-detect state.db under HERMES_HOME and pass it to build_digest(). Before: 11 sessions scanned (from error-biased request dumps only) After: 338 sessions scanned (from the real conversation database) Resolves Lexus2016#399
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[evolution]
Summary
The function in previously scanned only transcripts and snapshots. On installations using the SQLite SessionDB (), the table holds >90% of real sessions while on-disk files contain only error snapshots — making the introspection pipeline blind and error-biased.
What changed
Before/After
Files changed
All 16 existing tests pass.