feat(v0.10.3): search & pack quality fixes from user feedback

Five bugs surfaced after a month of real Task Journal use:

1. CRITICAL — task_search "OPS-306" crashed with
   "no such column: 306" because FTS5 read `-` as column-prefix
   syntax. Affects every hyphenated ID, path, colon-prefixed token,
   and glob. New tj_core::fts::sanitize_query phrase-quotes any
   query containing `-` `:` `*` `(` `)` `"` `/`; multi-word queries
   pass through untouched so default AND-tokens semantics survive.
   Applied at both MCP task_search and CLI Search call sites
   (incl. --all-projects).

2. HIGH — search missed terms living only in event body when the
   unicode61 tokenizer split source text differently from the query.
   When sanitized FTS5 returns 0 hits, fall back to LIKE %query%
   against search_fts.text. Same fallback in MCP + CLI.

3. HIGH — pack truncation dropped the NEWEST decision (most-important
   summary) instead of the oldest. render_active_decisions /
   render_evidence / render_rejected now ORDER BY ... DESC so end-of-pack
   truncation hits older rows. Also bumped FULL_BUDGET 10K → 24K —
   real tasks accumulate 50–100 events with detailed text and 10K
   was clipping the summary even after the DESC reshuffle.

4. MEDIUM — added --type / event_type filter on search.
   `task-journal search "switching" --type decision` and the
   matching MCP param restrict matches to one EventType. SQL adds
   `AND type = ?2`. Composes with the LIKE fallback.

5. LOW — two PreCompact firings within 60 s used to double-write
   the "Conversation compacted at T" boundary marker (multi-plugin
   race, retried hook). Dedup check inspects the most recent
   decision event for the active task and skips the append if the
   text already starts with the marker prefix AND the prior write
   was within DEDUP_WINDOW_SECS = 60.

Added: tj_core::fts module (sanitize_query, like_pattern) +
6 integration tests covering each bug. No schema changes — pack
cache invalidates on next event so the new ordering surfaces
organically without a forced rebuild.

bd: claude-memory-3wg (epic), claude-memory-jrd, claude-memory-2pi,
claude-memory-bo1, claude-memory-5e0, claude-memory-jaq.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Author: Shahinyanm

CHANGELOG.md

@@ -7,6 +7,72 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.10.3] - 2026-06-06
+
+**Search & pack quality fixes from real user feedback.** Five bugs hit
+during a month-long session: FTS5 query crashes on hyphenated
+identifiers (`OPS-306` → `no such column: 306`), event-body search
+missing hits the tokenizer split differently than the query, pack
+truncation cutting the **newest** decision (most important) instead
+of the oldest, no way to filter search by event type, and duplicate
+"Conversation compacted" markers when PreCompact fires twice within
+the same second.
+
+### Added
+- `tj_core::fts::sanitize_query` — phrase-quotes FTS5 metacharacters
+  (`-` `:` `*` `(` `)` `"` `/`) so identifiers like `OPS-306`, paths
+  like `src/main.rs`, and tokens like `ttl:30s` stop crashing the
+  `search_fts MATCH` planner. Multi-word queries pass through
+  unchanged so default AND semantics are preserved.
+- `tj_core::fts::like_pattern` — wraps a query as `%query%` for the
+  LIKE-fallback path described below.
+- `--type <event_type>` flag on `task-journal search` and matching
+  `event_type` field on the MCP `task_search` tool. Restricts hits
+  to a single event class (`decision`, `evidence`, `finding`, ...).
+- LIKE fallback in both CLI `search` and MCP `task_search`: when the
+  sanitized FTS5 phrase returns zero hits, the same query is rerun
+  against `search_fts.text LIKE %query%`. Recovers cases where the
+  unicode61 tokenizer split the source text differently from the
+  query string.
+
+### Changed
+- `render_active_decisions`, `render_evidence`, `render_rejected`
+  now `ORDER BY ... DESC` (newest-first). The summary/final decision
+  the agent records just before close lives at the *top* of its
+  section so end-of-pack truncation drops the **oldest** rows, not
+  the newest.
+- `FULL_BUDGET` bumped 10 KiB → 24 KiB. Real long-running tasks
+  (50–100 events with detailed decision text) blew past 10 KiB after
+  a couple of weeks and the budget was the binding constraint on
+  what survived. 24 KiB still fits comfortably inside any modern
+  LLM context window.
+
+### Fixed
+- B1 (CRITICAL): `task_search "OPS-306"` no longer crashes with
+  `MCP error -32603: no such column: 306`. Same fix covers all
+  paths/colon-prefixed tokens/glob-shaped queries.
+- B2 (HIGH): event-body terms now reach the user via the LIKE
+  fallback when an FTS5 token-split mismatch otherwise hides them.
+- B3 (HIGH): the **newest** decision is now the first line of the
+  Active decisions section and survives truncation; the user's
+  "final summary" pattern no longer gets clipped.
+- B5 (LOW): two `PreCompact` hook firings within 60 s no longer
+  double-write the boundary marker. The dedup check inspects the
+  most-recent decision event for the active task and skips the
+  append if it already looks like a recent compaction marker.
+
+### Migration
+- No schema changes. Existing tasks pick up the new ordering on the
+  next pack render (cache is keyed by mode, not order, so callers
+  may need to clear `task_pack_cache` once for visible effect — or
+  wait for the next event to invalidate it organically).
+
+### CLI / MCP API
+- CLI: `task-journal search <query> [--type TYPE]` is additive.
+- MCP `task_search`: new optional `event_type: Option<String>`
+  parameter. Existing callers that omit it see no behavior change
+  besides the FTS5 crash fix.
+
 ## [0.10.2] - 2026-06-02
 
 **`watchPaths` + FileChanged → auto-evidence on marker file edits.** X4

Cargo.lock

@@ -7,7 +7,7 @@ members = [
 ]
 
 [workspace.package]
-version = "0.10.2"
+version = "0.10.3"
 edition = "2021"
 rust-version = "1.88"
 license = "MIT"

Cargo.toml

@@ -16,7 +16,7 @@ name = "task-journal"
 path = "src/main.rs"
 
 [dependencies]
-tj-core = { package = "task-journal-core", version = "0.10.2", path = "../tj-core" }
+tj-core = { package = "task-journal-core", version = "0.10.3", path = "../tj-core" }
 anyhow = { workspace = true }
 clap = { workspace = true }
 tracing = { workspace = true }

crates/tj-cli/Cargo.toml

@@ -700,6 +700,10 @@ enum Commands {
         /// Search across all projects on this machine, not just the cwd one.
         #[arg(long)]
         all_projects: bool,
+        /// v0.10.3+: restrict matches to a single event type
+        /// (`decision`, `evidence`, `finding`, `rejection`, ...).
+        #[arg(long = "type", value_name = "TYPE")]
+        event_type: Option<String>,
     },
     /// Append a correction event referencing an earlier event_id.
     EventCorrect {
@@ -1738,6 +1742,45 @@ fn main() -> Result<()> {
 
                 // (2) Boundary marker.
                 let now = chrono::Utc::now().to_rfc3339_opts(chrono::SecondsFormat::Secs, true);
+
+                // v0.10.3: dedupe near-duplicate markers. Two PreCompact
+                // hook firings within DEDUP_WINDOW_SECS — caused by
+                // multi-plugin race, rapid compact-then-restore, or a
+                // retried hook — both append "Conversation compacted at
+                // T" events with the same wall-clock second. Skip if
+                // the most recent decision event already carries this
+                // marker text and was written under a minute ago.
+                const DEDUP_WINDOW_SECS: i64 = 60;
+                let last_marker: Option<(String, String)> = conn
+                    .query_row(
+                        "SELECT ei.timestamp, COALESCE(sf.text, '') \
+                         FROM events_index ei \
+                         LEFT JOIN search_fts sf ON sf.event_id = ei.event_id \
+                         WHERE ei.task_id = ?1 AND ei.type = 'decision' \
+                         ORDER BY ei.timestamp DESC LIMIT 1",
+                        rusqlite::params![&tc.task_id],
+                        |r| Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?)),
+                    )
+                    .ok();
+                if let Some((ts, text)) = last_marker {
+                    if text.starts_with("Conversation compacted at") {
+                        if let Ok(prev) = chrono::DateTime::parse_from_rfc3339(&ts) {
+                            let delta = (chrono::Utc::now()
+                                .signed_duration_since(prev.with_timezone(&chrono::Utc)))
+                            .num_seconds();
+                            if delta.abs() < DEDUP_WINDOW_SECS {
+                                // Marker recently appended — skip the
+                                // second one. Still print SOMETHING so
+                                // hook callers see a stable exit shape;
+                                // emit the previous event_id we'd have
+                                // duplicated would not be available here
+                                // without an extra query, so emit empty.
+                                return Ok(());
+                            }
+                        }
+                    }
+                }
+
                 let marker_text = format!(
                     "Conversation compacted at {now}; preceding events should be treated as a single reasoning unit."
                 );
@@ -2266,7 +2309,12 @@ fn main() -> Result<()> {
             query,
             limit,
             all_projects,
+            event_type,
         } => {
+            // v0.10.3: sanitize FTS5 query so hyphenated IDs / paths /
+            // colons no longer crash with "no such column" mid-search.
+            let fts_query = tj_core::fts::sanitize_query(&query);
+            let like_query = tj_core::fts::like_pattern(&query);
             if all_projects {
                 let state_dir = tj_core::paths::state_dir()?;
                 let hashes = tj_core::db::list_all_projects(&state_dir)?;
@@ -2276,19 +2324,17 @@ fn main() -> Result<()> {
                         Ok(c) => c,
                         Err(_) => continue,
                     };
-                    let mut stmt = match conn.prepare(
-                        "SELECT DISTINCT task_id FROM search_fts WHERE search_fts MATCH ?1 LIMIT ?2"
+                    let ids = match run_search(
+                        &conn,
+                        &fts_query,
+                        &like_query,
+                        event_type.as_deref(),
+                        limit,
                     ) {
-                        Ok(s) => s,
+                        Ok(v) => v,
                         Err(_) => continue,
                     };
-                    let rows = match stmt.query_map(rusqlite::params![&query, limit as i64], |r| {
-                        r.get::<_, String>(0)
-                    }) {
-                        Ok(r) => r,
-                        Err(_) => continue,
-                    };
-                    for id in rows.flatten() {
+                    for id in ids {
                         println!("{hash}\t{id}");
                     }
                 }
@@ -2304,14 +2350,7 @@ fn main() -> Result<()> {
                 if events_path.exists() {
                     tj_core::db::ingest_new_events(&conn, &events_path, &project_hash)?;
                 }
-                let mut stmt = conn.prepare(
-                    "SELECT DISTINCT task_id FROM search_fts WHERE search_fts MATCH ?1 LIMIT ?2",
-                )?;
-                let ids: Vec<String> = stmt
-                    .query_map(rusqlite::params![query, limit as i64], |r| {
-                        r.get::<_, String>(0)
-                    })?
-                    .collect::<Result<_, _>>()?;
+                let ids = run_search(&conn, &fts_query, &like_query, event_type.as_deref(), limit)?;
                 for id in ids {
                     println!("{id}");
                 }
@@ -2634,6 +2673,77 @@ fn topic_is_fts_safe(topic: &str) -> bool {
         .any(|c| matches!(c, '-' | '"' | '*' | ':' | '(' | ')'))
 }
 
+/// v0.10.3: shared search helper used by `Commands::Search` for both
+/// the cwd and `--all-projects` paths. Runs the sanitized FTS5 MATCH
+/// first; on zero hits, scans `search_fts.text` via `LIKE` so
+/// hyphenated identifiers (e.g. `OPS-306`) and substrings missed by
+/// the unicode61 tokenizer still surface.
+fn run_search(
+    conn: &rusqlite::Connection,
+    fts_query: &str,
+    like_query: &str,
+    event_type: Option<&str>,
+    limit: usize,
+) -> Result<Vec<String>> {
+    let (fts_sql, fts_uses_type) = match event_type {
+        Some(_) => (
+            "SELECT DISTINCT task_id FROM search_fts \
+             WHERE search_fts MATCH ?1 AND type = ?2 LIMIT ?3",
+            true,
+        ),
+        None => (
+            "SELECT DISTINCT task_id FROM search_fts \
+             WHERE search_fts MATCH ?1 LIMIT ?2",
+            false,
+        ),
+    };
+    let mut stmt = conn.prepare(fts_sql)?;
+    let ids: Vec<String> = if fts_uses_type {
+        let ty = event_type.unwrap();
+        stmt.query_map(rusqlite::params![fts_query, ty, limit as i64], |r| {
+            r.get::<_, String>(0)
+        })?
+        .collect::<rusqlite::Result<_>>()?
+    } else {
+        stmt.query_map(rusqlite::params![fts_query, limit as i64], |r| {
+            r.get::<_, String>(0)
+        })?
+        .collect::<rusqlite::Result<_>>()?
+    };
+    if !ids.is_empty() {
+        return Ok(ids);
+    }
+
+    let (like_sql, like_uses_type) = match event_type {
+        Some(_) => (
+            "SELECT DISTINCT task_id FROM search_fts \
+             WHERE text LIKE ?1 AND type = ?2 LIMIT ?3",
+            true,
+        ),
+        None => (
+            "SELECT DISTINCT task_id FROM search_fts \
+             WHERE text LIKE ?1 LIMIT ?2",
+            false,
+        ),
+    };
+    let mut stmt_like = conn.prepare(like_sql)?;
+    let ids_like: Vec<String> = if like_uses_type {
+        let ty = event_type.unwrap();
+        stmt_like
+            .query_map(rusqlite::params![like_query, ty, limit as i64], |r| {
+                r.get::<_, String>(0)
+            })?
+            .collect::<rusqlite::Result<_>>()?
+    } else {
+        stmt_like
+            .query_map(rusqlite::params![like_query, limit as i64], |r| {
+                r.get::<_, String>(0)
+            })?
+            .collect::<rusqlite::Result<_>>()?
+    };
+    Ok(ids_like)
+}
+
 fn run_rejected(topic: &str, all_projects: bool, limit: usize, since: Option<i64>) -> Result<()> {
     let cutoff: Option<String> = since.map(|d| {
         (chrono::Utc::now() - chrono::Duration::days(d))