docs: add behavior changes section and expand AI assistant guide

v0.19.0 What's New:
- Replace scattered "Configuration Changes" table and "Upgrade Notes" with
  consolidated "Behavior Changes" section
- Each change has before/after tables and restore instructions
- Covers: write_note overwrite guard, semantic search default, frontmatter on
  sync, project-prefixed permalinks, build_context JSON default, hybrid scoring,
  search_by_metadata removal, config format migration

AI Assistant Guide:
- Rename "Entity types" → "Note types" for frontmatter type field
- Expand search section with separate subsections for note type, date, tag, and
  frontmatter metadata filtering with filter operators table
- Expand schema section with schema format example, tool descriptions
  (schema_infer, schema_validate, schema_diff), and discovery/evolution workflow
- Add "Note Already Exists" error handling for overwrite guard
- Note build_context JSON default and semantic search enabled by default

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: phernandez

content/2.whats-new/0.v0.19.0.md

@@ -13,7 +13,7 @@ Knowledge bases grow organically, and over time note structure drifts — some `
 
 The workflow is conversational — just ask your AI assistant:
 
-```
+```yml
 You: "I've been writing a lot of person notes. Can you figure out
       what they have in common and create a schema?"
 
@@ -56,12 +56,12 @@ See the full [Schema System](/concepts/schema-system) guide for syntax, resoluti
 
 ## Semantic Search
 
-When semantic dependencies are installed, Basic Memory automatically uses hybrid search — combining keyword matching with meaning-based retrieval. Search for "login security improvements" and find notes about "authentication hardening" even though the exact words don't match.
+Basic Memory now includes semantic search by default — hybrid search combines keyword matching with meaning-based retrieval out of the box. Search for "login security improvements" and find notes about "authentication hardening" even though the exact words don't match.
 
 Three search modes are available:
 
-- **`hybrid`** (default when semantic deps installed) — Combines keyword and semantic search with rank fusion
-- **`text`** — Keyword-only search (the fallback without semantic deps)
+- **`hybrid`** (default) — Combines keyword and semantic search with score-based fusion
+- **`text`** — Keyword-only search
 - **`vector`** — Pure semantic similarity search
 
 ```python
@@ -75,14 +75,9 @@ search_notes(query="how to speed up the app", search_type="vector")
 search_notes(query="error handling", search_type="hybrid", min_similarity=0.7)
 ```
 
-Semantic dependencies are included in Homebrew installs. For uv, install with extras:
+Semantic search is included in all standard installs (Homebrew and uv). On first startup, Basic Memory automatically generates embeddings for existing notes.
 
-```bash
-uv tool install 'basic-memory[semantic]'
-bm reindex --embeddings
-```
-
-Vector and hybrid search return observation-level results — individual facts and relations, not just whole notes — so you find the specific piece of information you need.
+Search results default to entity-level — whole notes ranked by relevance, with matched chunk text showing which part was most relevant. Use the `entity_types` parameter to get observation-level results when needed.
 
 See the full [Semantic Search](/concepts/semantic-search) guide for configuration, embedding providers, and how it works under the hood.
 
@@ -94,8 +89,8 @@ You can now route individual projects through cloud while keeping others local.
 
 ```bash
 # Save or create a cloud API key
-bm cloud set-key bmc_...
-bm cloud create-key "my-laptop"
+bm cloud api-key save bmc_...
+bm cloud api-key create "my-laptop"
 
 # Route a specific project through cloud
 bm project set-cloud research
@@ -124,13 +119,52 @@ This makes cross-project references unambiguous. Within notes, the `project::not
 
 ---
 
-## Output Format Improvements
+## Unified Metadata Search
+
+`search_by_metadata` has been merged into `search_notes` — one tool for all searches. The `query` parameter is now optional, so you can search purely by frontmatter metadata.
+
+```python
+search_notes(metadata_filters={"status": "in-progress"})
+search_notes(metadata_filters={"tags": ["security", "oauth"]})
+search_notes(metadata_filters={"priority": {"$in": ["high", "critical"]}})
+search_notes(tags=["security"])  # convenience shorthand
+search_notes(status="draft")     # convenience shorthand
+```
+
+---
+
+## `tag:` Search Shorthand
+
+Search by tag using convenient shorthand syntax instead of structured filters:
+
+```python
+search_notes("tag:security")
+search_notes("tag:coffee AND tag:brewing")
+```
+
+---
+
+## `write_note` Overwrite Guard
+
+`write_note` is now non-idempotent by default. If a note already exists at the target path, the tool returns an error instead of silently overwriting. This prevents accidental data loss.
+
+- Pass `overwrite=True` to explicitly replace an existing note
+- Use `edit_note` for incremental updates to existing notes
+- Set `write_note_overwrite_default: true` in config to restore the old upsert behavior
+
+---
+
+## JSON Output Mode
+
+All MCP tools now support `output_format="json"` for machine-readable responses. The default remains `"text"` for human-readable output — no breaking changes for existing usage.
+
+`build_context` defaults to `"json"` with slimmed payloads (redundant fields stripped).
 
-`search_notes` and `read_note` now support an `output_format` parameter:
+CLI tool commands support `--format json`:
 
-- **`default`** — Structured data (for programmatic use)
-- **`ascii`** — Plain text table (for terminal display)
-- **`ansi`** — Colorized terminal output
+```bash
+bm tool search-notes --query "authentication" --format json
+```
 
 ---
 
@@ -153,40 +187,171 @@ The `note_type` sets the `type` frontmatter field (used for schema resolution an
 
 ---
 
-## `bm watch` Command
+## Score-Based Hybrid Fusion
 
-A new standalone watch command runs the file watcher as a dedicated long-running process:
+The hybrid search scoring algorithm has been replaced. The old Reciprocal Rank Fusion (RRF) compressed all fused scores to ~0.016, destroying ranking differentiation. The new formula preserves dominant signals and rewards dual-source agreement:
 
-```bash
-bm watch
-bm watch --project main
 ```
+score = max(vec, fts) + 0.3 * min(vec, fts)
+```
+
+Zero-score results now produce zero fused score instead of receiving a weight floor. Search result ordering may differ from v0.18 — results should be more accurate with better score differentiation.
 
-Use this when you're editing notes in an external editor (Obsidian, VS Code, etc.) and want the database index updated in real-time without running the full MCP server.
+---
+
+## Dashboard (`bm project info`)
+
+`bm project info` now displays an htop-inspired compact dashboard with:
+
+- Horizontal bar charts for note types (top 5)
+- Embedding coverage bar with Unicode block characters
+- Colored status dots for at-a-glance health
 
 ---
 
-## Upgrade Notes
+## Entity User Tracking
+
+Entities now track `created_by` and `last_updated_by` fields for attribution, so you can see whether a note was created by a human or an AI assistant.
 
-### Project config migration
+---
+
+## Improved Search Results
 
-Project configuration now uses structured entries with `path`, `mode`, and sync metadata. Legacy config formats (string-valued `projects`, `project_modes`, `cloud_projects`) are migrated automatically when Basic Memory loads your config. The migrated config is re-saved in the new format.
+Search results now surface more relevant context:
 
-### `default_project_mode` removal
+- `matched_chunk_text` populated for FTS-only hybrid results (no more fallback to truncated content)
+- Top chunks per result increased from 3 to 5, catching answers deeper in large notes
+- Content display limit doubled from 2,000 to 4,000 characters for results without matched chunks
 
-The `default_project_mode` setting is no longer used. `default_project` is the fallback when no project is explicitly passed to a tool or command.
+---
 
-### Semantic extras
+## `--json` CLI Flags
 
-Semantic search dependencies are optional extras. Install them explicitly if you want vector or hybrid search:
+Five additional CLI commands now support `--json` for machine-readable output:
 
 ```bash
-uv tool install 'basic-memory[semantic]'
+bm status --json
+bm project list --json
+bm schema validate --json
+bm schema infer --json
+bm schema diff --json
+```
+
+This complements the existing `bm project info --json` and `bm tool --format json` support, making all major CLI commands scriptable for CI pipelines and automation.
+
+---
+
+## Behavior Changes
+
+v0.19.0 changes several defaults and removes deprecated features. Each change is explained below with instructions for restoring the previous behavior where applicable.
+
+### `write_note` is non-destructive by default
+
+`write_note` no longer silently overwrites existing notes. If a note already exists at the target path, the tool returns an error instead of replacing it.
+
+| | Before (v0.18) | After (v0.19) |
+|---|---|---|
+| Writing to an existing path | Silently overwrites | Returns an error |
+| Explicit overwrite | Not needed | Pass `overwrite=True` |
+| Incremental edits | `write_note` or `edit_note` | Use `edit_note` |
+
+This prevents accidental data loss when an AI assistant writes a note without realizing one already exists.
+
+**Restore old behavior:** Set `write_note_overwrite_default: true` in `~/.basic-memory/config.json`.
+
+### Semantic search enabled by default
+
+Semantic search is now on out of the box. Embeddings are generated automatically on first startup — no opt-in or manual setup required. If `sqlite-vec` fails to load, search gracefully falls back to text-only mode.
+
+| | Before (v0.18) | After (v0.19) |
+|---|---|---|
+| `semantic_search_enabled` | `false` | `true` |
+| Embedding generation | Manual | Automatic on first startup |
+| Install size | Smaller (no embedding deps) | Larger (includes fastembed) |
+
+**Restore old behavior:** Set `semantic_search_enabled: false` in config, or set `BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=false`.
+
+### Frontmatter added on sync
+
+Files without YAML frontmatter now get it added automatically during sync. This ensures all notes have proper metadata for search, permalinks, and schema resolution.
+
+| | Before (v0.18) | After (v0.19) |
+|---|---|---|
+| `ensure_frontmatter_on_sync` | `false` | `true` |
+| Files without frontmatter | Left untouched | Frontmatter injected |
+
+**Restore old behavior:** Set `ensure_frontmatter_on_sync: false` in config.
+
+### Project-prefixed permalinks
+
+Generated permalinks now include the project slug by default, making cross-project references unambiguous.
+
+| | Before (v0.18) | After (v0.19) |
+|---|---|---|
+| `permalinks_include_project` | `false` | `true` |
+| Example permalink | `docs/authentication` | `main/docs/authentication` |
+
+Existing permalinks without a project prefix still resolve correctly — resolution tries project-prefixed first, then falls back to bare path.
+
+**Restore old behavior:** Set `permalinks_include_project: false` in config.
+
+### `build_context` defaults to JSON output
+
+`build_context` now returns JSON by default instead of text. The JSON response strips redundant fields for a slimmer payload. All other tools still default to `"text"`.
+
+| | Before (v0.18) | After (v0.19) |
+|---|---|---|
+| `build_context` default format | `"text"` | `"json"` |
+| Other tools default format | `"text"` | `"text"` (unchanged) |
+
+**Get text output:** Pass `output_format="text"` explicitly.
+
+### Hybrid search scoring algorithm
+
+The hybrid search scoring formula changed from Reciprocal Rank Fusion (RRF) to score-based fusion. The old RRF formula compressed all fused scores to ~0.016, destroying ranking differentiation. The new formula preserves dominant signals and rewards dual-source agreement.
+
+Search result ordering may differ from v0.18. Results should be more accurate with better score differentiation. Zero-score results now produce zero fused score instead of receiving a weight floor.
+
+### `search_by_metadata` removed
+
+`search_by_metadata` is no longer a standalone tool. Use `search_notes` with `metadata_filters` instead — same parameters, same behavior.
+
+```python
+# Before (v0.18)
+search_by_metadata(metadata_filters={"status": "draft"})
+
+# After (v0.19)
+search_notes(metadata_filters={"status": "draft"})
 ```
 
-### Config cleanup
+### `default_project_mode` removed
+
+The `default_project_mode` setting is no longer used. Use `default_project` as the fallback when no project is explicitly passed to a tool or command. Per-project routing (`set-cloud` / `set-local`) replaces the global mode toggle.
+
+### Project config format changed
+
+Project configuration now uses structured entries with `path`, `mode`, and sync metadata. Legacy config formats (string-valued `projects`, `project_modes`, `cloud_projects`) are migrated automatically when Basic Memory loads your config and re-saved in the new format.
+
+```json
+// Before (v0.18)
+{
+  "projects": {
+    "main": "/Users/you/basic-memory"
+  }
+}
+
+// After (v0.19)
+{
+  "projects": {
+    "main": {
+      "path": "/Users/you/basic-memory",
+      "mode": "local"
+    }
+  }
+}
+```
 
-Legacy configuration keys (`project_modes`, `cloud_projects`, `default_project_mode`) are migrated automatically and can be safely removed from your config file.
+Legacy configuration keys (`project_modes`, `cloud_projects`, `default_project_mode`) can be safely removed from your config file after migration.
 
 ---