Merge branch 'main' into fix/integration-test-failures

Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>

Author: phernandez

.claude/settings.json

@@ -0,0 +1,5 @@
+{
+  "enabledPlugins": {
+    "basic-memory@basicmachines": true
+  }
+}

.github/workflows/test.yml

@@ -78,21 +78,7 @@ jobs:
         python-version: [ "3.12", "3.13" ]
     runs-on: ubuntu-latest
 
-    # Postgres service (only available on Linux runners)
-    services:
-      postgres:
-        image: postgres:17
-        env:
-          POSTGRES_DB: basic_memory_test
-          POSTGRES_USER: basic_memory_user
-          POSTGRES_PASSWORD: dev_password
-        options: >-
-          --health-cmd pg_isready
-          --health-interval 10s
-          --health-timeout 5s
-          --health-retries 5
-        ports:
-          - 5433:5432
+    # Note: No services section needed - testcontainers handles Postgres in Docker
 
     steps:
       - uses: actions/checkout@v4
@@ -121,7 +107,7 @@ jobs:
         run: |
           uv pip install -e .[dev]
 
-      - name: Run tests (Postgres)
+      - name: Run tests (Postgres via testcontainers)
         run: |
           uv pip install pytest pytest-cov
           just test-postgres

CHANGELOG.md

@@ -1,5 +1,98 @@
 # CHANGELOG
 
+## v0.16.3 (2025-12-20)
+
+### Features
+
+- **#439**: Add PostgreSQL database backend support
+  ([`fb5e9e1`](https://github.com/basicmachines-co/basic-memory/commit/fb5e9e1))
+  - Full PostgreSQL/Neon database support as alternative to SQLite
+  - Async connection pooling with asyncpg
+  - Alembic migrations support for both backends
+  - Configurable via `BASIC_MEMORY_DATABASE_BACKEND` environment variable
+
+- **#441**: Implement API v2 with ID-based endpoints (Phase 1)
+  ([`28cc522`](https://github.com/basicmachines-co/basic-memory/commit/28cc522))
+  - New ID-based API endpoints for improved performance
+  - Foundation for future API enhancements
+  - Backward compatible with existing endpoints
+
+- Add project_id to Relation and Observation for efficient project-scoped queries
+  ([`a920a9f`](https://github.com/basicmachines-co/basic-memory/commit/a920a9f))
+  - Enables faster queries in multi-project environments
+  - Improved database schema for cloud deployments
+
+- Add bulk insert with ON CONFLICT handling for relations
+  ([`0818bda`](https://github.com/basicmachines-co/basic-memory/commit/0818bda))
+  - Faster relation creation during sync operations
+  - Handles duplicate relations gracefully
+
+### Performance
+
+- Lightweight permalink resolution to avoid eager loading
+  ([`6f99d2e`](https://github.com/basicmachines-co/basic-memory/commit/6f99d2e))
+  - Reduces database queries during entity lookups
+  - Improved response times for read operations
+
+### Bug Fixes
+
+- **#464**: Pin FastMCP to 2.12.3 to fix MCP tools visibility
+  ([`f227ef6`](https://github.com/basicmachines-co/basic-memory/commit/f227ef6))
+  - Fixes issue where MCP tools were not visible to Claude
+  - Reverts to last known working FastMCP version
+
+- **#458**: Reduce watch service CPU usage by increasing reload interval
+  ([`897b1ed`](https://github.com/basicmachines-co/basic-memory/commit/897b1ed))
+  - Lowers CPU usage during file watching
+  - More efficient resource utilization
+
+- **#456**: Await background sync task cancellation in lifespan shutdown
+  ([`efbc758`](https://github.com/basicmachines-co/basic-memory/commit/efbc758))
+  - Prevents hanging on shutdown
+  - Clean async task cleanup
+
+- **#434**: Respect --project flag in background sync
+  ([`70bb10b`](https://github.com/basicmachines-co/basic-memory/commit/70bb10b))
+  - Background sync now correctly uses specified project
+  - Fixes multi-project sync issues
+
+- **#446**: Fix observation parsing and permalink limits
+  ([`73d940e`](https://github.com/basicmachines-co/basic-memory/commit/73d940e))
+  - Handles edge cases in observation content
+  - Prevents permalink truncation issues
+
+- **#424**: Handle periods in kebab_filenames mode
+  ([`b004565`](https://github.com/basicmachines-co/basic-memory/commit/b004565))
+  - Fixes filename handling for files with multiple periods
+  - Improved kebab-case conversion
+
+- Fix Postgres/Neon connection settings and search index dedupe
+  ([`b5d4fb5`](https://github.com/basicmachines-co/basic-memory/commit/b5d4fb5))
+  - Optimized connection pooling for Postgres
+  - Prevents duplicate search index entries
+
+### Testing & CI
+
+- Replace py-pglite with testcontainers for Postgres testing
+  ([`c462faf`](https://github.com/basicmachines-co/basic-memory/commit/c462faf))
+  - More reliable Postgres testing infrastructure
+  - Uses Docker-based test containers
+
+- Add PostgreSQL testing to GitHub Actions workflow
+  ([`66b91b2`](https://github.com/basicmachines-co/basic-memory/commit/66b91b2))
+  - CI now tests both SQLite and PostgreSQL backends
+  - Ensures cross-database compatibility
+
+- **#416**: Add integration test for read_note with underscored folders
+  ([`0c12a39`](https://github.com/basicmachines-co/basic-memory/commit/0c12a39))
+  - Verifies folder name handling edge cases
+
+### Internal
+
+- Cloud compatibility fixes and performance improvements (#454)
+- Remove logfire instrumentation for cleaner production deployments
+- Truncate content_stems to fix Postgres 8KB index row limit
+
 ## v0.16.2 (2025-11-16)
 
 ### Bug Fixes

CLAUDE.md

@@ -15,10 +15,14 @@ See the [README.md](README.md) file for a project overview.
 ### Build and Test Commands
 
 - Install: `just install` or `pip install -e ".[dev]"`
-- Run all tests (with coverage): `just test` - Runs both unit and integration tests with unified coverage
-- Run unit tests only: `just test-unit` - Fast, no coverage
-- Run integration tests only: `just test-int` - Fast, no coverage
-- Generate HTML coverage: `just coverage` - Opens in browser
+- Run all tests (SQLite + Postgres): `just test`
+- Run all tests against SQLite: `just test-sqlite`
+- Run all tests against Postgres: `just test-postgres` (uses testcontainers)
+- Run unit tests (SQLite): `just test-unit-sqlite`
+- Run unit tests (Postgres): `just test-unit-postgres`
+- Run integration tests (SQLite): `just test-int-sqlite`
+- Run integration tests (Postgres): `just test-int-postgres`
+- Generate HTML coverage: `just coverage`
 - Single test: `pytest tests/path/to/test_file.py::test_function_name`
 - Run benchmarks: `pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"`
 - Lint: `just lint` or `ruff check . --fix`
@@ -30,6 +34,8 @@ See the [README.md](README.md) file for a project overview.
 
 **Note:** Project requires Python 3.12+ (uses type parameter syntax and `type` aliases introduced in 3.12)
 
+**Postgres Testing:** Uses [testcontainers](https://testcontainers-python.readthedocs.io/) which automatically spins up a Postgres instance in Docker. No manual database setup required - just have Docker running.
+
 ### Test Structure
 
 - `tests/` - Unit tests for individual components (mocked, fast)
@@ -76,8 +82,10 @@ See the [README.md](README.md) file for a project overview.
 - SQLite is used for indexing and full text search, files are source of truth
 - Testing uses pytest with asyncio support (strict mode)
 - Unit tests (`tests/`) use mocks when necessary; integration tests (`test-int/`) use real implementations
-- Test database uses in-memory SQLite
-- Each test runs in a standalone environment with in-memory SQLite and tmp_file directory
+- By default, tests run against SQLite (fast, no Docker needed)
+- Set `BASIC_MEMORY_TEST_POSTGRES=1` to run against Postgres (uses testcontainers - Docker required)
+- Each test runs in a standalone environment with isolated database and tmp_path directory
+- CI runs SQLite and Postgres tests in parallel for faster feedback
 - Performance benchmarks are in `test-int/test_sync_performance_benchmark.py`
 - Use pytest markers: `@pytest.mark.benchmark` for benchmarks, `@pytest.mark.slow` for slow tests
 
@@ -229,6 +237,11 @@ of using AI just for code generation, we've developed a true collaborative workf
 This approach has allowed us to tackle more complex challenges and build a more robust system than either humans or AI
 could achieve independently.
 
+**Problem-Solving Guidance:**
+- If a solution isn't working after reasonable effort, suggest alternative approaches
+- Don't persist with a problematic library or pattern when better alternatives exist
+- Example: When py-pglite caused cascading test failures, switching to testcontainers-postgres was the right call
+
 ## GitHub Integration
 
 Basic Memory has taken AI-Human collaboration to the next level by integrating Claude directly into the development workflow through GitHub:

README.md

@@ -433,42 +433,76 @@ See the [Documentation](https://memory.basicmachines.co/) for more info, includi
 - [Managing multiple Projects](https://docs.basicmemory.com/guides/cli-reference/#project)
 - [Importing data from OpenAI/Claude Projects](https://docs.basicmemory.com/guides/cli-reference/#import)
 
+## Logging
+
+Basic Memory uses [Loguru](https://github.com/Delgan/loguru) for logging. The logging behavior varies by entry point:
+
+| Entry Point | Default Behavior | Use Case |
+|-------------|------------------|----------|
+| CLI commands | File only | Prevents log output from interfering with command output |
+| MCP server | File only | Stdout would corrupt the JSON-RPC protocol |
+| API server | File (local) or stdout (cloud) | Docker/cloud deployments use stdout |
+
+**Log file location:** `~/.basic-memory/basic-memory.log` (10MB rotation, 10 days retention)
+
+### Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `BASIC_MEMORY_LOG_LEVEL` | `INFO` | Log level: DEBUG, INFO, WARNING, ERROR |
+| `BASIC_MEMORY_CLOUD_MODE` | `false` | When `true`, API logs to stdout with structured context |
+| `BASIC_MEMORY_ENV` | `dev` | Set to `test` for test mode (stderr only) |
+
+### Examples
+
+```bash
+# Enable debug logging
+BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory sync
+
+# View logs
+tail -f ~/.basic-memory/basic-memory.log
+
+# Cloud/Docker mode (stdout logging with structured context)
+BASIC_MEMORY_CLOUD_MODE=true uvicorn basic_memory.api.app:app
+```
+
 ## Development
 
 ### Running Tests
 
-Basic Memory supports dual database backends (SQLite and Postgres). Tests are parametrized to run against both backends automatically.
+Basic Memory supports dual database backends (SQLite and Postgres). By default, tests run against SQLite. Set `BASIC_MEMORY_TEST_POSTGRES=1` to run against Postgres (uses testcontainers - Docker required).
 
 **Quick Start:**
 ```bash
-# Run SQLite tests (default, no Docker needed)
+# Run all tests against SQLite (default, fast)
 just test-sqlite
 
-# Run Postgres tests (requires Docker)
+# Run all tests against Postgres (uses testcontainers)
 just test-postgres
+
+# Run both SQLite and Postgres tests
+just test
 ```
 
 **Available Test Commands:**
 
-- `just test-sqlite` - Run tests against SQLite only (fastest, no Docker needed)
-- `just test-postgres` - Run tests against Postgres only (requires Docker)
+- `just test` - Run all tests against both SQLite and Postgres
+- `just test-sqlite` - Run all tests against SQLite (fast, no Docker needed)
+- `just test-postgres` - Run all tests against Postgres (uses testcontainers)
+- `just test-unit-sqlite` - Run unit tests against SQLite
+- `just test-unit-postgres` - Run unit tests against Postgres
+- `just test-int-sqlite` - Run integration tests against SQLite
+- `just test-int-postgres` - Run integration tests against Postgres
 - `just test-windows` - Run Windows-specific tests (auto-skips on other platforms)
 - `just test-benchmark` - Run performance benchmark tests
-- `just test-all` - Run all tests including Windows, Postgres, and benchmarks
-
-**Postgres Testing Requirements:**
 
-To run Postgres tests, you need to start the test database:
-```bash
-docker-compose -f docker-compose-postgres.yml up -d
-```
+**Postgres Testing:**
 
-Tests will connect to `localhost:5433/basic_memory_test`.
+Postgres tests use [testcontainers](https://testcontainers-python.readthedocs.io/) which automatically spins up a Postgres instance in Docker. No manual database setup required - just have Docker running.
 
 **Test Markers:**
 
 Tests use pytest markers for selective execution:
-- `postgres` - Tests that run against Postgres backend
 - `windows` - Windows-specific database optimizations
 - `benchmark` - Performance tests (excluded from default runs)
 

justfile

@@ -7,44 +7,60 @@ install:
     @echo ""
     @echo "💡 Remember to activate the virtual environment by running: source .venv/bin/activate"
 
-# Run all tests with unified coverage report
-test: test-unit test-int
-
-# Run unit tests only (fast, no coverage)
-test-unit:
-    uv run pytest -p pytest_mock -v --no-cov tests
-
-# Run integration tests only (fast, no coverage)
-test-int:
-    uv run pytest -p pytest_mock -v --no-cov test-int
-
 # ==============================================================================
 # DATABASE BACKEND TESTING
 # ==============================================================================
 # Basic Memory supports dual database backends (SQLite and Postgres).
-# Tests are parametrized to run against both backends automatically.
+# By default, tests run against SQLite (fast, no dependencies).
+# Set BASIC_MEMORY_TEST_POSTGRES=1 to run against Postgres (uses testcontainers).
 #
 # Quick Start:
-#   just test-sqlite    # Run SQLite tests (default, no Docker needed)
-#   just test-postgres  # Run Postgres tests (requires Docker)
+#   just test              # Run all tests against SQLite (default)
+#   just test-sqlite       # Run all tests against SQLite
+#   just test-postgres     # Run all tests against Postgres (testcontainers)
+#   just test-unit-sqlite  # Run unit tests against SQLite
+#   just test-unit-postgres # Run unit tests against Postgres
+#   just test-int-sqlite   # Run integration tests against SQLite
+#   just test-int-postgres # Run integration tests against Postgres
 #
-# For Postgres tests, first start the database:
-#   docker-compose -f docker-compose-postgres.yml up -d
+# CI runs both in parallel for faster feedback.
 # ==============================================================================
 
-# Run tests against SQLite only (default backend, skip Postgres/Benchmark tests)
-# This is the fastest option and doesn't require any Docker setup.
-# Use this for local development and quick feedback.
-# Includes Windows-specific tests which will auto-skip on non-Windows platforms.
-test-sqlite:
-    uv run pytest -p pytest_mock -v --no-cov -m "not postgres and not benchmark" tests test-int
-
-# Run tests against Postgres only (requires docker-compose-postgres.yml up)
-# First start Postgres: docker-compose -f docker-compose-postgres.yml up -d
-# Tests will connect to localhost:5433/basic_memory_test
-# To reset the database: just postgres-reset
-test-postgres:
-    uv run pytest -p pytest_mock -v --no-cov -m "postgres and not benchmark" tests test-int
+# Run all tests against SQLite and Postgres
+test: test-sqlite test-postgres
+
+# Run all tests against SQLite
+test-sqlite: test-unit-sqlite test-int-sqlite
+
+# Run all tests against Postgres (uses testcontainers)
+test-postgres: test-unit-postgres test-int-postgres
+
+# Run unit tests against SQLite
+test-unit-sqlite:
+    uv run pytest -p pytest_mock -v --no-cov tests
+
+# Run unit tests against Postgres
+test-unit-postgres:
+    BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov tests
+
+# Run integration tests against SQLite
+test-int-sqlite:
+    uv run pytest -p pytest_mock -v --no-cov test-int
+
+# Run integration tests against Postgres
+# Note: Uses timeout due to FastMCP Client + asyncpg cleanup hang (tests pass, process hangs on exit)
+# See: https://github.com/jlowin/fastmcp/issues/1311
+test-int-postgres:
+    #!/usr/bin/env bash
+    set -euo pipefail
+    # Use gtimeout (macOS/Homebrew) or timeout (Linux)
+    TIMEOUT_CMD=$(command -v gtimeout || command -v timeout || echo "")
+    if [[ -n "$TIMEOUT_CMD" ]]; then
+        $TIMEOUT_CMD --signal=KILL 600 bash -c 'BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov test-int' || test $? -eq 137
+    else
+        echo "⚠️  No timeout command found, running without timeout..."
+        BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov test-int
+    fi
 
 # Reset Postgres test database (drops and recreates schema)
 # Useful when Alembic migration state gets out of sync during development