Finally fix db encryption between restarts (#98)

This pull request introduces major improvements to SQLCipher
integration, schema migration reliability, and cross-platform container
builds. The changes ensure that encrypted databases are initialized
securely and consistently, improve schema migration error handling, and
transition the build/runtime containers from Alpine to Ubuntu for better
compatibility. Additionally, the documentation and tests have been
updated to reflect these changes and to ensure encrypted database
headers.

**SQLCipher Integration and Database Initialization:**

* Refactored `internal/database/db.go` to construct SQLCipher DSNs with
all required pragmas and encryption key, ensuring settings are applied
before any page reads; removed post-open PRAGMA calls and inlined DSN
generation for both file and in-memory databases.
[[1]](diffhunk://#diff-d18caea19014a2e48b9dc31a7831f7a43b1c3068421642110862803da915e00dL30-R68)
[[2]](diffhunk://#diff-d18caea19014a2e48b9dc31a7831f7a43b1c3068421642110862803da915e00dR111-R162)
* Added a new `verifyKeyUsable` function to confirm the encryption key
can decrypt the schema, improving error reporting and reliability.
* Updated `AGENTS.md` to clarify that SQLCipher connection settings must
be applied via DSN, not post-open PRAGMA calls.

**Schema Migration and Reliability:**

* Changed schema migration logic to execute each SQL statement
individually, avoiding multi-statement driver errors and surfacing clear
failure contexts; added the `execSQLStatements` helper and improved
error handling for table info queries.
[[1]](diffhunk://#diff-d18caea19014a2e48b9dc31a7831f7a43b1c3068421642110862803da915e00dL214-R247)
[[2]](diffhunk://#diff-d18caea19014a2e48b9dc31a7831f7a43b1c3068421642110862803da915e00dL319-L323)
[[3]](diffhunk://#diff-d18caea19014a2e48b9dc31a7831f7a43b1c3068421642110862803da915e00dR360-R367)
[[4]](diffhunk://#diff-d18caea19014a2e48b9dc31a7831f7a43b1c3068421642110862803da915e00dR419-L401)
[[5]](diffhunk://#diff-d18caea19014a2e48b9dc31a7831f7a43b1c3068421642110862803da915e00dR480-R487)
* Updated documentation in `AGENTS.md` to require that schema migration
SQL statements be terminated with semicolons and comments placed on
their own lines.

**Container Build and Runtime Platform:**

* Migrated Docker build and runtime images from Alpine to Ubuntu 24.04,
including installation of SQLCipher and Go toolchain, and adjusted
build/run scripts for glibc compatibility and cross-arch support.
[[1]](diffhunk://#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557L2-R2)
[[2]](diffhunk://#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557L21-R50)
[[3]](diffhunk://#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557L49-R76)
[[4]](diffhunk://#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557L61-R97)

**Testing and Validation:**

* Added a new test to ensure databases are created with encrypted
headers (not plaintext), verifying SQLCipher is applied from the start.

**Documentation and Miscellaneous:**

* Updated encryption key generation and documentation to use a
consistent secret filename and clarified persistence requirements for
production.
[[1]](diffhunk://#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52L187-R189)
[[2]](diffhunk://#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R222-R223)
* Updated Go module to use a custom fork of `go-sqlite3` for
compatibility with SQLCipher and libsqlite3 tag.

Let me know if you want to dive deeper into any of these areas!

Author: nipuna-perera

AGENTS.md

@@ -34,4 +34,5 @@
 - Never commit secrets. Copy `.env.example` to `.env` for local dev.
 - SQLCipher is mandatory; run `make dev-setup` and follow README notes. In prod, provide 32-byte key via Docker secret.
 - Required env: `SIGNAL_PHONE_NUMBER`, AI provider vars. Backend listens on `LISTEN_ADDR` (default `:8081`).
-
+- SQLCipher connection pragmas are applied via DSN inside `internal/database`; avoid adding post-open `PRAGMA` calls (especially `cipher_migrate`).
+- Schema migrations execute statements individually—when adding SQL to `schema.sql`, ensure each statement is terminated with a semicolon and that comments stand on their own lines.

Dockerfile

@@ -1,5 +1,5 @@
 # Stage 1: Build Next.js frontend
-FROM node:24-alpine AS frontend-builder
+FROM node:24-bookworm AS frontend-builder
 
 WORKDIR /app/web
 
@@ -18,10 +18,36 @@ COPY web/ ./
 RUN npm run build
 
 # Stage 2: Build Go backend
-FROM golang:1.25-alpine AS backend-builder
-
-# Install SQLCipher and build dependencies, including static library
-RUN apk add --no-cache gcc musl-dev sqlite-dev sqlite-static sqlcipher-dev pkgconf openssl-dev openssl-libs-static
+FROM ubuntu:24.04 AS backend-builder
+
+ARG GO_VERSION=1.25.1
+ENV DEBIAN_FRONTEND=noninteractive
+ENV PATH=/usr/local/go/bin:$PATH
+
+# Install toolchain, SQLCipher dev headers, and download Go
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends \
+      build-essential pkg-config curl ca-certificates libsqlcipher-dev libssl-dev && \
+    rm -rf /var/lib/apt/lists/* && \
+    arch=$(dpkg --print-architecture) && \
+    case "$arch" in \
+      amd64) GOARCH=amd64 ;; \
+      arm64) GOARCH=arm64 ;; \
+      *) echo "Unsupported arch: $arch" && exit 1 ;; \
+    esac && \
+    curl -fsSL https://go.dev/dl/go${GO_VERSION}.linux-${GOARCH}.tar.gz -o /tmp/go.tgz && \
+    tar -C /usr/local -xzf /tmp/go.tgz && \
+    rm /tmp/go.tgz && \
+    # Ensure go is available
+    go version && \
+    # Align lib name expected by go-sqlite3 when using libsqlite3 tag
+    bash -lc 'libdir=$(dpkg-architecture -qDEB_HOST_MULTIARCH 2>/dev/null || echo aarch64-linux-gnu); \
+      if [ -e /usr/lib/$libdir/libsqlcipher.so ] && [ ! -e /usr/lib/$libdir/libsqlite3.so ]; then \
+        ln -s /usr/lib/$libdir/libsqlcipher.so /usr/lib/$libdir/libsqlite3.so; \
+      fi'
+
+# Enable CGO globally; actual flags are set at build invocation time below
+ENV CGO_ENABLED=1
 
 WORKDIR /app
 
@@ -46,9 +72,8 @@ ARG GIT_COMMIT=unknown
 ARG BUILD_TIME=unknown
 ARG GOARCH
 
-RUN CGO_ENABLED=1 \
-    CGO_CFLAGS="$(pkg-config --cflags sqlcipher) -DSQLITE_HAS_CODEC" \
-    CGO_LDFLAGS="$(pkg-config --libs sqlcipher) $(pkg-config --libs libcrypto libssl)" \
+RUN CGO_CFLAGS="-I/usr/include/sqlcipher -DSQLITE_HAS_CODEC" \
+    CGO_LDFLAGS="-lsqlcipher -lssl -lcrypto" \
     go build \
     -tags="sqlite_crypt libsqlite3" \
     -ldflags="-w -s \
@@ -58,14 +83,18 @@ RUN CGO_ENABLED=1 \
     -o summarizarr \
     ./cmd/summarizarr
 
-# Stage 3: Runtime
-FROM alpine:latest
+# Stage 3: Runtime (Debian/Ubuntu, glibc)
+FROM ubuntu:24.04
 
-# Update package index and add ca certificates, utilities, and SQLCipher runtime libraries
-RUN apk update && \
-    apk --no-cache add ca-certificates wget sqlcipher-libs openssl && \
-    addgroup -g 1001 summarizarr && \
-    adduser -D -s /bin/sh -u 1001 -G summarizarr summarizarr
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends ca-certificates wget sqlcipher && \
+    rm -rf /var/lib/apt/lists/* && \
+    bash -lc 'libdir=$(dpkg-architecture -qDEB_HOST_MULTIARCH 2>/dev/null || echo aarch64-linux-gnu); \
+      if [ -e /usr/lib/$libdir/libsqlcipher.so ] && [ ! -e /usr/lib/$libdir/libsqlite3.so ]; then \
+        ln -s /usr/lib/$libdir/libsqlcipher.so /usr/lib/$libdir/libsqlite3.so; \
+      fi' && \
+    groupadd -g 1001 summarizarr && \
+    useradd -u 1001 -g 1001 -M -s /bin/sh summarizarr
 
 WORKDIR /app
 
@@ -99,4 +128,4 @@ ENV DATABASE_PATH=/data/summarizarr.db
 ENV AI_PROVIDER=local
 ENV SUMMARIZATION_INTERVAL=1h
 
-ENTRYPOINT ["./summarizarr"]
+ENTRYPOINT ["./summarizarr"]

Makefile

@@ -184,9 +184,9 @@ dev-setup-encrypted: dev-key ## Setup development environment with encryption en
 
 prod-key: ## Generate production encryption key
 	@mkdir -p docker/secrets
-	@openssl rand -hex 32 > docker/secrets/sqlcipher_encryption_key
-	@chmod 600 docker/secrets/sqlcipher_encryption_key
-	@echo "$(GREEN)Generated production encryption key in docker/secrets/sqlcipher_encryption_key$(NC)"
+	@openssl rand -hex 32 > docker/secrets/encryption_key
+	@chmod 600 docker/secrets/encryption_key
+	@echo "$(GREEN)Generated production encryption key in docker/secrets/encryption_key$(NC)"
 	@echo "$(RED)IMPORTANT: Back up this key securely!$(NC)"
 
 test-backend: ## Test Go backend with SQLCipher

README.md

@@ -219,6 +219,8 @@ Keys are 32-byte (64 hex characters) for AES-256. Never commit keys to version c
 
 Note: Databases must be encrypted from the first run. There is no supported migration from unencrypted databases.
 
+Tip: Ensure `./data` exists and is writable before starting with `make docker` or Compose. Both the database file and `encryption.key` persist in this directory.
+
 ## Production Deployment
 
 ### Container Registry
@@ -259,4 +261,4 @@ MIT License - see [LICENSE](LICENSE) file for details.
 - [Signal CLI REST API](https://github.com/bbernhard/signal-cli-rest-api) - Signal integration
 - [Ollama](https://ollama.ai/) - Local AI capabilities
 - [Next.js](https://nextjs.org/) - Modern web framework
-- [Shadcn/ui](https://ui.shadcn.com/) - UI components
+- [Shadcn/ui](https://ui.shadcn.com/) - UI components

cmd/summarizarr/main.go

@@ -6,6 +6,7 @@ import (
 	"fmt"
 	"io"
 	"log/slog"
+	"crypto/sha256"
 	"os"
 	"os/signal"
 	"path/filepath"
@@ -59,6 +60,14 @@ func main() {
 		slog.Error("Failed to obtain encryption key", "error", err)
 		os.Exit(1)
 	}
+	// Log masked key fingerprint and source for troubleshooting in dev
+	{
+		finger := sha256.Sum256([]byte(encKey))
+		src := "generated"
+		if _, err := os.Stat(encMgr.SecretPath); err == nil { src = encMgr.SecretPath }
+		if _, err := os.Stat(encMgr.KeyFile); err == nil { src = encMgr.KeyFile }
+		slog.Info("Using encryption key", "source", src, "sha256_prefix", fmt.Sprintf("%x", finger)[:8])
+	}
 
 	db, err := database.NewDB(cfg.DatabasePath, encKey)
 	if err != nil {

go.mod

@@ -6,9 +6,9 @@ require (
 	github.com/alexedwards/scs/sqlite3store v0.0.0-20231113091146-cef4b05350c8
 	github.com/alexedwards/scs/v2 v2.9.0
 	github.com/coder/websocket v1.8.13
-	github.com/mattn/go-sqlite3 v1.14.32
+	github.com/mattn/go-sqlite3 v1.14.17-0.20240122133042-fb824c8e339e
 	github.com/sashabaranov/go-openai v1.41.1
 	golang.org/x/crypto v0.41.0
 )
 
-replace github.com/mattn/go-sqlite3 => github.com/mattn/go-sqlite3 v1.14.32
+replace github.com/mattn/go-sqlite3 => github.com/jgiannuzzi/go-sqlite3 v1.14.17-0.20240122133042-fb824c8e339e

go.sum

@@ -4,8 +4,8 @@ github.com/alexedwards/scs/v2 v2.9.0 h1:xa05mVpwTBm1iLeTMNFfAWpKUm4fXAW7CeAViqBV
 github.com/alexedwards/scs/v2 v2.9.0/go.mod h1:ToaROZxyKukJKT/xLcVQAChi5k6+Pn1Gvmdl7h3RRj8=
 github.com/coder/websocket v1.8.13 h1:f3QZdXy7uGVz+4uCJy2nTZyM0yTBj8yANEHhqlXZ9FE=
 github.com/coder/websocket v1.8.13/go.mod h1:LNVeNrXQZfe5qhS9ALED3uA+l5pPqvwXg3CKoDBB2gs=
-github.com/mattn/go-sqlite3 v1.14.32 h1:JD12Ag3oLy1zQA+BNn74xRgaBbdhbNIDYvQUEuuErjs=
-github.com/mattn/go-sqlite3 v1.14.32/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
+github.com/jgiannuzzi/go-sqlite3 v1.14.17-0.20240122133042-fb824c8e339e h1:0s+RSCZSEaZ3IM4fqMH1Nf5UDpttrn1R2QDutqiL2Zc=
+github.com/jgiannuzzi/go-sqlite3 v1.14.17-0.20240122133042-fb824c8e339e/go.mod h1:2eHXhiwb8IkHr+BDWZGa96P6+rkvnG63S2DGjv9HUNg=
 github.com/sashabaranov/go-openai v1.41.1 h1:zf5tM+GuxpyiyD9XZg8nCqu52eYFQg9OOew0gnIuDy4=
 github.com/sashabaranov/go-openai v1.41.1/go.mod h1:lj5b/K+zjTSFxVLijLSTDZuP7adOgerWeFyZLUhAKRg=
 golang.org/x/crypto v0.41.0 h1:WKYxWedPGCTVVl5+WHSSrOBT0O8lx32+zxmHxijgXp4=