Skip to content

development.md

Latest commit

 

History

History
244 lines (174 loc) · 9.49 KB

development.md

File metadata and controls

244 lines (174 loc) · 9.49 KB
Raw
title Development & Setup
description Guide to set up the development environment, spin up the database, and run the Scala backend.

Follow this guide to set up the development environment, spin up the database, and run the Scala backend.

Prerequisites

Ensure you have the following installed on your system:

  • mise-en-place (environment manager and task runner)
  • sbt (Scala build tool; brew install sbt)
  • A Docker-compatible container runtime for running PostgreSQL locally (e.g. Rancher Desktop or Colima)
  • GitHub CLI authenticated with your account — the backend depends on the lv.id.jc:dicechess-engine-scala artifact from GitHub Packages, which requires authentication even for public packages

Quickstart

1. Project Initialization

Run the setup task to install pinned tools and register lefthook git hooks:

mise run setup

2. Start PostgreSQL Container

Spin up the local PostgreSQL database instance in the background using Docker Compose:

mise run db:up

This runs a PostgreSQL instance listening on port 5432 with:

  • Database: dicechess_analytics
  • Username: dicechess_user
  • Password: dicechess_password

3. Run the API Server

mise run run

Database migrations are applied automatically: the backend runs Flyway on startup, so there is no separate migration step. Migration scripts live in src/main/resources/db/migration/.

Once started, the interactive API documentation (Swagger UI, generated from the Tapir endpoint definitions) is available at:

GitHub Packages Authentication

The backend depends on the game engine, lv.id.jc:dicechess-engine-scala, published to GitHub Packages Maven. Unlike Maven Central, GitHub Packages requires authentication even for public packages (a token with the read:packages scope) — so any sbt command that resolves dependencies needs a username and token.

Rather than repeat that credential dance in every task, the build resolves it once, in build.sbt:

GitHub Packages validates only the token — any non-empty username is accepted — so the build never needs a network call to discover the account name. (credentials is an sbt setting, evaluated on every load, so a network lookup here would slow down every command and break offline work.) The token is resolved as follows:

  1. In CI, the workflow exports GITHUB_TOKEN; the build uses it directly.
  2. Locally, when that variable is absent, the build reads it from the GitHub CLI via gh auth token, which returns the token from the OS keychain without touching the network — so it works offline and the token is never written to a file or a shell profile.

This is why the Scala build tasks are plain sbt ... with no credential prefix, and why a bare sbt invocation works too: just keep gh auth login current.

Task Runner Reference

mise tasks are configured in mise.toml for easy execution:

Command Description
mise run setup Installs pinned tools and registers lefthook git hooks.
mise run db:up Launches only the PostgreSQL container.
mise run db:down Stops and removes only the PostgreSQL container (data volume survives).
mise run stack:up Starts db + api + ui from published images.
mise run stack:down Stops and removes all compose services.
mise run check Repo-wide gate: scalafmt check plus coverage-gated tests on real PostgreSQL.
mise run format Runs scalafmt across the Scala sources.
mise run compile Compiles the backend.
mise run test Runs the test suite without the coverage/clean overhead.
mise run run Starts the API server on port 8000.
mise run eval:mc One-shot: evaluates Monte-Carlo accuracy against empirical DB stats.
mise run db:export-book Exports the opening book for the bots — see Opening-book export.
mise run docs:dev Starts the Astro/Starlight dev server at http://localhost:4321.
mise run docs:build Compiles the documentation site into static HTML inside docs/dist/.

Configuration

Environment variables (compatible with docker-compose):

Variable Default Notes
DATABASE_URL — postgres://, postgresql://, or postgresql+asyncpg:// accepted (the +asyncpg form is kept only for legacy .env compatibility — it is rewritten to a JDBC URL)
POSTGRES_HOST/PORT/DB/USER/PASSWORD docker-compose defaults used when DATABASE_URL is absent
HTTP_HOST / HTTP_PORT 0.0.0.0 / 8000
CORS_ORIGINS http://localhost:5173,http://localhost:3000 comma-separated

Rancher Desktop / testcontainers notes

The test suite uses testcontainers against a real PostgreSQL. On Rancher Desktop two machine-local accommodations are needed:

  1. ~/.testcontainers.properties:

    docker.host=unix\:///Users/<you>/.rd/docker.sock

  2. mise.local.toml at the repo root (gitignored), so every task gets the variable regardless of the shell session — the ryuk cleanup sidecar cannot start against the Rancher moby VM, so cleanup falls back to JVM shutdown hooks. CI on ubuntu-latest keeps ryuk enabled.

    [env]
TESTCONTAINERS_RYUK_DISABLED = "true"

The Docker API version is pinned to 1.43 via Test / javaOptions in build.sbt: docker-java does not negotiate and its default (1.32) is rejected by Docker 29+ daemons.

Git Hooks

mise run setup registers two lefthook tiers:

  • pre-commit: betterleaks secret scan + native scalafmt check on staged files (milliseconds per commit).
  • pre-push: a hermetic full-module scalafmt check (~1s). Tests deliberately stay in CI.

Run all pre-commit jobs manually across the whole codebase with mise run hook:run.

Data Ingestion

The historical archive (140k+ games from the frozen dicechess-lab SQLite database) has already been imported into the production PostgreSQL instance — the one-time Python ETL that performed it was retired together with the rest of the Python codebase.

Ongoing ingestion uses the transactional POST /api/games endpoint, which validates every game against dicechess-engine-scala before persisting it. See the Game Ingestion page for the contract.

Opening-book export

db:export-book generates opening_book.json — the data-driven opening book the engine's bots consume (OpeningBookBot). For every (position, dice) reached often enough in strong games it stores the continuation with the best win rate from the moving side's perspective.

mise run db:export-book [minGames] [minRating] [outputPath]
# defaults: 100   2000   opening_book.json
  • minGames — minimum games a continuation needs, after filtering, to be booked.
  • minRating — both players must be at least this strong (0 disables the strength filter).
  • Forced passes (no legal move for the roll) are excluded — they are not bookable moves.
  • On startup the app logs the target database (host/db and user, never the password), so you can confirm which database you are reading from before it runs.

Choosing the database

The exporter reads whatever DATABASE_URL / POSTGRES_* resolve to (see Configuration) — by default the local dev database, which is small, so the book comes out nearly empty. For a real book, point it at the full production database.

Keep the production URL in a git-ignored mise.prod.local.toml at the repo root, activated only under the prod environment so it never affects your other commands (e.g. a local mise run run):

# mise.prod.local.toml  — git-ignored via the mise.*.local.toml pattern
[env]
DATABASE_URL = "postgresql://dicechess_user:<PROD_PASSWORD>@<prod-host>:5432/dicechess_analytics"

mise config precedence is mise.{env}.local.toml > mise.local.toml > mise.{env}.toml > mise.toml, and without MISE_ENV set the prod files are ignored entirely. The -E / --env flag is the cleanest, shell-agnostic way to activate it for a single run:

mise -E prod run db:export-book 100 2000

PowerShell

The -E flag behaves identically in PowerShell — no environment-variable juggling:

mise -E prod run db:export-book 100 2000

Create the local config file from PowerShell with a here-string:

@'
[env]
DATABASE_URL = "postgresql://dicechess_user:<PROD_PASSWORD>@<prod-host>:5432/dicechess_analytics"
'@ | Set-Content -Path mise.prod.local.toml -Encoding utf8

:::caution Prefer mise -E prod run … over $env:MISE_ENV = 'prod'. In PowerShell a $env: assignment persists for the whole session, so a later mise run run could unexpectedly hit production. The -E flag scopes the environment to that one invocation. :::

The export is read-only (a pure aggregation), so it is safe to run against production — prefer off-peak hours, as it scans the full turns / games history. The resulting opening_book.json is keyed by the canonical position+dice key shared with the engine and is loaded into the bots at runtime via the engine's registerOpeningBookBot.