feat: add bazel-diff serve HTTP query service (#29) (#393)

* feat: add `bazel-diff serve` HTTP query service (#29)

Implements the "bazel-diff as a service" model from the BazelCon talk that
inspired this repo (issue #29), as a first self-contained increment: core
service + per-SHA hash cache.

New `com.bazel_diff.server` package + `serve` subcommand:
- HTTP/JSON server on the JDK's com.sun.net.httpserver (no new dependency)
  with GET /health and GET /impacted_targets?from&to[&targetType].
- Per-SHA hash cache behind a byte-oriented HashCacheStorage interface
  (LocalDiskHashCacheStorage impl, atomic writes); cache key folds in a
  fingerprint of the query-affecting flags. The interface is ready for a
  remote (e.g. S3) backend.
- GitClient/ProcessGitClient: subprocess git fetch/resolve/checkout.
- HashService (per-SHA generate+cache, single workspace lock) and
  ImpactedTargetsService, which reuse CalculateImpactedTargetsInteractor so
  results match `get-impacted-targets`.
- Initial `git fetch` gates readiness; a fetch failure "lame-ducks" the
  instance (health stays 503) rather than self-repairing.

Query-affecting flags mirror `generate-hashes`, so hashes the service
produces match a cold CLI run.

Tests: unit tests per component plus a real end-to-end test
(E2ETest#testServeEndToEnd) that drives the live server over HTTP against a
two-commit git repo with real `bazel query`. New code is ~97% line-covered.

Docs: README "Query Service (experimental)" section, with the `serve` help
wired into the generated CLI reference.

Deferred to follow-ups: remote (S3) cache backend, OCI image, k8s manifests,
load balancing / multi-instance, dynamic scaling, git auth config.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat: add in-process JGit engine for `serve` (default)

Adds a JGit-backed GitClient so the query service performs git fetch /
resolve / checkout in-process, without forking a `git` subprocess or
requiring a `git` binary on the PATH.

- JGitClient implements the existing GitClient interface (drop-in).
- `--gitEngine` selects the backend: `jgit` (default, in-process) or
  `subprocess` (shells out to --gitPath, exact C-git behavior, useful for
  workspaces relying on checkout filters/hooks JGit doesn't run).
- Pins org.eclipse.jgit 5.13.x (the last Java-8-compatible line, matching
  bazel-diff's documented minimum) and repins maven_install.json.

Note: "in process" removes the subprocess only; the working tree is still
materialized on disk because `bazel query` reads real files. JGit's
InMemoryRepository can't back a bazel workspace.

Tests: JGitClientTest (resolve/checkout/fetch incl. remote loop and error
paths against a real repo) + engine-selection tests; the serve E2E now
exercises JGit by default. New code remains ~97% line-covered.

Docs: README + generated serve help document --gitEngine and the on-disk
working-tree caveat.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

Author: tinder-maxwellelliott

MODULE.bazel

@@ -50,6 +50,9 @@ maven.install(
     artifacts = [
         "com.google.code.gson:gson:2.9.0@jar",
         "com.google.guava:guava:31.1-jre",
+        # JGit powers the `serve` command's in-process git engine. Pinned to the 5.13.x line, the
+        # last release series that still targets Java 8 (matching bazel-diff's documented minimum).
+        "org.eclipse.jgit:org.eclipse.jgit:5.13.3.202401111512-r",
         "com.willowtreeapps.assertk:assertk-jvm:0.25",
         "info.picocli:picocli:4.6.3@jar",
         "io.insert-koin:koin-core-jvm:3.1.6",

README.md

@@ -107,6 +107,66 @@ This will produce an impacted targets json list with target label, target distan
 ]
 ```
 
+## Query Service (experimental)
+
+Instead of running `generate-hashes` from scratch on every CI invocation, you can run `bazel-diff` as
+a long-running HTTP service that answers affectedness queries between two git revisions and caches
+the generated hashes per commit SHA. This is the "bazel-diff as a service" model described in the
+[BazelCon talk](https://youtu.be/9Dk7mtIm7_A?t=1875) that inspired this repo (see
+[issue #29](https://github.com/Tinder/bazel-diff/issues/29)).
+
+Start the service against a dedicated git clone of your workspace:
+
+```bash
+bazel-diff serve \
+  --workspacePath /path/to/workspace-clone \
+  --bazelPath bazel \
+  --cacheDir /var/cache/bazel-diff \
+  --port 8080
+```
+
+On startup the service performs an initial `git fetch` and only then reports healthy. For each
+request it resolves the `from`/`to` revisions, generates (and caches, keyed by commit SHA) the hashes
+for each, and reuses the exact same affectedness logic as `get-impacted-targets`.
+
+Endpoints:
+
+* `GET /health` — returns `200 OK` once the initial fetch has completed, `503` otherwise. A load
+  balancer should use this to route only to ready instances. If a fatal git error occurs at startup
+  the instance "lame-ducks" itself by continuing to report `503` so the load balancer removes it.
+* `GET /impacted_targets?from=<rev>&to=<rev>` — returns the impacted targets as JSON. The optional
+  `targetType` parameter (e.g. `&targetType=Rule,SourceFile`) filters by target type.
+
+```bash
+curl 'http://localhost:8080/impacted_targets?from=main&to=my-feature-branch'
+```
+
+```json
+{
+  "from": "9a1c0e2…",
+  "to": "3f7b8d4…",
+  "impactedTargets": ["//foo:bar", "//foo:baz"]
+}
+```
+
+Notes and current limitations:
+
+* The service checks out revisions inside `--workspacePath`, so point it at a dedicated clone, not a
+  working tree you edit. All workspace-mutating work (git checkout + `bazel query`) is serialized,
+  so a single instance answers one cold query at a time; the per-SHA cache absorbs the rest.
+* Git operations run in-process via JGit by default (no `git` binary required). Pass
+  `--gitEngine=subprocess` to shell out to the `git` binary at `--gitPath` instead -- useful for
+  workspaces that depend on checkout filters or hooks that JGit does not run. Note that JGit only
+  moves the git plumbing in-process; the working tree is still materialized on disk for `bazel query`.
+* Hashes are cached on local disk via `--cacheDir` and survive restarts. The cache layer is
+  pluggable behind a byte-oriented interface so a remote backend (e.g. S3) can be added without
+  touching callers.
+* Query-affecting flags (`--useCquery`, `--fineGrainedHashExternalRepos`, etc.) mirror
+  `generate-hashes`, and are folded into the cache key so a server started with different flags never
+  serves another configuration's cached hashes.
+* Containerization, multi-instance deployment manifests, and remote cache backends are not yet
+  included.
+
 <!-- BEGIN_SECTION: cli-help -->
 ## CLI Interface
 
@@ -133,6 +193,9 @@ Commands:
                           lock, .bazelrc, bazel-diff version, flag set) and
                           writes it as JSON. Used to decide whether a
                           Firecracker snapshot is safe to consume.
+  serve                 Runs bazel-diff as a long-running HTTP query service
+                          that returns the impacted targets between two git
+                          revisions, caching generated hashes per commit SHA.
 ```
 
 ### `generate-hashes` command
@@ -326,6 +389,81 @@ Command-line utility to analyze the state of the bazel build graph
                          Path to Bazel workspace directory. Required for module
                            change detection.
 ```
+
+### `serve` command
+
+```terminal
+Usage: bazel-diff serve [-hkvV] [--[no-]excludeExternalTargets]
+                        [--no-initial-fetch] [--[no-]useCquery]
+                        [-b=<bazelPath>] --cacheDir=<cacheDir>
+                        [--cqueryExpression=<cqueryExpression>]
+                        [--fineGrainedHashExternalReposFile=<fineGrainedHashExte
+                        rnalReposFile>] [--gitEngine=<gitEngine>]
+                        [--gitPath=<gitPath>] [--port=<port>]
+                        [-s=<seedFilepaths>] -w=<workspacePath>
+                        [-co=<bazelCommandOptions>]...
+                        [--cqueryCommandOptions=<cqueryCommandOptions>]...
+                        [--fineGrainedHashExternalRepos=<fineGrainedHashExternal
+                        Repos>]...
+                        [--ignoredRuleHashingAttributes=<ignoredRuleHashingAttri
+                        butes>]... [-so=<bazelStartupOptions>]...
+Runs bazel-diff as a long-running HTTP query service that returns the impacted
+targets between two git revisions, caching generated hashes per commit SHA.
+  -b, --bazelPath=<bazelPath>
+                            Path to Bazel binary. If not specified, the Bazel
+                              binary available in PATH will be used.
+      --cacheDir=<cacheDir> Directory where generated hashes are cached per
+                              commit SHA. Persists across restarts.
+      -co, --bazelCommandOptions=<bazelCommandOptions>
+                            Additional space separated Bazel command options
+                              used when invoking `bazel query`
+      --cqueryCommandOptions=<cqueryCommandOptions>
+                            Additional space separated Bazel command options
+                              used when invoking `bazel cquery`. No effect
+                              unless --useCquery is set.
+      --cqueryExpression=<cqueryExpression>
+                            Custom cquery expression to use instead of the
+                              default. No effect unless --useCquery.
+      --[no-]excludeExternalTargets
+                            If true, exclude external targets (do not query
+                              //external:all-targets).
+      --fineGrainedHashExternalRepos=<fineGrainedHashExternalRepos>
+                            Comma separated list of external repos for which
+                              fine-grained hashes are computed.
+      --fineGrainedHashExternalReposFile=<fineGrainedHashExternalReposFile>
+                            A text file with a newline separated list of
+                              external repos. Mutually exclusive with
+                              --fineGrainedHashExternalRepos.
+      --gitEngine=<gitEngine>
+                            Git backend: 'jgit' (in-process, no git binary
+                              required) or 'subprocess' (shells out to
+                              --gitPath). Defaults to 'jgit'.
+      --gitPath=<gitPath>   Path to the git binary, used only when
+                              --gitEngine=subprocess. Defaults to 'git' on the
+                              PATH.
+  -h, --help                Show this help message and exit.
+      --ignoredRuleHashingAttributes=<ignoredRuleHashingAttributes>
+                            Attributes that should be ignored when hashing rule
+                              targets.
+  -k, --[no-]keep_going     Run `bazel query` with --keep_going. Defaults to
+                              true.
+      --no-initial-fetch    Skip the initial 'git fetch' before reporting
+                              healthy. Useful for local/offline testing.
+      --port=<port>         Port to listen on. Defaults to 8080.
+  -s, --seed-filepaths=<seedFilepaths>
+                            A text file with a newline separated list of
+                              filepaths used as a SHA256 seed for all targets.
+      -so, --bazelStartupOptions=<bazelStartupOptions>
+                            Additional space separated Bazel client startup
+                              options used when invoking Bazel
+      --[no-]useCquery      If true, use cquery instead of query when
+                              generating dependency graphs.
+  -v, --verbose             Display query string, missing files and elapsed time
+  -V, --version             Print version information and exit.
+  -w, --workspacePath=<workspacePath>
+                            Path to the Bazel workspace git clone the service
+                              checks out and queries.
+```
 <!-- END_SECTION: cli-help -->
 
 ### What does the SHA256 value of `generate-hashes` represent?

cli/BUILD

@@ -38,6 +38,7 @@ kt_jvm_library(
         "@bazel_diff_maven//:info_picocli_picocli",
         "@bazel_diff_maven//:io_insert_koin_koin_core_jvm",
         "@bazel_diff_maven//:org_apache_commons_commons_pool2",
+        "@bazel_diff_maven//:org_eclipse_jgit_org_eclipse_jgit",
         "@bazel_diff_maven//:org_jetbrains_kotlinx_kotlinx_coroutines_core_jvm",
     ],
 )
@@ -212,6 +213,54 @@ kt_jvm_test(
     ],
 )
 
+kt_jvm_test(
+    name = "ServeCommandTest",
+    test_class = "com.bazel_diff.cli.ServeCommandTest",
+    runtime_deps = [":cli-test-lib"],
+)
+
+kt_jvm_test(
+    name = "LocalDiskHashCacheStorageTest",
+    test_class = "com.bazel_diff.server.LocalDiskHashCacheStorageTest",
+    runtime_deps = [":cli-test-lib"],
+)
+
+kt_jvm_test(
+    name = "GitClientTest",
+    test_class = "com.bazel_diff.server.GitClientTest",
+    runtime_deps = [":cli-test-lib"],
+)
+
+kt_jvm_test(
+    name = "JGitClientTest",
+    test_class = "com.bazel_diff.server.JGitClientTest",
+    runtime_deps = [":cli-test-lib"],
+)
+
+kt_jvm_test(
+    name = "HashServiceTest",
+    jvm_flags = [
+        "-Dnet.bytebuddy.experimental=true",
+    ],
+    test_class = "com.bazel_diff.server.HashServiceTest",
+    runtime_deps = [":cli-test-lib"],
+)
+
+kt_jvm_test(
+    name = "ImpactedTargetsServiceTest",
+    jvm_flags = [
+        "-Dnet.bytebuddy.experimental=true",
+    ],
+    test_class = "com.bazel_diff.server.ImpactedTargetsServiceTest",
+    runtime_deps = [":cli-test-lib"],
+)
+
+kt_jvm_test(
+    name = "BazelDiffServerTest",
+    test_class = "com.bazel_diff.server.BazelDiffServerTest",
+    runtime_deps = [":cli-test-lib"],
+)
+
 kt_jvm_library(
     name = "cli-test-lib",
     testonly = True,

cli/src/main/kotlin/com/bazel_diff/cli/BazelDiff.kt

@@ -12,7 +12,8 @@ import picocli.CommandLine.Spec
             GenerateHashesCommand::class,
             GetImpactedTargetsCommand::class,
             WarmupCommand::class,
-            FingerprintCommand::class],
+            FingerprintCommand::class,
+            ServeCommand::class],
     mixinStandardHelpOptions = true,
     versionProvider = VersionProvider::class,
 )