Skip to content

Commit 236a7f1

Browse files
feat: compute target distances in bazel-diff serve (#397)
Add a `/impacted_targets_with_distances` endpoint to the serve query service that returns each impacted target annotated with its build-graph distance metrics (targetDistance and packageDistance), reusing the same distance math as `get-impacted-targets --depEdgesFile`. Dependency-edge tracking is opt-in via a new `serve --trackDeps` flag. When enabled, the per-SHA hash cache persists the dep-edge adjacency list under `metadata.depEdges` and round-trips it on a cache hit. The flag is folded into the cache fingerprint so a deps-tracking server never reuses a deps-less cache entry. Hitting the distances endpoint without `--trackDeps` returns a clear 400. The distance computation in CalculateImpactedTargetsInteractor is extracted into a structured `computeImpactedTargetsWithDistances` reused by both the CLI writer path and the service, keeping CLI JSON output byte-identical. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent e55c3e9 commit 236a7f1

15 files changed

Lines changed: 557 additions & 52 deletions

README.md

Lines changed: 34 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -149,8 +149,36 @@ curl 'http://localhost:8080/impacted_targets?from=main&to=my-feature-branch'
149149
}
150150
```
151151

152+
* `GET /impacted_targets_with_distances?from=<rev>&to=<rev>` — like `/impacted_targets`, but each
153+
impacted target is annotated with its build-graph distance metrics: `targetDistance` (the number of
154+
dependency hops to the nearest directly-changed target) and `packageDistance` (how many of those
155+
hops cross a package boundary). Directly-changed targets sit at distance `0`. Requires the server
156+
to have been started with `--trackDeps` (see below); otherwise this endpoint returns `400`. The
157+
same optional `targetType` filter applies.
158+
159+
```bash
160+
curl 'http://localhost:8080/impacted_targets_with_distances?from=main&to=my-feature-branch'
161+
```
162+
163+
```json
164+
{
165+
"from": "9a1c0e2…",
166+
"to": "3f7b8d4…",
167+
"impactedTargets": [
168+
{"label": "//foo:bar", "targetDistance": 0, "packageDistance": 0},
169+
{"label": "//foo:baz", "targetDistance": 1, "packageDistance": 1}
170+
]
171+
}
172+
```
173+
152174
Notes and current limitations:
153175

176+
* Distance metrics (`/impacted_targets_with_distances`) require the dependency-edge graph, which is
177+
only tracked when the server is started with `--trackDeps`. Tracking deps grows each cached hash
178+
entry, so it is opt-in. The flag is folded into the cache key, so enabling or disabling it never
179+
reuses a previously cached entry of the other kind. This mirrors the `generate-hashes --depEdgesFile`
180+
/ `get-impacted-targets --depEdgesFile` flow used by the CLI.
181+
154182
* The service checks out revisions inside `--workspacePath`, so point it at a dedicated clone, not a
155183
working tree you edit. All workspace-mutating work (git checkout + `bazel query`) is serialized,
156184
so a single instance answers one cold query at a time; the per-SHA cache absorbs the rest.
@@ -394,8 +422,8 @@ Command-line utility to analyze the state of the bazel build graph
394422

395423
```terminal
396424
Usage: bazel-diff serve [-hkvV] [--[no-]excludeExternalTargets]
397-
[--no-initial-fetch] [--[no-]useCquery]
398-
[-b=<bazelPath>] --cacheDir=<cacheDir>
425+
[--no-initial-fetch] [--[no-]trackDeps] [--[no-]
426+
useCquery] [-b=<bazelPath>] --cacheDir=<cacheDir>
399427
[--cqueryExpression=<cqueryExpression>]
400428
[--fineGrainedHashExternalReposFile=<fineGrainedHashExte
401429
rnalReposFile>] [--gitEngine=<gitEngine>]
@@ -456,6 +484,10 @@ targets between two git revisions, caching generated hashes per commit SHA.
456484
-so, --bazelStartupOptions=<bazelStartupOptions>
457485
Additional space separated Bazel client startup
458486
options used when invoking Bazel
487+
--[no-]trackDeps Track dependency edges and persist them per commit
488+
SHA so build-graph distance metrics can be served
489+
via /impacted_targets_with_distances. Increases
490+
cache size and memory. Defaults to false.
459491
--[no-]useCquery If true, use cquery instead of query when
460492
generating dependency graphs.
461493
-v, --verbose Display query string, missing files and elapsed time

cli/src/main/kotlin/com/bazel_diff/cli/ServeCommand.kt

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -177,6 +177,16 @@ class ServeCommand : Callable<Int> {
177177
description = ["If true, exclude external targets (do not query //external:all-targets)."])
178178
var excludeExternalTargets = false
179179

180+
@CommandLine.Option(
181+
names = ["--trackDeps"],
182+
negatable = true,
183+
description =
184+
[
185+
"Track dependency edges and persist them per commit SHA so build-graph distance " +
186+
"metrics can be served via /impacted_targets_with_distances. Increases cache " +
187+
"size and memory. Defaults to false."])
188+
var trackDeps = false
189+
180190
override fun call(): Int {
181191
org.koin.core.context.GlobalContext.stopKoin()
182192
startKoin {
@@ -191,7 +201,7 @@ class ServeCommand : Callable<Int> {
191201
useCquery,
192202
cqueryExpression,
193203
keepGoing,
194-
false,
204+
trackDeps,
195205
fineGrainedHashExternalRepos,
196206
fineGrainedHashExternalReposFile,
197207
excludeExternalTargets,
@@ -238,8 +248,10 @@ class ServeCommand : Callable<Int> {
238248
storage,
239249
computeConfigFingerprint(),
240250
loadSeedFilepaths(),
241-
ignoredRuleHashingAttributes)
242-
val impactedTargetsService = ImpactedTargetsService(gitClient, hashService)
251+
ignoredRuleHashingAttributes,
252+
trackDeps)
253+
val impactedTargetsService =
254+
ImpactedTargetsService(gitClient, hashService, depsTracked = trackDeps)
243255

244256
val ready = AtomicBoolean(false)
245257
val server = BazelDiffServer(port, impactedTargetsService) { ready.get() }
@@ -316,6 +328,9 @@ class ServeCommand : Callable<Int> {
316328
append("ignoredRuleHashingAttributes=")
317329
.append(ignoredRuleHashingAttributes.sorted().joinToString(","))
318330
.append('\n')
331+
// Distinct cache keys for deps-tracked vs deps-less runs: a deps-tracking server must never
332+
// serve a deps-less cache entry (its distance queries would have no edges to traverse).
333+
append("trackDeps=").append(trackDeps).append('\n')
319334
append("version=").append(VersionProvider().version.firstOrNull() ?: "unknown").append('\n')
320335
}
321336
return sha256 { putBytes(canonical.toByteArray()) }.toHexString().take(12)

cli/src/main/kotlin/com/bazel_diff/interactor/CalculateImpactedTargetsInteractor.kt

Lines changed: 37 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,13 @@ import org.koin.core.component.inject
1717

1818
data class TargetDistanceMetrics(val targetDistance: Int, val packageDistance: Int) {}
1919

20+
/** An impacted target paired with its build-graph distance metrics. */
21+
data class ImpactedTargetWithDistance(
22+
val label: String,
23+
val targetDistance: Int,
24+
val packageDistance: Int
25+
)
26+
2027
class CalculateImpactedTargetsInteractor : KoinComponent {
2128
private val gson: Gson by inject()
2229
private val logger: Logger by inject()
@@ -94,6 +101,33 @@ class CalculateImpactedTargetsInteractor : KoinComponent {
94101
toModuleGraphJson: String? = null,
95102
excludeExternalTargets: Boolean = false,
96103
) {
104+
val impactedTargets =
105+
computeImpactedTargetsWithDistances(
106+
from,
107+
to,
108+
depEdges,
109+
targetTypes,
110+
fromModuleGraphJson,
111+
toModuleGraphJson,
112+
excludeExternalTargets)
113+
outputWriter.use { writer -> writer.write(gson.toJson(impactedTargets)) }
114+
}
115+
116+
/**
117+
* Computes the impacted targets between [from] and [to] together with their build-graph distance
118+
* metrics, filtered by [targetTypes]/[excludeExternalTargets] and ordered the same way the
119+
* non-distance path orders its output. Returned in-memory so both the CLI writer path
120+
* ([executeWithDistances]) and the query service consume identical data without reparsing JSON.
121+
*/
122+
fun computeImpactedTargetsWithDistances(
123+
from: Map<String, TargetHash>,
124+
to: Map<String, TargetHash>,
125+
depEdges: Map<String, List<String>>,
126+
targetTypes: Set<String>?,
127+
fromModuleGraphJson: String? = null,
128+
toModuleGraphJson: String? = null,
129+
excludeExternalTargets: Boolean = false,
130+
): List<ImpactedTargetWithDistance> {
97131
val typeFilter = TargetTypeFilter(targetTypes, to)
98132

99133
// Quick check: if module graph JSON is identical, skip module change detection entirely
@@ -121,21 +155,12 @@ class CalculateImpactedTargetsInteractor : KoinComponent {
121155
}
122156

123157
val ordering = impactedTargetOrdering(to, from)
124-
impactedTargets
158+
return impactedTargets
125159
.filterKeys { typeFilter.accepts(it) }
126160
.filterKeys { !excludeExternalTargets || !it.startsWith("//external:") }
127161
.toSortedMap(ordering)
128-
.let { filtered ->
129-
outputWriter.use { writer ->
130-
writer.write(
131-
gson.toJson(
132-
filtered.map {
133-
mapOf(
134-
"label" to it.key,
135-
"targetDistance" to it.value.targetDistance,
136-
"packageDistance" to it.value.packageDistance)
137-
}))
138-
}
162+
.map {
163+
ImpactedTargetWithDistance(it.key, it.value.targetDistance, it.value.packageDistance)
139164
}
140165
}
141166

cli/src/main/kotlin/com/bazel_diff/interactor/DeserialiseHashesInteractor.kt

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,11 @@ import java.io.FileReader
99
import org.koin.core.component.KoinComponent
1010
import org.koin.core.component.inject
1111

12-
data class HashFileData(val hashes: Map<String, TargetHash>, val moduleGraphJson: String?)
12+
data class HashFileData(
13+
val hashes: Map<String, TargetHash>,
14+
val moduleGraphJson: String?,
15+
val depEdges: Map<String, List<String>> = emptyMap()
16+
)
1317

1418
class DeserialiseHashesInteractor : KoinComponent {
1519
private val gson: Gson by inject()
@@ -40,9 +44,19 @@ class DeserialiseHashesInteractor : KoinComponent {
4044
val hashesMap: Map<String, String> = gson.fromJson(jsonObject.get("hashes"), hashesShape)
4145
val hashes = hashesMap.mapValues { TargetHash.fromJson(it.value) }
4246

43-
val moduleGraphJson = jsonObject.getAsJsonObject("metadata")?.get("moduleGraphJson")?.asString
47+
val metadata = jsonObject.getAsJsonObject("metadata")
48+
val moduleGraphJson = metadata?.get("moduleGraphJson")?.asString
4449

45-
return HashFileData(hashes, moduleGraphJson)
50+
// The query service persists the dependency-edge adjacency list (label -> direct dep labels)
51+
// under metadata.depEdges when started with --trackDeps, so build-graph distance metrics can
52+
// be computed on a cache hit without re-tracking deps. Absent for CLI-produced hashes.
53+
val depEdges: Map<String, List<String>> =
54+
metadata?.get("depEdges")?.let {
55+
val depShape = object : TypeToken<Map<String, List<String>>>() {}.type
56+
gson.fromJson<Map<String, List<String>>>(it, depShape)
57+
} ?: emptyMap()
58+
59+
return HashFileData(hashes, moduleGraphJson, depEdges)
4660
} else {
4761
// Legacy format - just a flat map of hashes
4862
val shape = object : TypeToken<Map<String, String>>() {}.type

cli/src/main/kotlin/com/bazel_diff/server/BazelDiffServer.kt

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,9 @@ import org.koin.core.component.inject
2121
* lame-ducked by a fatal git error.
2222
* - `GET /impacted_targets?from=<rev>&to=<rev>[&targetType=Rule,SourceFile]` -- returns `{"from":
2323
* <sha>, "to": <sha>, "impactedTargets": [...]}`.
24+
* - `GET /impacted_targets_with_distances?from=<rev>&to=<rev>[&targetType=...]` -- like above but
25+
* each impacted target is `{"label", "targetDistance", "packageDistance"}`. Requires the server
26+
* to have been started with `--trackDeps`; returns `400` otherwise.
2427
*
2528
* Built on the JDK's [HttpServer] so the service needs no new third-party dependency. The handler
2629
* pool is unbounded (cached) so that health checks are always served even while a long hash
@@ -47,6 +50,8 @@ class BazelDiffServer(
4750
}
4851
httpServer.createContext("/health", ::handleHealth)
4952
httpServer.createContext("/impacted_targets", ::handleImpactedTargets)
53+
httpServer.createContext(
54+
"/impacted_targets_with_distances", ::handleImpactedTargetsWithDistances)
5055
httpServer.start()
5156
server = httpServer
5257
logger.i { "bazel-diff query service listening on port ${boundPort()} " }
@@ -80,6 +85,24 @@ class BazelDiffServer(
8085
}
8186

8287
private fun handleImpactedTargets(exchange: HttpExchange) =
88+
handleQuery(exchange) { from, to, targetTypes ->
89+
impactedTargetsProvider.getImpactedTargets(from, to, targetTypes)
90+
}
91+
92+
private fun handleImpactedTargetsWithDistances(exchange: HttpExchange) =
93+
handleQuery(exchange) { from, to, targetTypes ->
94+
impactedTargetsProvider.getImpactedTargetsWithDistances(from, to, targetTypes)
95+
}
96+
97+
/**
98+
* Shared handling for the impacted-targets endpoints: enforces GET + readiness, parses and
99+
* validates `from`/`to`/`targetType`, then serializes the result of [compute] as JSON, mapping
100+
* the known failure modes to the appropriate status codes.
101+
*/
102+
private fun handleQuery(
103+
exchange: HttpExchange,
104+
compute: (from: String, to: String, targetTypes: Set<String>?) -> Any
105+
) =
83106
withExchange(exchange) {
84107
if (!exchange.requestMethod.equals("GET", ignoreCase = true)) {
85108
respondJson(exchange, 405, mapOf("error" to "method not allowed, use GET"))
@@ -102,8 +125,9 @@ class BazelDiffServer(
102125
params["targetType"]?.split(",")?.map { it.trim() }?.filter { it.isNotEmpty() }?.toSet()
103126

104127
try {
105-
val result = impactedTargetsProvider.getImpactedTargets(from, to, targetTypes)
106-
respondJson(exchange, 200, result)
128+
respondJson(exchange, 200, compute(from, to, targetTypes))
129+
} catch (e: DistancesUnavailableException) {
130+
respondJson(exchange, 400, mapOf("error" to (e.message ?: "distances unavailable")))
107131
} catch (e: GitClientException) {
108132
logger.e(e) { "git error computing impacted targets" }
109133
respondJson(exchange, 400, mapOf("error" to "git error: ${e.message}"))

cli/src/main/kotlin/com/bazel_diff/server/HashService.kt

Lines changed: 31 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -40,13 +40,17 @@ interface HashProvider {
4040
* server started with different flags never reads another configuration's cached hashes.
4141
* @param seedFilepaths seed files passed through to [BuildGraphHasher].
4242
* @param ignoredRuleHashingAttributes rule attributes ignored when hashing.
43+
* @param trackDeps when true, persist each revision's dependency-edge adjacency list in the cache
44+
* so build-graph distance metrics can be served. Requires the hasher to have been built with
45+
* dep-tracking on (so [TargetHash.deps] is populated).
4346
*/
4447
class HashService(
4548
private val gitClient: GitClient,
4649
private val storage: HashCacheStorage,
4750
private val configFingerprint: String,
4851
private val seedFilepaths: Set<Path>,
4952
private val ignoredRuleHashingAttributes: Set<String>,
53+
private val trackDeps: Boolean = false,
5054
) : HashProvider, KoinComponent {
5155
private val buildGraphHasher: BuildGraphHasher by inject()
5256
private val bazelModService: BazelModService by inject()
@@ -89,24 +93,42 @@ class HashService(
8993
buildGraphHasher.hashAllBazelTargetsAndSourcefiles(
9094
seedFilepaths, ignoredRuleHashingAttributes)
9195
val moduleGraphJson = runBlocking { bazelModService.getModuleGraphJson() }
96+
val depEdges = depEdgesOf(hashes)
9297
storage.put(
93-
cacheKey(sha), serialize(hashes, moduleGraphJson).toByteArray(StandardCharsets.UTF_8))
94-
HashFileData(hashes, moduleGraphJson)
98+
cacheKey(sha),
99+
serialize(hashes, moduleGraphJson, depEdges).toByteArray(StandardCharsets.UTF_8))
100+
HashFileData(hashes, moduleGraphJson, depEdges)
95101
}
96102

103+
/**
104+
* The dependency-edge adjacency list (label -> direct dep labels) when [trackDeps] is on, else
105+
* empty. Derived from [TargetHash.deps], the same way `generate-hashes --depEdgesFile` derives
106+
* it.
107+
*/
108+
private fun depEdgesOf(hashes: Map<String, TargetHash>): Map<String, List<String>> =
109+
if (trackDeps) hashes.mapValues { it.value.deps ?: emptyList() } else emptyMap()
110+
97111
/**
98112
* Serializes hashes into the same JSON shape `generate-hashes` writes (see
99113
* [com.bazel_diff.interactor.GenerateHashesInteractor]), so cached entries are interchangeable
100-
* with hashes produced by the CLI. Target type is always included for the richest data.
114+
* with hashes produced by the CLI. Target type is always included for the richest data. When
115+
* [depEdges] is non-empty (i.e. --trackDeps), it is persisted under `metadata.depEdges` so
116+
* distance metrics can be served on a cache hit.
101117
*/
102-
private fun serialize(hashes: Map<String, TargetHash>, moduleGraphJson: String?): String {
118+
private fun serialize(
119+
hashes: Map<String, TargetHash>,
120+
moduleGraphJson: String?,
121+
depEdges: Map<String, List<String>>
122+
): String {
123+
val serializedHashes = hashes.mapValues { it.value.toJson(true) }
103124
val output =
104-
if (moduleGraphJson != null) {
105-
mapOf(
106-
"hashes" to hashes.mapValues { it.value.toJson(true) },
107-
"metadata" to mapOf("moduleGraphJson" to moduleGraphJson))
125+
if (moduleGraphJson != null || depEdges.isNotEmpty()) {
126+
val metadata = mutableMapOf<String, Any>()
127+
if (moduleGraphJson != null) metadata["moduleGraphJson"] = moduleGraphJson
128+
if (depEdges.isNotEmpty()) metadata["depEdges"] = depEdges
129+
mapOf("hashes" to serializedHashes, "metadata" to metadata)
108130
} else {
109-
hashes.mapValues { it.value.toJson(true) }
131+
serializedHashes
110132
}
111133
return gson.toJson(output)
112134
}

0 commit comments

Comments
 (0)