Skip to content

follow-up: decide whether HubTargets.mid should drive a benchmark measurement or be documented as informational-only #2089

Description

@carlos-alm

Deferred from PR #2039 review.

Original reviewer comment: inline comment #2039 (comment) (scripts/lib/hub-selection.ts:110)

Context:

selectHubTargets() in scripts/lib/hub-selection.ts computes and returns mid (the node at the median edge-count rank among kind-filtered candidates) as part of HubTargets. Neither query-benchmark.ts nor benchmark.ts ever passes targets.mid into a benchDepths/benchQuery call — it only appears in a console.error log line in query-benchmark.ts and in the targets block of the output JSON. It drives zero actual timed measurement in either script.

This is pre-existing behavior, not something PR #2039 introduced — the original (pre-#2039) selectTargets() in query-benchmark.ts had the exact same shape (mid computed, logged, never benchmarked). #2039's scope was fixing the kind-filter/non-determinism bug (#1904), so it preserved this behavior unchanged rather than expanding the benchmark's measurement surface.

Two ways to resolve, either is reasonable:

  1. Wire it in: add benchDepths(fnDepsData, targets.mid, ...) (and/or fnImpactData) calls in both scripts so mid drives an actual medium-connectivity measurement point (parallel to how hub/leaf are used today), extending the output JSON schema accordingly. Would need to consider whether the "Pre-publish benchmark gate" / "Perf canary" threshold configs need a new baseline entry for it.
  2. Formalize as informational-only: keep computing mid (useful for visibility/debugging) but make the HubTargets.mid doc comment explicit that it is not currently wired into any timed benchmark, so future readers don't mistake the interface/tests for evidence of an active measurement.

PR #2039 took approach 2 as a documentation-only change (added a doc comment on the mid field) since actually adding a new benchmark dimension is a distinct feature change with its own design questions, out of scope for a determinism/kind-filter bug fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    follow-upDeferred work from PR reviews that needs tracking

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions