Foil hole acquisition suggestions for a given grid square endpoint required

[d-j-hatton]

There already exists an endpoint to provide a suggested list of grid squares for acquisition using a quality assessment model and a latent space representation. The same functionality is required at the grid square level, suggesting foil holes rather than grid squares

[vredchenko]

Implemented and released. Closing with evidence.

Endpoint

GET /gridsquares/{gridsquare_uuid}/prediction_model/{prediction_model_name}/latent_rep/{latent_rep_model_name}/suggested_holes → list[FoilHole]

Added in #296 (get_suggested_hole_collections, api_server.py). It mirrors the grid-square suggested_squares endpoint but at the foil-hole level: rank holes by their quality-prediction score, take the top half, and greedily select while capping each latent-space cluster. The per-cluster cap is 4 for holes (vs 2 for squares), since a single grid square is unlikely to contain every cluster found across the whole grid.

Live demonstration on real data

Ran against the dev backend (v0.1.1) over the prod-dump dataset, using prediction model resnet-holes and latent-rep model dae-hole.

Grid square l1A0CbciSIKulw0tvYQYcQ — 30 holes with a score → top 15 considered, 5 distinct dae-hole clusters:

  #  foilhole_uuid              score  cluster  pick
  0  JHjpnpURRaC721Kf7MVydQ   0.98541        5  SELECT
  1  0xZv-HqtRpu-RRp2s2nhaw   0.95045        8  SELECT
  2  tHj4pNOwRvKss3qZRLWU7g   0.93891        8  SELECT
  3  gAptuzwLT7OpyQ5B4ekqhQ   0.92278        8  SELECT
  4  BjpImbykSe2qcevsZxPfEw   0.92225        5  SELECT
  5  lZFsx_bHRMqItwAUfSGljw   0.92191        8  SELECT
  6  zkfhD1LRRQ2SlmlZ2CPXxg   0.91464        8          <- cluster 8 already at cap 4
  7  AzqCPtLkS8uZW1bLfnSkaQ   0.90546        5  SELECT
  8  Wgg_rC2gSx63ig2YS21gvw   0.90456        5  SELECT
  9  ZyMYoicLSJmZk5ITGBCErg   0.89863        6  SELECT
 10  mgXYfurYSZKPwM78SEFaJA   0.89759        8          <- cluster 8 at cap
 11  t9vrgBSNQHazqQSBLSYgRQ   0.89403        0  SELECT
 12  Kd8hALIyQbWujcXyrf7VXA   0.89374        5          <- cluster 5 already at cap 4
 13  9v8x8PBTRqGnBYTxLegIbA   0.88623        0  SELECT
 14  91a1lBkCRrS7aPy4pm74rA   0.88411        5          <- cluster 5 at cap

Endpoint returned HTTP 200 with 11 foil holes. An independent reproduction of the selection algorithm straight from the database returned the same 11 holes in the same order (SETS EQUAL: True, ORDER EQUAL: True), with the per-cluster cap ≤ 4 and every returned hole inside the top half — confirming both cluster 8 and cluster 5 are correctly capped at 4.

Unit coverage: tests/smartem_backend/test_foilholes.py::TestSuggestedHoleCollections (holes missing a cluster index are skipped) passes.

Consumers

• Frontend — uses it. The square view calls suggested_holes (useSquareMapData.ts) and renders a "Suggestions (N)" overlay in SquareMap.tsx. The generated Orval client method and MSW mock are already published in @smartem/api.
• Agent — does not use it (it only performs ingest/write calls; no suggestion reads).
• RabbitMQ worker — produces the inputs, does not call the endpoint. handle_foilhole_model_prediction writes the CurrentQualityPrediction scores and handle_model_parameter_update writes the dae-hole cluster_indices; the suggestion is then computed on demand inside the API endpoint.

Release

• Shipped in stable smartem-decisions-v0.1.1 (published 2026-06-29): PyPI wheel/sdist, GHCR container, GitHub Release, plus the version-stamped openapi-0.1.1.json.
• OpenAPI spec regenerated (docs/api/openapi.json) and the docs site is live — the endpoint is served in the Swagger UI at https://diamondlightsource.github.io/smartem-devtools.

Fully addresses the request; closing.