Skip to content

Automatic Temporal Extent Calculation for Collections#999

Merged
matthewhanson merged 13 commits into
mainfrom
mah/auto-temporal-extent
Jun 20, 2026
Merged

Automatic Temporal Extent Calculation for Collections#999
matthewhanson merged 13 commits into
mainfrom
mah/auto-temporal-extent

Conversation

@matthewhanson

@matthewhanson matthewhanson commented Oct 30, 2025

Copy link
Copy Markdown
Member

Adds automatic temporal extent calculation for STAC collections based on the
actual datetime range of items in the collection.

Changes

• Automatic calculation: Collections without a predefined temporal extent now have it calculated
dynamically from their items when served via the API
• Opt-in behavior: Only collections without extent.temporal.interval defined will have it auto-calculated
• Performant implementation: Uses efficient sorting queries (ascending/descending) rather than aggregations
• Empty collection handling: Collections with no items return [[null, null]]
• Endpoints affected: /collections and /collections/{collectionId}

Implementation

• Added getTemporalExtentFromItems() function in src/lib/database.js that uses two parallel sort queries to find earliest and latest item datetimes
• Added populateTemporalExtentIfMissing() helper in src/lib/api.js to check and populate temporal extent
• Updated both collection endpoints to call the helper function

Configuration

When ingesting collections, omit extent.temporal.interval to enable automatic calculation:

{
  "id": "my-collection",
  "extent": {
    "spatial": {...}
    // No temporal extent defined - will be calculated automatically
  }
}

Closes #1050

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds automatic temporal extent calculation for STAC collections based on item datetime ranges. Collections without predefined temporal extents will have them calculated dynamically when served via the API, using efficient sort queries to find the earliest and latest item datetimes.

Key changes:

  • Added getTemporalExtentFromItems() function that uses parallel ascending/descending sort queries to efficiently find temporal boundaries
  • Collections without extent.temporal.interval now have it auto-calculated from their items
  • Empty collections return [[null, null]] as their temporal extent

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/lib/database.js Added getTemporalExtentFromItems() function to query earliest and latest item datetimes using parallel sort operations
src/lib/api.js Added populateTemporalExtentIfMissing() helper and integrated it into collection endpoints
tests/system/test-api-temporal-extent.js Added comprehensive test suite covering temporal extent calculation for collections with items and empty collections
README.md Added documentation for the automatic temporal extent feature
CHANGELOG.md Added entry documenting the new automatic temporal extent calculation feature

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/lib/api.js Outdated
Comment on lines +1267 to +1268
const hasTemporalExtent = collection.extent?.temporal?.interval?.[0]?.[0] !== undefined
|| collection.extent?.temporal?.interval?.[0]?.[1] !== undefined

Copilot AI Oct 30, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check is insufficient to determine if temporal extent is defined. A collection with extent.temporal.interval set to [[null, null]] will fail this check (both values are not undefined), but this should be treated as missing temporal extent since null values indicate no data. The check should verify that at least one of the values is non-null: collection.extent?.temporal?.interval?.[0]?.[0] || collection.extent?.temporal?.interval?.[0]?.[1]

Suggested change
const hasTemporalExtent = collection.extent?.temporal?.interval?.[0]?.[0] !== undefined
|| collection.extent?.temporal?.interval?.[0]?.[1] !== undefined
const start = collection.extent?.temporal?.interval?.[0]?.[0];
const end = collection.extent?.temporal?.interval?.[0]?.[1];
const hasTemporalExtent = start != null || end != null;

Copilot uses AI. Check for mistakes.
Comment thread src/lib/api.js Outdated
Comment thread README.md Outdated
@matthewhanson matthewhanson requested a review from Copilot October 30, 2025 03:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.md Outdated
Comment thread README.md Outdated
matthewhanson and others added 2 commits October 29, 2025 23:48
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

@pjhartzell pjhartzell left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable to me.

- Resolved conflicts in database.js by including both getTemporalExtentFromItems and buildFieldsFilter exports
- Accepted main's restructured README (now a landing page)
- Accepted main's package-lock.json
- Added automatic temporal extent documentation to docs/usage/index.md
@matthewhanson matthewhanson added this to the 5.1.0 milestone Mar 5, 2026
@matthewhanson

Copy link
Copy Markdown
Member Author

Synced to main + fixed the test failures

Rebased onto current main (merge) and ported this feature through the TypeScript migration that landed since this branch was opened. Three things were broken and are now fixed — this PR's tests had never passed in CI:

  1. Ported to TS. src/lib/api.js and src/lib/database.js were converted to .ts on main. The new getTemporalExtentFromItems (now in database.ts) and populateTemporalExtentIfMissing (in api.ts) are reimplemented with proper types; getTemporalExtentFromItems added to the Backend interface in types.ts. Also switched the OpenSearch call to the dbQuery helper and _sourceIncludes (the typed equivalent of the old body._source).

  2. Fixed the populate condition (behavior change — please confirm). The guard was start != null || end != null ("has extent if either bound is set"), so a collection with a partial/open extent like [["2013-06-01", null]] was treated as complete and never populated — which is why GET /collections/:id returns temporal extent from items failed. Changed to start != null && end != null, i.e. recompute from items whenever either bound is missing. This matches what the test asserts (it expects the interval to become the item min/max). Note this means a declared-but-open start (2013) is replaced by the earliest item datetime (2015) — that's the intent encoded in the test, flagging it so you can confirm.

  3. Fixed a missing fixture. The test referenced tests/fixtures/stac/LC80100102015002LGN00.json, which was never committed (so the before-hook always ENOENT'd). Repointed it to the existing LC80100102015050LGN00.json — the test overrides id/datetime anyway.

Verified locally (clean env)

typecheck ✓ · lint ✓ · test:unit (69) ✓ · test:system (174, incl. the 3 temporal-extent tests) ✓ · build

CI note

audit-prod will be red here until #1118 merges to main — this branch still carries the pre-#1118 .nsprc/lockfile, which is the same repo-wide audit-gate failure #1118 fixes. Recommend merging #1118 first, then this picks up the fix on the next sync.

matthewhanson and others added 4 commits June 20, 2026 12:17
Align the temporal-extent test with the completed TS migration (#1095),
which set ava `extensions: ["ts"]`. As a lone `.js` file it only ran via
the explicit glob in system-tests.sh and would be skipped by a plain
`ava` invocation. Typed `t.context` via `StandUpResult` to match the
other migrated system tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the converted .ts test file (the prior commit recorded only the .js
removal because the combined `git add` aborted on the already-removed
.js pathspec).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Populate a collection's temporal extent per-bound: a missing start is
filled from the earliest item datetime and a missing end from the
latest, but any bound already declared on the collection is preserved
(previously an open-ended extent had BOTH bounds recomputed, discarding
a declared start). Update the system test, docs, and CHANGELOG to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@matthewhanson matthewhanson added this pull request to the merge queue Jun 20, 2026
Merged via the queue into main with commit 2acf0c1 Jun 20, 2026
4 checks passed
@jkeifer jkeifer mentioned this pull request Jul 1, 2026
1 task
jkeifer added a commit that referenced this pull request Jul 1, 2026
The automatic temporal extent feature (#999) replaced a collection's entire
extent.temporal.interval array with a single computed interval whenever the
overall extent needed backfilling, discarding any declared sub-intervals
(interval[1..n]). Only compute the overall extent (interval[0]) from items and
preserve declared sub-intervals. Adds a regression test covering a collection
with multiple intervals.
jkeifer added a commit that referenced this pull request Jul 1, 2026
The automatic temporal extent feature (#999) replaced a collection's entire
extent.temporal.interval array with a single computed interval whenever the
overall extent needed backfilling, discarding any declared sub-intervals
(interval[1..n]). Only compute the overall extent (interval[0]) from items and
preserve declared sub-intervals. Adds a regression test covering a collection
with multiple intervals.
@matthewhanson matthewhanson modified the milestones: 5.1.0, 6.0.0 Jul 1, 2026
jkeifer added a commit that referenced this pull request Jul 1, 2026
The automatic temporal extent feature (#999) replaced a collection's entire
extent.temporal.interval array with a single computed interval whenever the
overall extent needed backfilling, discarding any declared sub-intervals
(interval[1..n]). Only compute the overall extent (interval[0]) from items and
preserve declared sub-intervals. Adds a regression test covering a collection
with multiple intervals.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support automatic temporal extent calculation for collections

3 participants