Skip to content

Trace latency: add Postgres trace bounds and storage state #286

@thorrester

Description

@thorrester

Parent: #281

Goal

Store bounded trace lookup metadata and trace storage state in Postgres so read paths can avoid unnecessary Delta/object-store work.

Postgres should own transactional metadata and coordination state here, not span payloads or broad analytical scans.

Scope

  • Add Postgres tables or rows for trace bounds and storage state.
  • Track enough information to bound trace-id lookups by time/partition.
  • Track storage freshness state such as latest observed Delta version and a monotonic refresh epoch.
  • Keep updates idempotent and safe under concurrent writers/pods.
  • Expose the state needed by query cache keys and maintenance coordination.

High-level design

This is control-plane metadata. The row count and payload size should stay bounded relative to trace volume. DataFusion/Delta remains the source of span data; Postgres helps the system decide which Delta partitions and versions are worth reading.

The storage state should become the common freshness marker for result-cache invalidation and request-path refresh detection.

Acceptance criteria

  • Trace bounds can be written and read through Postgres.
  • Query code can obtain a refresh epoch or equivalent storage freshness marker.
  • Bounds updates tolerate duplicate or concurrent writes.
  • No broad span payloads or analytical query results are stored in Postgres.
  • Tests cover bounds lookup, missing bounds, and concurrent update behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions