Add df.instances indexes for ordered listing (monitoring plan, PR 2)

[crprashant]

What

Add/replace indexes on df.instances so that df.list_instances() can list rows in its natural order (newest-first) without a sort, on both the unfiltered and status-filtered paths:

• New idx_instances_created_at on (created_at DESC, id) — serves the unfiltered listing ... ORDER BY created_at DESC LIMIT, which previously had no supporting index.
• Replace idx_instances_status(status) with the composite idx_instances_status(status, created_at DESC, id) — serves the status-filtered listing WHERE status = $1 ORDER BY created_at DESC LIMIT and still covers the pending-instance scan via the leading status column.

The trailing id prepares the access path for the keyset pagination planned for a later PR (ORDER BY created_at DESC, id ASC); df.list_instances() does not order by id yet, so this PR does not change the current result ordering.

Files: src/lib.rs (fresh-install DDL), sql/pg_durable--0.2.3--0.2.4.sql (upgrade DDL), docs/upgrade-testing.md (version-specific notes).

Why

This is PR 2 of the agreed, incremental monitoring-functions plan on #167 (issues #87 and #146). Efficient ordered/paginated listing for external clients (#146) needs an index that matches the ORDER BY created_at DESC access pattern; today the unfiltered path always sorts and the status-filtered path can't use the single-column status index for the sort.

Design note (RLS)

df.instances has a row-level-security policy (instances_user_isolation) filtering submitted_by = current_user, so a per-user index leading with submitted_by would be more selective for a single session. The created_at-leading design is intentional: it is optimal for the admin / external-client global-listing path (#146) that reads across submitters, and still removes the per-query sort. A submitted_by-leading refinement can be revisited later if profiling shows the per-user path dominates.

Upgrade & compatibility

• Scenario A: The upgrade script recreates both indexes with column lists and DESC ordering identical to the fresh-install DDL, so pg_get_indexdef() is byte-identical on both paths. Verified locally — both paths produce idx_instances_status … (status, created_at DESC, id) and idx_instances_created_at … (created_at DESC, id).
• Scenario B1: Performance-only. The new .so issues the same ORDER BY created_at DESC queries (referencing only existing columns) against older schemas; they stay correct and simply fall back to a sort until ALTER EXTENSION UPDATE is applied.
• Scenario B2: No data migration — DROP INDEX / CREATE INDEX rebuild access-path metadata only. The CREATE INDEX builds take a SHARE lock on df.instances; on a large table, run the upgrade in a maintenance window.

0.2.4 is still unreleased, so the DDL is appended to the in-dev sql/pg_durable--0.2.3--0.2.4.sql (no version bump, no new upgrade script).

Scope

Indexes only — no changes to df.list_instances() behavior or signature. The N+1 fix, label_filter + keyset pagination, and the truncation policy land in subsequent PRs per the plan.

cc @pinodeca