Fix: Remove status partitioning on v1_(runs|dags|tasks)_olap

[mrkaye97]

Description

Removes status partitioning, which we think will improve performance in multiple parts of the system a bunch. It'll make updates easier since we won't ever need to move rows between partitions (which postgres seems to not recommend), and it should improve query performance a bunch too. Also removed the old tenant index on the runs table in favor of the new one that's superseded it.

Same basic process as before:

1. Creates new tables that are basically copies of the existing ones, except that they have only (inserted_at, id) as the PK and are partitioned differently
2. Creates triggers to insert data into those tables
3. Backfills those tables
4. Progresses to a second migration where we drop the old tables rename the new ones to cut over

Type of change

• Bug fix (non-breaking change which fixes an issue)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hatchet-docs	Ready	Preview, Comment	Apr 24, 2026 7:39pm

[Copilot]

Pull request overview

Updates OLAP schema/indexing for v1_runs_olap to replace the existing tenant-oriented index with a new (tenant_id, readable_status, inserted_at DESC) index, and provides a migration to roll the change out across partition levels.

Changes:

• Add a new v1_runs_olap index optimized for tenant + status + recency lookups.
• Introduce a no-tx Goose migration that drops the old ix_v1_runs_olap_tenant_id index and creates the new index across leaf and parent partitions.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

File	Description
sql/schema/v1-olap.sql	Adds the new ix_v1_runs_olap_tenant_status_ins_at index definition for v1_runs_olap.
cmd/hatchet-migrate/migrate/migrations/20260410190713_v1_0_97.go	Drops the old runs OLAP tenant index and builds the new tenant+status+inserted_at index across partitions.

[github-actions]

Benchmark results

goos: linux
goarch: amd64
pkg: github.com/hatchet-dev/hatchet/pkg/scheduling/v1
cpu: AMD Ryzen 9 7950X3D 16-Core Processor          
              │ /tmp/old.txt │         /tmp/new.txt         │
              │    sec/op    │   sec/op     vs base         │
RateLimiter-8    49.85µ ± 4%   51.84µ ± 8%  ~ (p=0.394 n=6)

              │ /tmp/old.txt │         /tmp/new.txt          │
              │     B/op     │     B/op      vs base         │
RateLimiter-8   137.7Ki ± 0%   137.7Ki ± 0%  ~ (p=0.734 n=6)

              │ /tmp/old.txt │          /tmp/new.txt          │
              │  allocs/op   │  allocs/op   vs base           │
RateLimiter-8    1.022k ± 0%   1.022k ± 0%  ~ (p=1.000 n=6) ¹
¹ all samples are equal

_{Compared against main (3acccfa)}

[abelanger5]

idea to speed things up: we can do runs, tasks and dags in parallel by starting 3 separate transactions in parallel and running the migration in non-tx mode

[abelanger5]

rare edge case, but what happens if this runs while we add or drop a partition? I'm guessing it won't cause much of an impact on the add partition since we have 24 hours before data starts getting written to the partition. But more curious about the case when we delete a partition.

[mrkaye97]

I guess we'd either hang trying to drop the partition or the migration would fail with a no such table error and we'd retry

[abelanger5]

since the old v1_runs_olap could technically have multiple statuses, do we want to pick the larger status here or just not worry about that edge case?

[mrkaye97]

yeah, initially I was thinking that since the runs table doesn't have a retry count we don't know how to do this, but maybe we can naively just pick the largest one in case of a conflict and it'll work...