Skip to content

fix: data export - replace O(n²) org unit descendants cross-join with path index scan [DHIS2-21490]#23880

Merged
jbee merged 2 commits into
masterfrom
DHIS2-21490
May 12, 2026
Merged

fix: data export - replace O(n²) org unit descendants cross-join with path index scan [DHIS2-21490]#23880
jbee merged 2 commits into
masterfrom
DHIS2-21490

Conversation

@jbee

@jbee jbee commented May 12, 2026

Copy link
Copy Markdown
Contributor

Summary

While testing I noticed that data export was unreasonably slow. I found that the SQL to expand direct OU matches to include all children was the cause. I ended up discussing this with Jason who came up with the SQL in this PR. The same query that took 2 minutes or so before now was instant.

Query plan

before:

Nested Loop Left Join ... (actual time=2.156..681.690 rows=1333 loops=1)
  Join Filter: ((ou_1.path)::text ~~ ((parent_ou.path)::text || '%'::text))
  Rows Removed by Join Filter: 1,771,738

Query plan here: https://explain.depesz.com/s/eK52 (638 ms)

after:

Nested Loop (actual time=0.040..1.049 rows=1333 loops=1)
  Join Filter: ou_1.path ~~ root.path || '%'
  ->  Seq Scan on organisationunit ou_1  (1333 rows)
  ->  Materialize (rows=1 loops=1333)

Query plan here: https://explain.depesz.com/s/s5Voe (8 ms)

@jbee jbee self-assigned this May 12, 2026
@jbee jbee changed the title fix: data export - SQL to add children for all direct matches fix: data export - SQL expand direct match to include children in O(n) May 12, 2026
@sonarqubecloud

Copy link
Copy Markdown

@jason-p-pickering jason-p-pickering changed the title fix: data export - SQL expand direct match to include children in O(n) fix: data export - replace O(n²) org unit descendants cross-join with path index scan [DHIS2-21490] May 12, 2026
@jbee
jbee enabled auto-merge (squash) May 12, 2026 13:34
@jbee
jbee merged commit fefefbb into master May 12, 2026
18 checks passed
@jbee
jbee deleted the DHIS2-21490 branch May 12, 2026 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants