Skip to content

Perf: 1Million Challenge for CDA TS GET#1692

Open
krowvin wants to merge 4 commits intodevelopfrom
feature/timeseries-read-perf
Open

Perf: 1Million Challenge for CDA TS GET#1692
krowvin wants to merge 4 commits intodevelopfrom
feature/timeseries-read-perf

Conversation

@krowvin
Copy link
Copy Markdown
Collaborator

@krowvin krowvin commented Apr 14, 2026

Work on #1685

Improve unfiltered /timeseries read performance + perf coverage

  • move the main unfiltered /timeseries read path off the Oracle retrieve_ts_out_tab hot path and onto direct AV_TSV_DQU/jOOQ reads while preserving retrieve_ts response semantics in Java
  • keep the direct path aligned with Oracle behavior using integration parity coverage, including a DST-window regression case for regular series
  • Java/Gradle benchmark harness
  • add explicit fixture teardown for the benchmark runner so local perf runs exit cleanly and leave the workspace in a predictable state

Benchmark

Local single-request benchmarks against the Docker/Testcontainers Oracle dev environment, using page-size equal to the requested row count:

  • legacy path, 250000 points: 40.679s
  • new direct path, 250000 points: 4.652501s
  • legacy path, 500000 points: HTTP 500

The current Java benchmark harness also confirms the direct path now returns the expected count for the large seeded series:

  • direct path, 250000 points: 250000 reported rows, correct final timestamp, 4.652501s

The benchmark task writes JSON artifacts under load_data/performance/results/.

Validation

  • ./gradlew :cwms-data-api:integrationTests --tests cwms.cda.api.TimeSeriesDirectReadParityIT
  • ./gradlew :cwms-data-api:timeseriesReadBenchmark -Pbenchmark.pointCount=250000 -Pbenchmark.runs=1 -Pbenchmark.forceReseed=true

Notes

  • scope here is the main unfiltered /timeseries read path; /timeseries/filtered is still on the legacy path
  • could move to larger streaming/lower-materialization for more improvements

Seeding 250,000 TS values for SPK and reading them back via CDA returns in 4.6s

Before changes

Current 500k+ Fails
Current 250k in 40.679s

Changes

250k in 4.793s
500k in 7.498s
1,000,000 in about 14.9s

@krowvin krowvin requested a review from MikeNeilson April 14, 2026 12:01
@krowvin
Copy link
Copy Markdown
Collaborator Author

krowvin commented Apr 14, 2026

Test failures appear to be related to the updated CWMS DB schema / AV_LOC_ALIAS/LOCATION_CODE changes not the src specific to this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant