feat(queryRunner): raise maxResultSize upper bound to 10000#27468
Merged
feat(queryRunner): raise maxResultSize upper bound to 10000#27468
Conversation
The previous 1000 cap forced admins to pick a conservative row limit for Query Runner executions even when their ingestion runner could comfortably return larger result sets. Raise the schema maximum to 100000 so operators have headroom to expose more rows to SQL Studio and downstream consumers. Enforcement and the default (100) are unchanged; the backend still injects the value from QueryRunnerConfig.querySettings.maxResultSize on every trigger, so this only widens the ceiling admins can opt into. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
github-actions
Bot
added
Ingestion
safe to test
Contributor
There was a problem hiding this comment.
Pull request overview
Raises the allowed upper bound for QueryRunnerRequest.maxResultSize so admins can configure larger query result limits for Query Runner / SQL Studio executions without requiring another schema change.
Changes:
- Increased
QueryRunnerRequest.maxResultSizeJSON schema maximum from1000to100000.
Reviewer feedback: 50k is a better trade-off between giving admins headroom and keeping the batch workflow response from returning unwieldy payloads. Enforcement and defaults unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pmbrull
changed the title
Reviewer feedback: 10k is a more conservative ceiling that still gives admins meaningful headroom over the previous 1k cap while keeping payload sizes predictable through the batch workflow transport. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pmbrull
changed the title
Code Review ✅ ApprovedIncreases the maxResultSize upper bound to 10000 for the query runner. No issues found. OptionsDisplay: compact → Showing less information. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
tomasmontielp
approved these changes
Apr 17, 2026
Contributor
🔴 Playwright Results — 2 failure(s), 16 flaky✅ 2985 passed · ❌ 2 failed · 🟡 16 flaky · ⏭️ 110 skipped
Genuine Failures (failed on all attempts)❌
|
|
pmbrull
added a commit
that referenced
this pull request
Apr 20, 2026
* feat(queryRunner): raise maxResultSize upper bound to 100000 The previous 1000 cap forced admins to pick a conservative row limit for Query Runner executions even when their ingestion runner could comfortably return larger result sets. Raise the schema maximum to 100000 so operators have headroom to expose more rows to SQL Studio and downstream consumers. Enforcement and the default (100) are unchanged; the backend still injects the value from QueryRunnerConfig.querySettings.maxResultSize on every trigger, so this only widens the ceiling admins can opt into. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(queryRunner): lower maxResultSize ceiling from 100000 to 50000 Reviewer feedback: 50k is a better trade-off between giving admins headroom and keeping the batch workflow response from returning unwieldy payloads. Enforcement and defaults unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(queryRunner): further lower maxResultSize ceiling to 10000 Reviewer feedback: 10k is a more conservative ceiling that still gives admins meaningful headroom over the previous 1k cap while keeping payload sizes predictable through the batch workflow transport. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes https://github.com/open-metadata/ai-platform/issues/555
Summary
QueryRunnerRequest.maxResultSizeschema maximum from1000to10000.100on the companion Collate config) and minimum (1) unchanged.query_runner_utils.validate_and_enforce_query_limit) and backend injection fromQueryRunnerConfig.querySettings.maxResultSizealready respect whatever integer the schema allows.Motivation
The previous
1000cap forced admins to pick a conservative row ceiling for Query Runner / SQL Studio executions even when their ingestion runner could comfortably return larger result sets. Widening the maximum to10000gives operators a meaningful, conservative bump while keeping payload sizes predictable through the batch workflow transport (Argo response → REST body → UI/LLM).Why this schema matters (and why Collate needs a matching PR)
The Query Runner feature has its ceiling enforced at two schema boundaries:
QueryRunnerConfig.querySettings.maxResultSizeQueryRunnerRequest.maxResultSize(this PR)The field appears in both because the Python runner doesn't read the Collate DB — the value has to be packed into the OSS-owned
automationsrequest envelope for the generic runner to consume it. Bounds in both are defense-in-depth: widening only one doesn't help — injection would still be capped by the other.So this OSS PR is the transport-side half. The Collate-side PR raises the admin-config ceiling and updates the help docs: https://github.com/open-metadata/openmetadata-collate/pull/3696
Both need to merge for the higher ceiling to be usable end-to-end.
Test plan
mvn -pl openmetadata-spec clean installregenerates the Java model with the new bound.make generateregenerates the Pydantic model andQueryRunnerRequest(maxResultSize=10000)validates;QueryRunnerRequest(maxResultSize=10001)fails with a validation error.maxResultSizevalues up to10000without client-side rejection.maxResultSize = 5000, execute a query withoutLIMIT, verify the Python workflow injectsLIMIT 5000and returns successfully.🤖 Generated with Claude Code