Skip to content

feat(warehouse): add adaptive warehouse support#36

Open
GClunies wants to merge 1 commit into
datacoves:mainfrom
GClunies:snowcap-27
Open

feat(warehouse): add adaptive warehouse support#36
GClunies wants to merge 1 commit into
datacoves:mainfrom
GClunies:snowcap-27

Conversation

@GClunies

Copy link
Copy Markdown
Contributor

Summary

Adds support for Snowflake adaptive warehouses (closes #27). Adaptive warehouses are declarable in YAML/Python via warehouse_type: ADAPTIVE with max_query_performance_level, replacing the ~11 warehouses we currently manage through one-off SQL. Follows the Gen2 single-class pattern from #22.

Changes

  • WarehouseType.ADAPTIVE; max_query_performance_level typed as Optional[WarehouseSize] (XSMALL–X4LARGE per Snowflake docs; X5/X6LARGE rejected, default left to Snowflake = XLARGE)
  • Validation rejects properties that don't apply to adaptive warehouses (warehouse_size, cluster counts, scaling policy, suspend/resume, query acceleration, resource constraint, generation) via a single ADAPTIVE_UNSUPPORTED_FIELDS set, with real defaults read from dataclasses.fields
  • fetch_warehouse round-trips SHOW WAREHOUSES output for adaptive warehouses with no spurious drift, reusing the same field set; tolerates SHOW output that lacks the new adaptive columns
  • CREATE renders as CREATE WAREHOUSE ... WITH WAREHOUSE_TYPE = 'ADAPTIVE' MAX_QUERY_PERFORMANCE_LEVEL = ... (documented-equivalent form) — no lifecycle override or new resource class
  • STANDARD↔ADAPTIVE conversion ALTERs pinned by tests; plan errors on conversions Snowflake doesn't support (to/from X5LARGE/X6LARGE)
  • Grants (USAGE/OPERATE/MONITOR) work unchanged — covered by test
  • Warehouse docstring + regenerated docs page; plan previews no longer show size: None for adaptive warehouses

Notes

  • QUERY_THROUGHPUT_MULTIPLIER intentionally not modeled (out of scope; fetch omits it so out-of-band values can't cause drift)
  • SHOW WAREHOUSES adaptive columns and the conversion ALTER forms are pinned by unit tests but unverified against a live Enterprise/AWS account (adaptive is Enterprise+/AWS-only)
  • tools/generate_resource_docs.py has pre-existing template drift (blank frontmatter description, missing CLI-label slot) that would wipe content from all resource pages on the next full regen — worth its own fix; this PR hand-restored the two affected lines on warehouse.md

1564 unit tests passing; ruff and mypy clean.

Adaptive warehouses let Snowflake pick size and cluster count
automatically, capped by MAX_QUERY_PERFORMANCE_LEVEL instead of
WAREHOUSE_SIZE/MIN|MAX_CLUSTER_COUNT. Follows the Gen2 single-class
pattern: ADAPTIVE joins WarehouseType, max_query_performance_level
reuses WarehouseSize (XSMALL..X4LARGE, X5/X6LARGE rejected), and a
shared ADAPTIVE_UNSUPPORTED_FIELDS set drives both __post_init__
validation (defaults read from dataclasses.fields) and fetch_warehouse
null-normalization so SHOW WAREHOUSES output round-trips without
spurious drift.

CREATE renders via the documented-equivalent WAREHOUSE_TYPE = 'ADAPTIVE'
form, so no lifecycle override or new resource class is needed. Both
STANDARD<->ADAPTIVE conversion deltas are pinned by tests, and plan
fails fast on conversions Snowflake doesn't support (to or from
X5LARGE/X6LARGE). Grants need no changes (grant code keys only on
ResourceType.WAREHOUSE) and are covered by a test. Docs are updated in
the Warehouse docstring and regenerated.
@noel

noel commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Thanks @GClunies — clean reuse of the existing warehouse pattern, and the design holds up well. I checked out the branch and verified:

  • f.default design is sound. Every field in ADAPTIVE_UNSUPPORTED_FIELDS resolves to a real default (none use default_factory, which would yield MISSING). Deriving _ADAPTIVE_FIELD_DEFAULTS from dataclasses.fields instead of hand-copying is a nice touch.
  • Nulling is symmetric. __post_init__ and fetch_warehouse null the same field set, so declared and fetched adaptive state both land on None — no spurious drift there.
  • Conversion guard is correct in both directions (the adaptive side has warehouse_size=None, so the X5/X6LARGE check still fires).
  • Ran test_blueprint / test_data_provider / test_grant / test_resources: 367 passed.

Requested before merge

  1. Verify omitted max_query_performance_level against a live account. Snowflake defaults it to XLARGE when omitted. The round-trip tests either set it explicitly (LARGE→LARGE) or simulate SHOW not reporting it (→None) — none cover "config omits it, live SHOW reports XLARGE." In that case fetch returns "XLARGE" while the manifest has None, which would drift every plan (and emit an ALTER to unset it). Please confirm on a live Enterprise/AWS account; if it drifts, defaulting the spec's max_query_performance_level to XLARGE for ADAPTIVE would make both sides match. This is the one I'd gate merge on.

Requested follow-up (separate issue is fine)

  1. File an issue for the doc-generator drift you flagged. generate_resource_docs.py blanking content on the next full regen is a latent landmine — worth capturing now while it's fresh, even though the fix is out of scope here.

Looks good

  • Defensive fetch: tolerates SHOW missing the adaptive columns and a non-WarehouseSize size value.
  • Plan-time error on unsupported conversions rather than failing mid-apply.
  • Good round-trip + grant coverage; size: None no longer shown in plan previews.

Solid work — pending the live check on the default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(warehouse): support adaptive warehouses (CREATE ADAPTIVE WAREHOUSE)

2 participants