Skip to content

Unblock postgres_instance_config PATCH (#163)#215

Draft
sdairs wants to merge 2 commits into
mainfrom
issue-163-pg-config-optionality
Draft

Unblock postgres_instance_config PATCH (#163)#215
sdairs wants to merge 2 commits into
mainfrom
issue-163-pg-config-optionality

Conversation

@sdairs
Copy link
Copy Markdown
Collaborator

@sdairs sdairs commented May 23, 2026

Summary

  • Flip every PgConfig field (23) plus both PostgresInstanceConfig nested fields (pgConfig, pgBouncerConfig) to Option<T> with skip_serializing_if = "Option::is_none", so Client::postgres_instance_config_patch can express a real partial update. Pair the change with 25 OPTIONALITY_EXEMPTIONS entries in spec_coverage_test.rs; the staleness detector will tell us automatically when upstream fixes the spec.
  • Re-enable the deferred PATCH round-trip in integration_postgres_test.rs (GET baseline → PATCH max_connections → poll-until-applied → PATCH back).
  • Add a gated CLICKHOUSE_CLOUD_POSTGRES_CONFIG_PROBE=1 probe phase that fires the 6 × 2 behaviour-matrix scenarios from the issue's follow-up comment via raw reqwest and prints a markdown table on stderr. Default off so it doesn't run on every integration test.

Closes #163.

Follow-ups

  • Capture the behaviour matrix and file the upstream spec issue (draft text already lives in Cover postgres_instance_config_post + postgres_instance_config_patch #163). To capture:
    CLICKHOUSE_CLOUD_POSTGRES_CONFIG_PROBE=1 \
      cargo test -p clickhouse-cloud-api --test integration_postgres_test cloud_postgres_crud_lifecycle -- --ignored --nocapture
    
  • Once the spec is fixed upstream, the new OPTIONALITY_EXEMPTIONS entries become stale and field_optionality_matches_spec will flag them — remove the block.

Test plan

  • cargo build --tests (whole workspace)
  • cargo clippy --tests
  • cargo test (all non-ignored tests pass)
  • cargo test -p clickhouse-cloud-api --test spec_coverage_test field_optionality_matches_live_spec -- --ignored — 25 new exemptions all register against the live spec, no stale entries
  • cargo test -p clickhouse-cloud-api --test integration_postgres_test -- --ignored --nocapture (live cloud creds required)
  • CLICKHOUSE_CLOUD_POSTGRES_CONFIG_PROBE=1 cargo test -p clickhouse-cloud-api --test integration_postgres_test -- --ignored --nocapture — capture behaviour matrix for upstream

🤖 Generated with Claude Code

The `pgConfig` schema has no `required` array and no field descriptions
start with "Optional", so the resolver marks every property required and
the generated model emits bare `T` fields. Serializing
`PgConfig { max_connections: 200, ..Default::default() }` then sends
zero/null/first-variant for every other property, and the live API
rejects all of them with
`Validation failed for following fields: pg_config.*`. The wrapping
`postgresInstanceConfig` also lists both nested objects as required,
making a partial PATCH unrepresentable.

Flip all 23 `PgConfig` fields and both `PostgresInstanceConfig` nested
fields to `Option<T>` with `skip_serializing_if = "Option::is_none"`,
add matching `OPTIONALITY_EXEMPTIONS` entries (caught automatically as
stale when the spec is fixed upstream), and re-enable the deferred
PATCH round-trip in the postgres integration test. The integration
test also gains a gated `CLICKHOUSE_CLOUD_POSTGRES_CONFIG_PROBE=1`
phase that captures the 6×2 behaviour matrix the upstream report
needs to cite real responses rather than assumptions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@sdairs sdairs requested a review from iskakaushik as a code owner May 23, 2026 15:49
@sdairs sdairs had a problem deploying to cloud-integration May 23, 2026 15:49 — with GitHub Actions Failure
@sdairs
Copy link
Copy Markdown
Collaborator Author

sdairs commented May 23, 2026

Blocked: response shape returns string-encoded numbers

Marking this PR as draft and blocked. The request-shape fix in this branch works (PATCH/POST now succeed against the live API per the captured matrix), but the response shape can't be deserialised into the spec-declared types.

What we see

For pgConfig fields the spec declares as integer / number, the live API returns the value wrapped in a JSON string. Example response from a successful PATCH (matrix body 2):

{"result":{"pgConfig":{"max_connections":"200"},"pgBouncerConfig":{},"message":""}}

max_connections is "200" (string), not 200 (number). Deserialising into Option<i64> fails with:

JSON error: invalid type: string "107", expected i64 at line 1 column 46

This affects every strictly-numeric pgConfig field — max_connections, effective_io_concurrency, max_worker_processes, max_parallel_workers, max_parallel_workers_per_gather, max_parallel_maintenance_workers, random_page_cost. The mixed-type fields (type: ["string","integer"] like work_mem, *_timeout, etc.) are already typed as serde_json::Value, so they're unaffected.

The same asymmetry applies in both directions: requests must send JSON numbers (string-encoded values are rejected), but responses come back string-encoded. So we can't just flip the field types — the model would then serialise wrong on the request side.

Why this PR is paused

A client-side workaround (custom deserialize_with that accepts both number and string) was considered and rejected — we don't want to mangle types to paper over an API inconsistency. This needs to be fixed on the API side: response numeric fields should match the spec's declared integer/number types.

Next steps

  • File / link upstream API issue for response-side type mismatch on pgConfig numeric fields
  • Unblock once the API returns numbers as numbers
  • Re-enable the gated integration round-trip (patch pgConfig.max_connections in integration_postgres_test.rs)

@sdairs sdairs marked this pull request as draft May 23, 2026 17:31
@sdairs sdairs added the blocked Can't be progressed label May 23, 2026
The behaviour matrix captured in #163 shows the live API rejects any
postgresInstanceConfig PATCH or POST body that omits either nested
object: omitting pgBouncerConfig yields `BAD_REQUEST: request
body.pgBouncerConfig: 'undefined'`, omitting pgConfig yields the
symmetric error. Sending `{}` for either is accepted (matrix body 2 →
200). The previous commit had flipped both to Option<T> with
skip_serializing_if so partial PATCHes could send only the changed
half, but that shape doesn't match the API.

Flip pg_config and pg_bouncer_config on PostgresInstanceConfig back to
required (bare T with #[serde(default)] so the default envelope still
serialises to `{ "pgConfig": {}, "pgBouncerConfig": {} }`), drop the
two corresponding OPTIONALITY_EXEMPTIONS entries, and update the
fixture/integration tests to construct the new shape. Inner pgConfig
fields stay Option<T> with skip_serializing_if — those remain
opt-in per the spec's all-optional partial-update semantics.

Response-side deserialisation is still broken (the API returns
strictly-numeric pgConfig fields wrapped in JSON strings), which is
tracked separately and blocks the integration round-trip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@sdairs sdairs had a problem deploying to cloud-integration May 23, 2026 17:35 — with GitHub Actions Error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blocked Can't be progressed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cover postgres_instance_config_post + postgres_instance_config_patch

1 participant