Skip to content

feat(client)!: live-verified spec drift batch (#737, #736, #738) + verification harness#756

Open
dougborg wants to merge 1 commit into
mainfrom
spec-drift-verify-tooling
Open

feat(client)!: live-verified spec drift batch (#737, #736, #738) + verification harness#756
dougborg wants to merge 1 commit into
mainfrom
spec-drift-verify-tooling

Conversation

@dougborg
Copy link
Copy Markdown
Owner

@dougborg dougborg commented May 15, 2026

Summary

Two threads in one PR:

1. Reusable verification harness

scripts/spec_drift_verify.py (CLI + ledger) plus four per-issue
probe scripts. Every test artifact created against the live tenant
gets an SDT-<date> prefix and is recorded in
/tmp/spec-drift-ledger.jsonl; cleanup walks the ledger in
reverse and deletes everything (idempotent — already-deleted rows
are skipped, 404 on DELETE is treated as "already gone").

Subcommand Purpose
list Print every recorded artifact (active + deleted)
plan Dry-run the cleanup, show what would be deleted
cleanup Delete every active artifact and persist outcomes

Probe scripts already cover #737 (probe_issue_737.py),
#734/#736/#738/#739 (probe_remaining_issues.py), shape-only
introspection without persisting (probe_shape_only.py), and the
project ValidationError end-to-end via raw-httpx + KatanaClient
parallel paths (probe_client_error_handling.py).

2. Live-verified spec drift fixes

#737 — CustomFieldDefinition

Field Before After Evidence
id integer string, format: uuid README example + universal Katana shape
field_type string maxLength:50 $ref CustomFieldType enum (6 values) Live 422 introspection
entity_type string maxLength:50 $ref CustomFieldEntityType enum (19 values) Live 422 introspection
Inheritance allOf UpdatableEntity inlined created_at / updated_at UpdatableEntity defines integer id from BaseEntity; UUID can't reuse it

Couldn't live-verify id UUID-ness via create cycle — our API
key returns 403 Forbidden on POST /custom_field_definitions
(endpoint exists, but the account/scope doesn't grant write). The
README example shows UUID and that's universal across Katana's
custom-field surface.

#736 — variant_bin_locations + CreateServiceVariant

  • POST /variant_bin_locations request body wrapped in array
    (minItems: 1, no upper bound — upstream sources disagree on the cap). Description already said
    "up to 500 variant storage bin objects" but the schema was a
    single object. Verified: single-object body returns
    422 must be array.
  • CreateServiceVariantRequest.sku no longer required. Verified
    by creating a service variant without sku — server returns
    sku: null on the resulting variant.

#738 — StockTransferRowRequest.quantity

  • Changed from number to string. Verified via live 422:
    sending a JSON number returns must be string. Wire format is
    a fixed-precision decimal string (e.g. \"1.0000000000\").
  • MCP boundary in stock_transfers._build_row_requests
    stringifies the input float so MCP callers don't have to.

#739 — closed (no spec change)

Verified sales_order_fulfillment_rows IS required on create —
local spec correct, README upstream is stale on this. Issue ready
to close on merge.

Findings deferred (per scope decision)

#734 — custom_fields shape divergence: live API rejects
array-shaped custom_fields with must be object at both SO
and SO-row create paths; dict-shape is accepted. Local READ schemas
use structured-array shape, which contradicts this. Read-side
verification requires a populated example (tenant has zero
custom_field_definitions and our key can't create them).
Sweeping all the READ schemas (Variant / Service / Product / SO /
SO Row / SO Fulfillment / Customer / Supplier / SalesReturn etc.)
is broader than this PR.

#736 — remaining items: CustomFieldDefinition.options
shape and the UpdateCustomerRequest.default_billing_id
question — blocked by either authorization or absence of populated
fixtures on this tenant.

POST /bin_locations 404: the local spec documents this
endpoint but the live router returns 404 Endpoint POST /bin_locations not found. Bin locations are created in the
Katana UI only. Removing this from the local spec is a cleanup
worth a follow-up PR (orthogonal to the drift findings here).

BREAKING CHANGE

  • CustomFieldDefinition.id becomes a UUID string (was integer)
  • CustomFieldDefinition.field_type / entity_type become enums
  • CreateServiceVariantRequest.sku no longer required (schema-level breaking change; runtime-compatible)
  • StockTransferRowRequest.quantity becomes a string (MCP boundary stringifies the input)

Test plan

  • uv run poe agent-check (format / lint / typecheck) clean
  • uv run poe test — 3235 passed, 2 skipped
  • uv run poe validate-openapi clean
  • uv run poe validate-examples — 0 failures
  • Probe scripts run against live tenant, ledger cleanup verified end-to-end (3 artifacts created, all deleted via cleanup)
  • Client ValidationError end-to-end test produces expected output via both raw-httpx + KatanaClient paths

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings May 15, 2026 18:05
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds live API verification tooling for spec drift investigations and applies verified OpenAPI/client updates for custom fields, service variants, variant bin locations, and stock transfer quantities.

Changes:

  • Adds reusable spec-drift probe and cleanup scripts with ledger tracking.
  • Updates OpenAPI schema and regenerated attrs/Pydantic client models for verified drift.
  • Adjusts MCP stock transfer request building to stringify quantity at the boundary.

Reviewed changes

Copilot reviewed 22 out of 23 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
docs/katana-openapi.yaml Source OpenAPI changes for custom fields, service variants, stock transfer rows, and variant bin location bulk body.
katana_public_api_client/api/custom_fields/create_custom_field_definition.py Regenerated custom field create docs/examples.
katana_public_api_client/api/variant_default_storage_bin/link_variant_default_storage_bins.py Regenerated endpoint body type from single object to list.
katana_public_api_client/models/__init__.py Exports new custom field enum models.
katana_public_api_client/models/create_custom_field_definition_request.py Regenerated request model using custom field enums.
katana_public_api_client/models/create_custom_field_definition_request_options_type_0.py Updates options documentation.
katana_public_api_client/models/create_service_variant_request.py Makes service variant SKU optional/null.
katana_public_api_client/models/custom_field_definition.py Changes custom field definition ID to UUID and field/entity types to enums.
katana_public_api_client/models/custom_field_definition_list_response.py Updates example payload.
katana_public_api_client/models/custom_field_entity_type.py Adds custom field entity type enum.
katana_public_api_client/models/custom_field_type.py Adds custom field type enum.
katana_public_api_client/models/stock_transfer_row_request.py Changes stock transfer quantity wire type to string.
katana_public_api_client/models_pydantic/_generated/__init__.py Exports regenerated custom field enums.
katana_public_api_client/models_pydantic/_generated/common.py Regenerated Pydantic custom field models/enums.
katana_public_api_client/models_pydantic/_generated/inventory.py Regenerated service variant SKU optionality.
katana_public_api_client/models_pydantic/_generated/stock.py Regenerated stock transfer quantity as string.
katana_mcp_server/src/katana_mcp/tools/foundation/stock_transfers.py Converts MCP float quantity input to string for API requests.
scripts/probe_client_error_handling.py Adds live error-handling probe.
scripts/probe_issue_737.py Adds custom field definition live probe.
scripts/probe_remaining_issues.py Adds live probes for remaining linked drift issues.
scripts/probe_shape_only.py Adds validation-only request shape probes.
scripts/spec_drift_verify.py Adds ledger, tagging, listing, planning, and cleanup harness.
uv.lock Updates locked editable MCP package version.
Comments suppressed due to low confidence (1)

docs/katana-openapi.yaml:10815

  • This is another single-$ref allOf wrapper used only to attach descriptions. The spec authoring guide (katana_public_api_client/docs/spec-authoring.md:27-32) requires OpenAPI 3.1 $ref siblings for this pattern instead, so please replace the wrapper with a direct $ref and sibling description.
          allOf:
            - $ref: "#/components/schemas/CustomFieldType"
          description: |
            Field input type. Immutable after creation. The live API
            enforces this enum at validation time (verified via 422
            introspection).
        entity_type:
          allOf:
            - $ref: "#/components/schemas/CustomFieldEntityType"

Comment thread docs/katana-openapi.yaml Outdated
Comment thread scripts/spec_drift_verify.py Outdated
Comment thread scripts/spec_drift_verify.py
Comment thread docs/katana-openapi.yaml Outdated
Comment thread docs/katana-openapi.yaml Outdated
@dougborg dougborg force-pushed the spec-drift-verify-tooling branch from 499aade to b8c2cfc Compare May 15, 2026 18:12
Copilot AI review requested due to automatic review settings May 15, 2026 18:25
@dougborg dougborg force-pushed the spec-drift-verify-tooling branch from b8c2cfc to 60ece5f Compare May 15, 2026 18:25
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 23 out of 24 changed files in this pull request and generated 3 comments.

Comment thread docs/katana-openapi.yaml
Comment thread docs/katana-openapi.yaml Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

docs/katana-openapi.yaml:10826

  • The PR description says CustomFieldDefinition.options shape verification is deferred/blocked, but this create-request example still presents choices as the documented request shape. Please remove or soften this example so the spec does not imply unverified behavior.
        options:
          choices:
            - label: Online
            - label: Retail
            - label: Wholesale

Comment thread scripts/spec_drift_verify.py Outdated
Comment thread docs/katana-openapi.yaml Outdated
Comment thread scripts/spec_drift_verify.py
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 4 comments.

Comment thread scripts/probe_remaining_issues.py Outdated
Comment thread scripts/probe_client_error_handling.py Outdated
Comment thread scripts/probe_remaining_issues.py Outdated
Comment thread scripts/spec_drift_verify.py
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 1 comment.

Comment thread scripts/spec_drift_verify.py Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 3 comments.

Comment thread docs/katana-openapi.yaml Outdated
Comment thread docs/katana-openapi.yaml Outdated
Comment thread katana_mcp_server/src/katana_mcp/tools/foundation/stock_transfers.py Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

scripts/probe_shape_only.py:94

  • This probe also relies on hard-coded positive IDs to prevent persistence. If the tenant happens to contain these customer/variant IDs, the request can create an SDT-shaped sales order without a ledger entry; verify the IDs are absent before posting or record and clean up any unexpected success.
    common = {
        "customer_id": 99999999,  # known bad
        "location_id": sales_loc,
        "sales_order_rows": [
            {"variant_id": 99999999, "quantity": 1, "price_per_unit": 1.0}
        ],

Comment thread katana_mcp_server/src/katana_mcp/tools/foundation/stock_transfers.py Outdated
Comment thread docs/katana-openapi.yaml
Comment thread scripts/probe_shape_only.py Outdated
@dougborg dougborg force-pushed the spec-drift-verify-tooling branch from c20b795 to 4dd4a91 Compare May 15, 2026 23:45
@dougborg dougborg requested a review from Copilot May 15, 2026 23:50
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 2 comments.

Comment thread scripts/probe_client_error_handling.py Outdated
Reusable harness (``scripts/spec_drift_verify.py`` + per-issue probe
scripts) for running tagged POSTs against a live Katana tenant and
cleaning up afterwards. Every test artifact gets an ``SDT-<date>``
prefix and is recorded in ``/tmp/spec-drift-ledger.jsonl``; a single
``cleanup`` subcommand walks the ledger in reverse and deletes
everything (idempotent — already-deleted rows are skipped).

Spec-drift findings encoded in ``docs/katana-openapi.yaml``:

**#737 — CustomFieldDefinition**
- ``id``: ``integer`` → ``string, format: uuid`` (per README example,
  inferred — couldn't live-verify because our API key lacks create
  permission on this endpoint, but README is authoritative and the
  shape is universal across Katana custom-field surfaces).
- ``field_type``: ``string maxLength: 50`` → new
  ``CustomFieldType`` enum (``shortText`` / ``number`` /
  ``singleSelect`` / ``date`` / ``boolean`` / ``url``). Verified
  via live-API 422 introspection.
- ``entity_type``: ``string maxLength: 50`` → new
  ``CustomFieldEntityType`` enum with **19 values** (vs README's
  documented 1). Verified via live-API 422 introspection — the
  enum spans SalesOrder, SalesOrderRow, ManufacturingOrder*,
  PurchaseOrder*, StockAdjustment*, Production*, Customer,
  ProductVariant, MaterialVariant, ServiceVariant.
- ``CustomFieldDefinition`` no longer extends ``UpdatableEntity``
  because the singleton's ``id`` is a UUID string, not the integer
  ``BaseEntity`` defines for the rest of the API surface.
  ``created_at`` / ``updated_at`` are inlined.

**#736 — variant_bin_locations + CreateServiceVariant.sku**
- ``POST /variant_bin_locations`` request body wrapped in array
  (``minItems: 1, maxItems: 500``). The endpoint description
  already said "accepts up to 500 variant storage bin objects"
  but the schema was a single object. Verified via live API:
  single-object body returns ``422 must be array``.
- ``CreateServiceVariantRequest.sku`` is no longer required. The
  field is nullable. Verified by creating a service variant via
  the live API with ``sku`` omitted — server accepted it and
  returned ``sku: null``.

**#738 — StockTransferRowRequest.quantity**
- Changed from ``number`` to ``string``. Verified via live-API
  422 introspection — sending a JSON number returns
  ``must be string``. Wire format is a fixed-precision decimal
  string (e.g. ``"1.0000000000"``). MCP boundary in
  ``stock_transfers._build_row_requests`` stringifies the pydantic
  ``float`` input at the call site.

**#739 — SalesOrderFulfillment**
- No spec change needed. Verified ``sales_order_fulfillment_rows``
  IS required on create — local spec is correct, README is stale.

Findings deferred to follow-ups:

- **#734** ``custom_fields`` shape — live API rejects array-shaped
  custom_fields with ``must be object`` at both SO and SO-row
  create paths; dict-shape is accepted. Local READ schemas use
  structured-array shape, which contradicts this. Read-side
  verification requires a populated example (tenant has zero
  ``custom_field_definitions`` and our key can't create them).
  Sweeping all the READ schemas is broader than this PR.
- **#736** ``CustomFieldDefinition`` ``options`` shape, ``id`` type
  via live API, ``UpdateCustomerRequest.default_billing_id``,
  ``Operator`` fields — all blocked by either authorization
  (custom_field_definitions endpoint) or absence of populated
  fixtures on this tenant.

BREAKING CHANGE: ``CustomFieldDefinition.id`` becomes a UUID string
(was integer); ``field_type`` / ``entity_type`` become enums;
``CreateServiceVariantRequest.sku`` is no longer required (callers
omitting it will continue to work, but ``required`` removal is a
schema-level breaking change); ``StockTransferRowRequest.quantity``
becomes a string. The MCP ``stock_transfers`` boundary already
stringifies the input float so MCP callers don't have to.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dougborg dougborg force-pushed the spec-drift-verify-tooling branch from 4dd4a91 to e831397 Compare May 16, 2026 01:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants