Skip to content

fix: only treat call-signature and type changes as breaking in release metadata#32

Merged
gjtorikian merged 5 commits into
mainfrom
fix/breaking-change-classification
Jun 22, 2026
Merged

fix: only treat call-signature and type changes as breaking in release metadata#32
gjtorikian merged 5 commits into
mainfrom
fix/breaking-change-classification

Conversation

@gjtorikian

Copy link
Copy Markdown
Collaborator

Summary

scripts/sdk-release-metadata.mjs over-classified changes as breaking, producing feat! (major-bump) commits for what are really backend API changes. It inherited the spec differ's classification for field / enum-value / response / request changes, and trusted every breaking-severity compat symbol — including the model properties the compat snapshot tracks (~2000 of them).

This tightens the rule so a change is breaking (feat!) only when:

  • the call signature a user invokes changes — parameters / constructor (parameter_removed, parameter_renamed, parameter_requiredness_increased, parameter_type_narrowed, signature, operation param changes), or
  • a whole type is removed/renamed — model / enum / service, or a method is removed/renamed.

Everything else becomes a non-breaking backend API change (fix):

  • a field renamed or removed, removed response data, field type/format/required changes
  • an enum value removed
  • a response/request-body type renamed

How

  • Spec-diff path: a capSeverity helper downgrades would-be-breaking field-*, value-*, response-changed, and request-body-changed facts to fix. The made-required/optional changelog wording, which previously rode on severity === 'breaking', now carries an explicit madeRequired flag on the fact.
  • Compat path: compatChangeIsBreaking skips backend-only categories (field_type_changed, enum_member_value_changed, return_type_changed, default_value_changed) and, for symbol_removed/symbol_renamed, distinguishes a model property (field → not breaking) from a service/client method or whole type (→ breaking) by checking the symbol's owner against the IR's service/type names (now exposed from buildIndexes).

By this same rule the tooling change itself is not breaking → fix:, no !.

Test plan

Verified with synthetic and real fixtures (no automated test exists for this script):

  • Mixed spec diff → model-removed + operation-removed render under ⚠️ Breaking; field-removed, made-required, value-removed, response-changed render under Fixes.
  • Field + response changes only → no feat!; rollup is fix.
  • Real .oagen/ruby/compat-report.jsonencryptor parameter removal and SSO / UserManagement method removals stay breaking.
  • Property-vs-method discriminator → ActionAuthenticationDenied.context (property) and a field_type_changed are dropped; SSO.get_authorization_url (method) stays breaking.
  • --format json output keeps correct prefixes; madeRequired does not leak into the emitted changes[].

🤖 Generated with Claude Code

gjtorikian and others added 2 commits June 17, 2026 17:30
Two improvements to the changelog produced by sdk-release-metadata.mjs,
both aimed at the local "--spec-commit --sdk-repo --format changelog" flow:

1. Auto-derive the SDK compat report from the checkout when --compat-report
   is omitted. It snapshots the current tree (candidate) and the tree at the
   base ref (baseline, via a throwaway git worktree) and diffs them — no SDK
   regeneration. This is what surfaces SDK-surface changes like method
   renames, which the spec diff alone cannot see. Fails open (spec-diff-only
   changelog) and is skipped when --compat-report is passed, so CI is
   unaffected. Language is inferred from the repo basename (workos-python →
   python) or set via --lang.

2. Name the types in operation response/request-body changes: instead of
   "Changed response for `X.list`", emit "Changed response of `X.list` from
   `UserOrganizationMembership` to `UserOrganizationMembershipList`", read
   from the old/new IR (the diff report only carries a boolean). Falls back
   to the generic wording when a type name isn't resolvable.

Verified end-to-end against workos-python PR #670 (rename + enriched response
lines) and across all per-language compat reports under --strict-scopes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e metadata

sdk-release-metadata.mjs marked field, enum-value, and response/request-shape
changes as breaking — it inherited the spec differ's classification and trusted
every breaking-severity compat symbol, including the model properties the compat
snapshot tracks. That emitted feat! (major-bump) commits for what are really
backend API changes.

Cap them so only a changed call signature (parameters, constructor) or a
removed/renamed type (model/enum/service) or removed method stays breaking. Field
renames/removals, removed response data, dropped enum values, and response/request
body type renames now land under Fixes. The compat path tells a model property
(field — not breaking) apart from a service method using the IR's service/type
names, so SDK-surface method removals/renames stay breaking.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Jun 18, 2026

Copy link
Copy Markdown

Greptile Summary

This PR tightens the breaking-change classification in scripts/sdk-release-metadata.mjs so that only genuine call-signature changes (parameter additions/removals/renames, operation param changes, whole-type/service removals or renames) produce feat! major-bump commits, while backend API shape changes (field renames/removals, response/request body changes, enum value changes) are downgraded to fix.

  • Spec-diff path: a new capSeverity helper downgrades field-*, value-removed/modified, response-changed, and request-body-changed facts from breaking to fix; the madeRequired flag is captured before capping so changelog wording stays accurate; mixed required/optional groups now fall through to the generic summary instead of mis-reporting one direction.
  • Compat path: compatChangeIsBreaking filters out backend-only categories and distinguishes model properties (not breaking) from service/client method removals (breaking) by consulting IR service- and type-name sets exposed from buildIndexes; the workflow's raw COMPAT_BREAKING rollup override block is removed and replaced by the filtered metadata script output.
  • Non-CI convenience: new prepareCompatInputs derives a compat report from an existing SDK checkout via a throwaway git worktree, and correctly passes the pinned historical spec path (_specPath) from prepareSpecInputs so manual changelog repairs use the right spec version.

Confidence Score: 4/5

Safe to merge for the intended goal; the classification tightening is well-structured, fails open everywhere, and the workflow rollup is now correctly driven by the filtered metadata output.

The core logic — capSeverity, compatChangeIsBreaking, and the workflow override removal — is sound and correctly implemented with conservative fallbacks. The one notable gap is that COMPAT_BREAKING is still read back in the compose step after its only consumer (the rollup override block) was removed, leaving dead code that could confuse future maintainers. No correctness or version-bump regression is introduced by the PR itself.

.github/workflows/generate-prs.yml — the COMPAT_BREAKING variable (and its /tmp/classify-compat-breaking.json persistence) is now dead after the rollup-override removal and can be cleaned up.

Important Files Changed

Filename Overview
scripts/sdk-release-metadata.mjs Core classification logic refactored: adds capSeverity for spec-diff path, compatChangeIsBreaking for compat path, madeRequired flag, and a new prepareCompatInputs function; logic is well-structured with conservative fallbacks
.github/workflows/generate-prs.yml Removes the raw COMPAT_BREAKING override block that was forcing major bumps; adds a warning-emitting fallback for unrecognized diff kinds; COMPAT_BREAKING is now dead in the compose step

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[oagen diff report] --> B[factsFromDiff]
    B --> C{capSeverity}
    C -->|field-*, value-removed, value-modified\nresponse-changed, request-body-changed\n+ classification==breaking| D[severity → fix]
    C -->|All other kinds\nor non-breaking classification| E[severity unchanged]

    F[compat report] --> G{Source?}
    G -->|--compat-report provided CI path| H[factsFromCompat]
    G -->|--sdk-repo without --compat-report non-CI path| I[prepareCompatInputs derives report via git worktree]
    I --> H

    H --> J{compatChangeIsBreaking?}
    J -->|field_type_changed, enum_member_value_changed\nreturn_type_changed, default_value_changed| K[drop — non-breaking]
    J -->|symbol_removed/renamed: owner in typeNames| L[drop — model property]
    J -->|symbol_removed/renamed: owner in serviceNames or Client| M[keep — breaking]
    J -->|all other categories| M

    D --> N[groupFacts]
    E --> N
    M --> N
    N --> O[entriesFromGroups]
    O -->|has feat! entries| P[ROLLUP_TYPE=feat ROLLUP_BANG=!]
    O -->|has feat entries| Q[ROLLUP_TYPE=feat]
    O -->|fix only| R[ROLLUP_TYPE=fix]
    O -->|zero entries| S{oagen diff breaking > 0?}
    S -->|yes| T[warn + feat! conservative bump]
    S -->|no| U[feat or fix from diff counts]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[oagen diff report] --> B[factsFromDiff]
    B --> C{capSeverity}
    C -->|field-*, value-removed, value-modified\nresponse-changed, request-body-changed\n+ classification==breaking| D[severity → fix]
    C -->|All other kinds\nor non-breaking classification| E[severity unchanged]

    F[compat report] --> G{Source?}
    G -->|--compat-report provided CI path| H[factsFromCompat]
    G -->|--sdk-repo without --compat-report non-CI path| I[prepareCompatInputs derives report via git worktree]
    I --> H

    H --> J{compatChangeIsBreaking?}
    J -->|field_type_changed, enum_member_value_changed\nreturn_type_changed, default_value_changed| K[drop — non-breaking]
    J -->|symbol_removed/renamed: owner in typeNames| L[drop — model property]
    J -->|symbol_removed/renamed: owner in serviceNames or Client| M[keep — breaking]
    J -->|all other categories| M

    D --> N[groupFacts]
    E --> N
    M --> N
    N --> O[entriesFromGroups]
    O -->|has feat! entries| P[ROLLUP_TYPE=feat ROLLUP_BANG=!]
    O -->|has feat entries| Q[ROLLUP_TYPE=feat]
    O -->|fix only| R[ROLLUP_TYPE=fix]
    O -->|zero entries| S{oagen diff breaking > 0?}
    S -->|yes| T[warn + feat! conservative bump]
    S -->|no| U[feat or fix from diff counts]
Loading

Reviews (4): Last reviewed commit: "chore: warn when the rollup fallback bum..." | Re-trigger Greptile

Comment thread scripts/sdk-release-metadata.mjs Outdated
Comment thread scripts/sdk-release-metadata.mjs
Comment thread scripts/sdk-release-metadata.mjs
Comment thread scripts/sdk-release-metadata.mjs
devin-ai-integration Bot and others added 3 commits June 21, 2026 13:53
The 'Compose PR metadata' step was re-reading the unfiltered
COMPAT_BREAKING array and forcing ROLLUP_TYPE=feat / ROLLUP_BANG=!
whenever any raw severity=="breaking" compat row existed.  This
bypassed the compatChangeIsBreaking filter applied by the metadata
script, causing backend-only compat changes (field_type_changed,
return_type_changed, model-property symbol_removed on non-service
types) to produce unwarranted major-bump PRs.

The metadata script already emits feat! entries for genuine compat
breaks via factsFromCompat, and lines 297-306 promote the rollup
based on those filtered entries.  The raw override was redundant and
harmful — remove it so the filtered result is the single source of
truth for the rollup decision.

Co-Authored-By: bot_apk <apk@cognition.ai>
- field-added: add to BACKEND_ONLY_DIFF_KINDS so a breaking-classified
  required field addition is capped to `fix` instead of riding through as
  `feat!`, matching the stated policy that field shape changes are never
  breaking.
- mixed requiredness: only collapse a field-required-changed group to a
  single "required"/"optional" line when every fact moves the same way;
  mixed directions now fall through to the generic summary / per-fact
  details rather than misreporting one direction.
- historical compat: prepareSpecInputs now exposes the --spec-commit spec
  snapshot as _specPath, and prepareCompatInputs prefers it, so a
  historical changelog repair extracts compat against that commit's spec
  instead of the current checkout. CI (which passes --compat-report) is
  unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The fallback at the end of the classify step only runs when the metadata
script produced zero entries and there are no behavior changes. If oagen
diff still reports breaking changes in that state, it means factsFromDiff
doesn't recognize the diff kind. Keep the conservative feat! default
(over-bumping is safer than shipping an unclassified break as a patch),
but emit a CI warning so the unhandled kind is investigated instead of
silently producing a vague major-bump PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gjtorikian gjtorikian merged commit 9d653a7 into main Jun 22, 2026
5 checks passed
@gjtorikian gjtorikian deleted the fix/breaking-change-classification branch June 22, 2026 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant