Skip to content

Commit 2bc964f

Browse files
lirenheCopilot
andcommitted
Add evaluation test case for adding optional body to ARM action
Add new test case covering how to add an optional body parameter to an existing ARM TypeSpec action operation in a new API version. Includes evaluation results in typespec-test.json. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 912f823 commit 2bc964f

2 files changed

Lines changed: 73 additions & 1 deletion

File tree

tools/sdk-ai-bots/azure-sdk-qa-bot-evaluation/results/typespec-test.json

Lines changed: 72 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1757,9 +1757,80 @@
17571757
"knowledge_match_result": "fail",
17581758
"overall_score": 7.1
17591759
},
1760+
{
1761+
"testcase": "Adding optional body parameter to existing ARM action operation",
1762+
"expected": {
1763+
"answer": "Yes, adding an optional body is the compatible way to evolve this. Don't change the existing stable version in place — add a new API version and introduce the body there.\n\nKeep the same action/operationId, define a request model, and set OptionalRequestBody = true:\nmodel ApplyMaintenanceWindowRequest {\n durationInSeconds?: int32;\n force?: boolean;\n}\n@action(\"applyMaintenanceWindow\")\n@operationId(\"managedApplyMaintenanceWindow_Post\")\npost is ArmResourceActionSync<\n ManagedClusterResource,\n ApplyMaintenanceWindowRequest,\n ArmResponse<ApplyMaintenanceWindowResult>,\n OptionalRequestBody = true\n>;\n\nIf the action returns no body, use the no-content POST template instead.",
1764+
"references": [
1765+
{
1766+
"title": "Adding a Stable Version when the Last Version was Stable | Making Changes to your TypeSpec spec",
1767+
"link": "https://azure.github.io/typespec-azure/docs/howtos/versioning/arm/05-stable-after-stable"
1768+
},
1769+
{
1770+
"title": "ARM ResourceManager operation templates reference",
1771+
"link": "https://azure.github.io/typespec-azure/docs/libraries/azure-resource-manager/reference/interfaces"
1772+
},
1773+
{
1774+
"title": "TypeSpec versioning",
1775+
"link": "https://azure.github.io/typespec-azure/docs/getstarted/versioning"
1776+
},
1777+
{
1778+
"title": "Versioning | Simple Scenarios | Adding a Parameter to an Operation and Making Another Parameter Optional",
1779+
"link": "https://azure.github.io/typespec-azure/docs/howtos/arm/versioning"
1780+
},
1781+
{
1782+
"title": "Adding a stable version after a stable version",
1783+
"link": "https://azure.github.io/typespec-azure/docs/howtos/versioning/arm/05-stable-after-stable"
1784+
},
1785+
{
1786+
"title": "Breaking-change FAQ for migration/versioning details",
1787+
"link": "https://azure.github.io/typespec-azure/docs/migrate-swagger/faq/breakingchange"
1788+
},
1789+
{
1790+
"title": "ARM resource actions how-to",
1791+
"link": "https://azure.github.io/typespec-azure/docs/howtos/arm/resource-operations"
1792+
}
1793+
]
1794+
},
1795+
"actual": {
1796+
"answer": "Model the change as a versioned operation: keep the old no-body action for old versions and introduce a new action with an optional body for the new version, sharing the same route.\n\nAssuming you already have a `Versions` enum and `@versioned(Versions)` on your namespace, you can do:\n\n```typespec\n@armResourceOperations\ninterface ManagedResources {\n // Old operation (no body) – present in old versions only\n @renamedFrom(Versions.`2025-10-01-preview`, \"post\")\n @removed(Versions.`2025-10-01-preview`)\n @sharedRoute\n postNoBody is ArmResourceActionSync;\n\n // New operation – available in the new version, with optional body\n @sharedRoute\n post is ArmResourceActionSync<\n /* Resource */ ManagedResource,\n /* Request */ ApplyMaintenanceWindowRequest?, // optional body\n /* Response */ ApplyMaintenanceWindowResponse\n >;\n}\n```\n\nKey points:\n- Use `@sharedRoute` on both operations so they share the same path and method.\n- Use `@renamedFrom` and `@removed` on the old op so it only appears in older api-versions.\n- Define the new `post` as `ArmResourceActionSync<Resource, Request?, Response>` with the request model marked optional (`?`).\n\nAdjust `Versions.<...>`, `ManagedResource`, and the request/response models to match your RP. Keep `@action(\"applyMaintenanceWindow\")` and `@operationId(\"managedApplyMaintenanceWindow_Post\")` on the new `post` if those are still correct for the latest version.",
1797+
"references": [
1798+
{
1799+
"title": "Interfaces and Operations | Azure.ResourceManager | `ArmResourceActionSync`",
1800+
"link": "https://azure.github.io/typespec-azure/docs/libraries/azure-resource-manager/reference/interfaces"
1801+
},
1802+
{
1803+
"title": "Versioning",
1804+
"link": "https://azure.github.io/typespec-azure/docs/howtos/arm/versioning"
1805+
},
1806+
{
1807+
"title": "Versioning the LRO headers | answer",
1808+
"link": ""
1809+
}
1810+
]
1811+
},
1812+
"similarity": -1.0,
1813+
"gpt_similarity": -1.0,
1814+
"similarity_result": "N/A",
1815+
"similarity_threshold": 4,
1816+
"groundedness": -1.0,
1817+
"gpt_groundedness": -1.0,
1818+
"groundedness_reason": "N/A",
1819+
"groundedness_result": "N/A",
1820+
"groundedness_threshold": 3.0,
1821+
"response_completeness": -1.0,
1822+
"response_completeness_result": "N/A",
1823+
"response_completeness_threshold": 4,
1824+
"response_completeness_reason": "N/A",
1825+
"reference_match": 0.33,
1826+
"reference_match_result": "fail",
1827+
"knowledge_match": 0.0,
1828+
"knowledge_match_result": "fail",
1829+
"overall_score": 0.33
1830+
},
17601831
{
17611832
"average_score": NaN,
1762-
"total_evals": 34,
1833+
"total_evals": 35,
17631834
"similarity_pass_rate": 34,
17641835
"groundedness_pass_rate": 23,
17651836
"response_completeness_pass_rate": 34,

0 commit comments

Comments
 (0)