Skip to content

Commit 919fe75

Browse files
NinjaRocksclaude
andcommitted
P0-4 verification: live-API smoke harness + Anthropic.SDK binding-gap docs
- Add tests/AnthropicVerification/ throwaway harness that hits the live Anthropic API for the three model IDs cited in Appendix B / Chapter 4.2.4 (claude-opus-4-7, claude-sonnet-4-6, claude-haiku-4-5-20251001). All three passed on 2026-05-02 with funded key. - Pin harness to Microsoft.Extensions.AI 10.3.0 via VersionOverride; required because Anthropic.SDK 5.10.0 is compiled against M.E.AI 10.3 and crashes at runtime against the central pin (10.5.0) with a MissingMethodException on HostedMcpServerTool.AuthorizationToken. Per-project override does NOT work on samples that depend on Microsoft.Agents.AI 1.3 (which itself requires M.E.AI 10.5 and trips CS1705) -- documented in the sample's README. - Record the 2026-05-02 verification pass and three remaining mitigation paths in docs/verification-log.md. - Add a Known issues section to the repo README and a callout to the 04.2.4-anthropic-agents README so readers aren't surprised by the runtime failure on the print-tagged code. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent a0ff438 commit 919fe75

6 files changed

Lines changed: 162 additions & 1 deletion

File tree

README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -135,6 +135,12 @@ If you intend to run any sample unattended (CI, scheduled jobs, scripts), set pe
135135

136136
CI runs on every push and pull request, plus a weekly Monday build to catch transitive package drift. The matrix builds every sample in the `samples/` tree against the pinned versions in `Directory.Packages.props`. A green badge above means the print-tagged code still compiles end-to-end. If the badge is red, check the latest [Actions run](https://github.com/CodeShayk/generative-ai-dotnet-samples/actions/workflows/ci.yml) for the breaking package or API change before assuming the samples are wrong.
137137

138+
CI catches build-level drift but not all runtime regressions. For the model-ID surface that ships in Appendix B, run the live-API smoke harness at [`tests/AnthropicVerification/`](tests/AnthropicVerification/) -- it iterates each cited Claude model ID against the live Anthropic API and reports `PASS`/`FAIL` per model. Each verification pass is recorded in [`docs/verification-log.md`](docs/verification-log.md).
139+
140+
## Known issues
141+
142+
- **`Anthropic.SDK` 5.10 ↔ `Microsoft.Extensions.AI` 10.5 binding gap.** `Anthropic.SDK` 5.10.0 is compiled against `Microsoft.Extensions.AI.Abstractions` 10.3.0; the central pin moved to 10.5.0 (required by `Microsoft.Agents.AI` 1.3), which reshapes `HostedMcpServerTool.AuthorizationToken`. Calling Claude through Anthropic.SDK's `IChatClient` bridge therefore throws `MissingMethodException` at runtime. This affects `samples/ch04-agent-framework/04.2.4-anthropic-agents` -- it builds clean but does not run as-shipped. The throwaway harness at `tests/AnthropicVerification/` works because it has no `Microsoft.Agents.AI` dependency and pins M.E.AI to 10.3.0 locally via `VersionOverride`; the same override does **not** work on the chapter sample because Agents.AI 1.3 itself rejects 10.3 with `CS1705`. Mitigation paths and pre-print decision are tracked in `docs/verification-log.md` (entry `2026-05-02`). Likely cleared once `Anthropic.SDK` 5.11+ ships rebuilt against M.E.AI 10.5+.
143+
138144
## Versioning and tags
139145

140146
| Tag | Meaning |

docs/verification-log.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,3 +91,45 @@ Copy this block to start a new entry. Date format `YYYY-MM-DD`.
9191
---
9292

9393
*New entries go below this line, most recent first.*
94+
95+
---
96+
97+
## 2026-05-02 -- P0-4 Anthropic live-API smoke
98+
99+
**Status:** Yellow -- model IDs and URLs verified green; one real defect (M.E.AI ↔ Anthropic.SDK runtime binding gap) blocks the existing Anthropic sample and must be cleared before print.
100+
101+
**Test runs in this entry:** initial pass blocked on zero credit balance; re-run later the same day with funded key returned all three model IDs `OK`.
102+
103+
### Packages
104+
- [x] `Anthropic.SDK` -- v5.10.0 (latest stable on NuGet); central pin unchanged.
105+
- [x] `Microsoft.Extensions.AI` -- v10.5.0 central pin; harness overrides to v10.3.0 to match Anthropic.SDK's compile-time target (see Issues).
106+
107+
### Code samples
108+
- [x] Verification harness at `tests/AnthropicVerification/` builds clean with M.E.AI 10.3.0 override.
109+
- [x] Live-API model-ID smoke through the harness -- **green for all three IDs** (`PASS claude-opus-4-7: OK`; `PASS claude-sonnet-4-6: OK`; `PASS claude-haiku-4-5-20251001: OK`).
110+
- [ ] Existing `samples/ch04-agent-framework/04.2.4-anthropic-agents` -- builds clean but **fails at runtime** with `MissingMethodException` from `Anthropic.SDK.Messaging.ChatClientHelper.CreateMessageParameters` -> `Microsoft.Extensions.AI.FunctionInvokingChatClient.GetResponseAsync` -> `Microsoft.Agents.AI.ChatClientAgent.RunCoreAsync`. Re-confirmed with funded key on 2026-05-02; the defect is independent of credits (see Issues).
111+
112+
### URLs
113+
- [x] `https://docs.anthropic.com/` (cited at `Manuscript/Chapter-04.md:3609`) -- HTTP 200.
114+
- [x] `https://platform.claude.com/docs/en/about-claude/models/overview` (cited at `Manuscript/Appendix-B-Model-Quick-Reference.md:36`) -- HTTP 200.
115+
- [x] Sanity: `https://platform.claude.com/`, `https://docs.anthropic.com/en/docs/about-claude/models/overview` -- both HTTP 200.
116+
- [ ] Microsoft Learn / Azure / NuGet links -- deferred; not in the P0-4 scope.
117+
118+
### Anthropic API surface
119+
- [x] Model IDs in `Manuscript/Appendix-B-Model-Quick-Reference.md:32-34` and `Manuscript/Chapter-04.md:557` (`claude-opus-4-7`, `claude-sonnet-4-6`, `claude-haiku-4-5-20251001`) -- all three callable; harness round-trip returned the expected `OK` response.
120+
- [x] `Anthropic.SDK` package surface used in chapter examples (`new AnthropicClient(key).Messages` as `IChatClient`) -- wire-compatible with the live API; full request/response round-trip confirmed when paired with M.E.AI 10.3.0.
121+
122+
### Issues found / actions taken
123+
- **Defect (blocks print): `samples/ch04-agent-framework/04.2.4-anthropic-agents` does not run against the repo's central pin.** Throws `MissingMethodException: Method not found: 'System.String Microsoft.Extensions.AI.HostedMcpServerTool.get_AuthorizationToken()'.` from `Anthropic.SDK.Messaging.ChatClientHelper.CreateMessageParameters`. Cause: `Anthropic.SDK` 5.10.0's nuspec declares `Microsoft.Extensions.AI.Abstractions >= 10.3.0` but the assembly is compiled against 10.3.0 specifically; M.E.AI 10.5.0 changed the shape of `HostedMcpServerTool.AuthorizationToken` (property-getter no longer present). CI is green because builds succeed against the *abstractions* package surface; runtime binding does not.
124+
- **Tried: per-project `VersionOverride` to 10.3.0 (initially proposed as option 3).** Verified to *not work* for this sample: `Microsoft.Agents.AI` 1.3.0 is itself compiled against `Microsoft.Extensions.AI.Abstractions` 10.5.0, so downgrading abstractions to 10.3.0 raises `CS1705 (uses higher version than referenced assembly)` at compile time. Override reverted; sample is back to clean build, broken runtime.
125+
- **The harness avoids the conflict only because it has no `Microsoft.Agents.AI` dependency** -- pure Anthropic.SDK + M.E.AI. Override pin 10.3.0 stays on the harness as the runtime mitigation there.
126+
- **Remaining viable paths (pre-print decision):**
127+
1. Wait for `Anthropic.SDK` 5.11+ rebuilt against M.E.AI 10.5+. Cleanest, but not on NuGet as of 2026-05-02 (latest stable: 5.10.0). Watch <https://www.nuget.org/packages/Anthropic.SDK>.
128+
2. Replace Anthropic.SDK's IChatClient bridge with a small custom `IChatClient` that calls `AnthropicClient.Messages.GetClaudeMessageAsync` directly. ~50 lines in the sample; bypasses the broken `ChatClientHelper`. Changes the chapter narrative slightly (the sample now ships a thin adapter rather than relying on the SDK's built-in adapter).
129+
3. Switch the sample to the OpenAI SDK pointed at Anthropic's OpenAI-compatibility endpoint. Strongest provider-neutral story, but rewrites the chapter section.
130+
- **CI gap:** the build matrix needs at least one runtime-execution lane for samples that have non-cloud-key prerequisites (currently 04.2.4 needs ANTHROPIC_API_KEY so cannot run unattended; consider adding a stub-key smoke that exercises the M.E.AI binding via `IChatClient` invocation up to the wire-send point).
131+
- Verification harness lives at `tests/AnthropicVerification/` (throwaway, not in `AI in .Net.sln`). Re-run: `ANTHROPIC_API_KEY=<key> dotnet run --project tests/AnthropicVerification`.
132+
133+
### Next-pass to-dos
134+
- [ ] Pick one of paths 1-3 above for the chapter sample. If path 1 (wait for SDK 5.11), open a tracking issue and re-check NuGet weekly. If path 2 (custom IChatClient), implement the bridge, re-run, confirm green. If path 3 (OpenAI-compat endpoint), revise the chapter prose for §4.2.4.
135+
- [ ] Run the broader URL audit (Microsoft Learn, Azure, NuGet) and fold into the next weekly entry.

samples/ch04-agent-framework/04.2.4-anthropic-agents/README.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,9 @@
22

33
Companion code for **Generative AI in .NET**, Chapter 4 section 4.2.4 ("Anthropic Agents").
44

5-
Wraps `Anthropic.SDK`'s message client as `IChatClient` via `.AsIChatClient(modelId)`, then hands it to a standard `ChatClientAgent`. The exact same agent API works against OpenAI, Azure OpenAI, Ollama, etc.
5+
Uses `new AnthropicClient(apiKey).Messages` -- which already implements `IChatClient` -- and hands it to a standard `ChatClientAgent`. The same agent API works against OpenAI, Azure OpenAI, Ollama, etc.
6+
7+
> **Known issue (2026-05-02): this sample does not run as-shipped against the repo's central package pins.** It builds clean, but at runtime `ChatClientAgent.RunAsync(...)` throws `MissingMethodException: Method not found: 'System.String Microsoft.Extensions.AI.HostedMcpServerTool.get_AuthorizationToken()'.`. Root cause: `Anthropic.SDK` 5.10.0 was compiled against `Microsoft.Extensions.AI.Abstractions` 10.3.0; the repo pins 10.5.0 (required by `Microsoft.Agents.AI` 1.3.0), which reshapes that property. Per-project `VersionOverride` to 10.3.0 does not help here -- Agents.AI 1.3 itself rejects 10.3 with `CS1705` at compile time. **Mitigation paths under evaluation** (see [`docs/verification-log.md`](../../../docs/verification-log.md), entry `2026-05-02`): (1) wait for `Anthropic.SDK` 5.11+ rebuilt against 10.5+, (2) ship a thin custom `IChatClient` adapter that calls `AnthropicClient.Messages.GetClaudeMessageAsync` directly, or (3) re-target the sample at Anthropic's OpenAI-compatibility endpoint via the OpenAI SDK. The chapter prose still describes the intended pattern; once the underlying packages reconcile, the sample runs unchanged.
68

79
## Run it
810

@@ -21,3 +23,7 @@ Override the model with `ANTHROPIC_MODEL` (defaults to `claude-haiku-4-5-2025100
2123
## Prerequisites
2224

2325
- .NET 9 SDK and an Anthropic API key.
26+
27+
## Verifying model IDs are still callable
28+
29+
A throwaway harness at [`tests/AnthropicVerification/`](../../../tests/AnthropicVerification/) hits the live API with each model ID listed in Appendix B. The harness has no `Microsoft.Agents.AI` dependency, so it can pin to M.E.AI 10.3.0 and run end-to-end -- use it during the pre-print verification passes to confirm Anthropic has not rotated the cited identifiers.
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
<Project Sdk="Microsoft.NET.Sdk">
2+
3+
<PropertyGroup>
4+
<OutputType>Exe</OutputType>
5+
<TargetFramework>net9.0</TargetFramework>
6+
<Nullable>enable</Nullable>
7+
<ImplicitUsings>enable</ImplicitUsings>
8+
<LangVersion>latest</LangVersion>
9+
<RootNamespace>AnthropicVerification</RootNamespace>
10+
<AssemblyName>AnthropicVerification</AssemblyName>
11+
</PropertyGroup>
12+
13+
<ItemGroup>
14+
<PackageReference Include="Anthropic.SDK" />
15+
<!--
16+
Anthropic.SDK 5.10.0 was compiled against Microsoft.Extensions.AI.Abstractions 10.3.0;
17+
the repo's central pin (10.5.0) reshapes HostedMcpServerTool.AuthorizationToken and
18+
causes a runtime MissingMethodException. Pin this throwaway harness to the SDK's
19+
compile-time targets via VersionOverride.
20+
-->
21+
<PackageReference Include="Microsoft.Extensions.AI" VersionOverride="10.3.0" />
22+
<PackageReference Include="Microsoft.Extensions.AI.Abstractions" VersionOverride="10.3.0" />
23+
</ItemGroup>
24+
25+
</Project>
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
using Anthropic.SDK;
2+
using Microsoft.Extensions.AI;
3+
4+
var apiKey = Environment.GetEnvironmentVariable("ANTHROPIC_API_KEY");
5+
if (string.IsNullOrWhiteSpace(apiKey))
6+
{
7+
Console.Error.WriteLine("ANTHROPIC_API_KEY not set.");
8+
return 1;
9+
}
10+
11+
string[] modelsToVerify =
12+
[
13+
"claude-opus-4-7",
14+
"claude-sonnet-4-6",
15+
"claude-haiku-4-5-20251001",
16+
];
17+
18+
IChatClient chat = new AnthropicClient(apiKey).Messages;
19+
20+
var any = false;
21+
foreach (var model in modelsToVerify)
22+
{
23+
try
24+
{
25+
var response = await chat.GetResponseAsync(
26+
"Reply with exactly: OK",
27+
new ChatOptions { ModelId = model });
28+
Console.WriteLine($"PASS {model}: {response.Text?.Trim()}");
29+
}
30+
catch (Exception ex)
31+
{
32+
any = true;
33+
Console.WriteLine($"FAIL {model}: {ex.GetType().Name}: {ex.Message}");
34+
}
35+
}
36+
37+
return any ? 2 : 0;
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Anthropic Live-API Smoke Harness
2+
3+
Throwaway test project used to verify, against the live Anthropic API, that the Claude model identifiers cited in the manuscript are still callable. Lives outside `AI in .Net.sln` on purpose -- it is not part of the shipped sample set.
4+
5+
## What it does
6+
7+
Iterates the three model IDs cited in `Manuscript/Appendix-B-Model-Quick-Reference.md` and `Manuscript/Chapter-04.md`, sends each one a fixed `"Reply with exactly: OK"` prompt via `IChatClient`, and prints `PASS`/`FAIL` per model. Exits 0 if all three respond, 2 if any fail, 1 if `ANTHROPIC_API_KEY` is unset.
8+
9+
## Why it pins `Microsoft.Extensions.AI` to 10.3.0
10+
11+
`Anthropic.SDK` 5.10.0 (latest stable at time of writing) is compiled against `Microsoft.Extensions.AI.Abstractions` 10.3.0. The repo's central pin is 10.5.0, which reshapes `HostedMcpServerTool.AuthorizationToken` -- runtime calls through the SDK's `IChatClient` bridge throw `MissingMethodException`. The harness sidesteps this by overriding the pin locally:
12+
13+
```xml
14+
<PackageReference Include="Microsoft.Extensions.AI" VersionOverride="10.3.0" />
15+
<PackageReference Include="Microsoft.Extensions.AI.Abstractions" VersionOverride="10.3.0" />
16+
```
17+
18+
The override works here because the harness has no `Microsoft.Agents.AI` dependency. The chapter sample at `samples/ch04-agent-framework/04.2.4-anthropic-agents/` cannot use the same override (Agents.AI 1.3 is itself compiled against M.E.AI 10.5 and refuses to bind to 10.3 with `CS1705`); see that sample's README and `docs/verification-log.md` for the workaround status.
19+
20+
When `Anthropic.SDK` 5.11+ ships rebuilt against M.E.AI 10.5+, drop both overrides.
21+
22+
## Run it
23+
24+
```bash
25+
cd code/generative-ai-dotnet-samples
26+
ANTHROPIC_API_KEY='sk-ant-...' dotnet run --project tests/AnthropicVerification
27+
```
28+
29+
Expected output:
30+
31+
```
32+
PASS claude-opus-4-7: OK
33+
PASS claude-sonnet-4-6: OK
34+
PASS claude-haiku-4-5-20251001: OK
35+
```
36+
37+
If a model returns `FAIL`, update `Manuscript/Appendix-B-Model-Quick-Reference.md` to the current Anthropic-published identifier and re-run.
38+
39+
## When to re-run
40+
41+
- During the four pre-print weekly verification passes.
42+
- Whenever Anthropic announces a new Claude release (model IDs include date suffixes that may roll forward).
43+
- After bumping `Anthropic.SDK` or `Microsoft.Extensions.AI*` versions.
44+
45+
Record results in `docs/verification-log.md`.

0 commit comments

Comments
 (0)