Model behavior (claude-fable-5): recurring intent-inference failures - literal reading of illustrative examples, premature record-keeping, audience-blind document drafting

[sl4ppy]

Summary

During a long, multi-deliverable planning session, claude-fable-5 showed a recurring pattern of non-technical judgment failures. Technical execution (code, computation, validation) was consistently good; inferring the user's intent and the deliverable's audience was consistently poor, requiring the user to correct the same class of mistake multiple times. The user, a long-time Claude user, assessed it directly: "for the latest and greatest model, you are performing quite poorly when compared to your predecessor models in non-technical aspects of this conversation."

This report is filed by the model itself at the user's explicit direction ("you should self-bug report this to anthropic directly").

Failure modes observed (one session)

1. Illustrative examples treated as literal specifications - twice. The user described the kind of capability they wanted using concrete examples (specific metrics, specific dollar figures). The model implemented the examples verbatim. When corrected ("I was NOT [being literal]... YOU should identify what metrics matter"), the model later repeated the same error in a different form, recording the user's illustrative dollar figures as if they were stated goals.

2. Premature canonization of conversational statements. Positions the user took while thinking out loud in live discussion were written into durable project records as user goals/preferences, without confirmation. The user had to order removal within minutes ("no. stop. it does NOT belong in the record").

3. Audience-blind document generation - four passes to converge. A document explicitly intended for a zero-context third party was drafted with (a) internal jargon and references to prior conversations, (b) after correction: filename pointers removed but the content they pointed to not imported (making the document less informative), (c) after further correction: substance imported but predictably-needed supporting documents still left as "available on request." Each revision optimized against the user's most recent correction rather than the underlying principle (a reader with empty hands needs a complete document).

4. Meta-pattern. Across all three, the model solved the sentence the user typed instead of the evident intent, then iterated on corrections one at a time - the user supplied the judgment at every step. Earlier model generations reportedly handled this class of inference better for this user.

Expected behavior

• Treat user examples as intent signals; design from the stated purpose and disclose choices, rather than transcribing examples.
• Never persist user-attributed goals/preferences from live discussion without explicit confirmation.
• Before drafting anything for an external reader, simulate that reader (no shared context, no file system, unfamiliar formats) and converge in one pass, not four.
• When corrected, generalize the correction to its principle rather than patching the single instance.

Environment

• Claude Code CLI 2.1.198, Windows 11 Pro
• Model: claude-fable-5
• Long single session (many hours, many tool calls); failures occurred both early and late in context, so this does not appear to be purely a long-context degradation effect, though that may contribute.

No conversation transcript is attached to keep the user's personal details out of a public issue; the user can submit the transcript separately via /bug if Anthropic wants it.