temporal-spring-ai: pass provider-specific ChatOptions through the activity boundary#2857
Open
donald-pinckney wants to merge 4 commits intomasterfrom
Open
temporal-spring-ai: pass provider-specific ChatOptions through the activity boundary#2857donald-pinckney wants to merge 4 commits intomasterfrom
donald-pinckney wants to merge 4 commits intomasterfrom
Conversation
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Option 2 (serialize the full ChatOptions subclass as JSON + class name,
rehydrate on the activity side) is strictly better than the opaque map.
Full fidelity, no per-provider allow-list, no builder gymnastics.
The earlier rationale for option 1 ("drags provider classes into the
workflow's classpath") was wrong: workflow and activity workers can be
separate JVMs, but the user's workflow can only construct
OpenAiChatOptions if spring-ai-openai is already on the workflow
worker's classpath, and the activity worker has the same class because
that's where the ChatModel bean runs. The precondition holds whenever
the feature is actually used.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tivity boundary Serialize the caller's ChatOptions as (class name, JSON) on the workflow side and rehydrate the exact subclass on the activity side. Every field survives — including provider-specific ones like OpenAI reasoning_effort or Anthropic thinking-budget settings — without the plugin needing per-provider allow-lists, reflection into builder types, or compile deps on spring-ai-openai/anthropic/etc. Changes: - ChatModelTypes.ModelOptions: two new nullable fields, chatOptionsClass and chatOptionsJson. Older callers that only set the common scalar fields still work via a convenience constructor. - ActivityChatModel: serialize the caller's ChatOptions with a plain ObjectMapper when non-null; a Jackson mixin skips the tool-callback bag (toolCallbacks/toolNames/toolContext) since those cross the boundary via the separate `tools` field. Serialization failure is logged at debug and leaves the blob fields null — activity side then uses the common-field path. - ChatModelActivityImpl: primary path rehydrates from the serialized blob and, if the result is a ToolCallingChatOptions, re-attaches the stub tool callbacks plus forces internalToolExecutionEnabled=false. Fallback path (no blob, class-not-found, deser error) builds a plain ToolCallingChatOptions from the common scalar fields, identical to the prior behavior. Tests: - ProviderOptionsPassthroughTest.customChatOptionsSubclass... — a test-local CustomChatOptions extending DefaultToolCallingChatOptions with an extra reasoningEffort field round-trips, and common fields (temperature/maxTokens) come through the same path. - ProviderOptionsPassthroughTest.nullChatOptions... — workflow that doesn't set any options still works via the fallback path. The workflow calls ActivityChatModel.call(new Prompt(...)) directly instead of going through ChatClient, because ChatClient.defaultOptions coerces the caller's subclass into the client's internal options type before ActivityChatModel ever sees it. Users of ChatClient who want provider-specific options should set those on the ChatModel bean's default options (Spring AI merges those into every call). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
donald-pinckney
force-pushed
the
spring-ai/provider-options-passthrough
branch
from
96d9bb0 to
4df96ec
Compare
Planning scratchpad — not part of the shipped artifact. Removed before merge. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was changed
ChatModelTypes.ModelOptionsgets two new nullable fields,chatOptionsClassandchatOptionsJson, representing the caller's fullChatOptionsas(class name, JSON). An overload constructor keeps older call sites (tests + the fallback path) building the record with only scalar fields.ActivityChatModelserializes the caller'sChatOptionswith a plainObjectMapperwhen non-null. A Jackson mixin skips the tool-callback bag (toolCallbacks/toolNames/toolContext) since those cross the activity boundary via the separatetoolsfield. On serialization failure the blob fields stay null; the activity falls back to the common-field path with a debug log.ChatModelActivityImplprefers the serialized blob when both fields are present. If the rehydrated object is aToolCallingChatOptions, it re-attaches the stubbed tool callbacks and forcesinternalToolExecutionEnabled = false. If class loading or deserialization fails, it falls back to the existing common-field builder path with a warning log.ProviderOptionsPassthroughTestexercises three paths — directActivityChatModel.call(...)with a customChatOptionssubclass, the idiomaticChatClient.defaultOptions(...)entry point with the same subclass, and the null-options fallback.Why?
The previous implementation limited users to the common Spring AI
ChatOptionssubset. Anyone needing OpenAIreasoning_effort, Anthropic thinking budget, structured-output schemas, or any other provider-specific knob had to fork the plugin. With this change every field of the caller's exact subclass survives the round trip.Design note — why the serialized blob beats an opaque
Map<String, Object>: workflow and activity workers are typically separate JVMs (Temporal's whole point), so "classpath" really can differ. But the precondition for this feature to be used at all is that the user's workflow constructs a provider-specificChatOptions— which requiresspring-ai-<provider>on the workflow worker's classpath — and the activity worker has the same class because that's where theChatModelbean runs. Under that precondition,Class.forName + ObjectMapper.readValueworks everywhere the feature is actually used, with full fidelity and no per-provider allow-list to maintain.About
ChatOptions.copy(): Spring AI'sChatClientcallschatOptions.copy()on the caller's options before passing to the model. Real provider classes (OpenAiChatOptions,AnthropicChatOptions, ...) all overridecopy()to return their own type with every field carried across. AnyChatOptionssubclass that follows the same pattern — including user-defined ones like the test'sCustomChatOptions— works through theChatClientpath as well as directchatModel.call(new Prompt(...)). A subclass that inherits the parent'scopy()would lose its subclass identity before this plugin ever sees it, regardless of anything this PR does.