Consider caching JSON schema generation in function calling

[Seol-JY]

Current Performance Issue

Function calling applications regenerate identical JSON schemas on every request, causing significant CPU overhead.

Root cause analysis:

// User code pattern
chatClient.prompt("What's the weather?")
    .tools(new WeatherTools())  // New instance every time
    .call();

// Execution chain:
// 1. ToolCallbacks.from(new WeatherTools())
// 2. MethodToolCallbackProvider.getToolCallbacks() - no caching
// 3. ToolDefinitions.from(toolMethod) - called for each @Tool method
// 4. JsonSchemaGenerator.generateForMethodInput(method) - CPU intensive

Evidence from source code:

MethodToolCallbackProvider.getToolCallbacks() creates new tool definitions every time:

public ToolCallback[] getToolCallbacks() {
    return this.toolObjects.stream()
        .map(toolMethod -> MethodToolCallback.builder()
            .toolDefinition(ToolDefinitions.from(toolMethod))  // No caching here
            .build())
        .toArray(ToolCallback[]::new);
}

ToolDefinitions.from() calls schema generation without caching:

public static ToolDefinition from(Method method) {
    return builder(method).build();  // Always creates new
}

public static DefaultToolDefinition.Builder builder(Method method) {
    return DefaultToolDefinition.builder()
        .inputSchema(JsonSchemaGenerator.generateForMethodInput(method));  // Expensive call
}

JsonSchemaGenerator.generateForMethodInput() performs expensive operations every time:

public static String generateForMethodInput(Method method, SchemaOption... schemaOptions) {
    // Complex schema generation with reflection
    for (int i = 0; i < method.getParameterCount(); i++) {
        ObjectNode parameterNode = SUBTYPE_SCHEMA_GENERATOR.generateSchema(parameterType);  // CPU intensive
    }
    return schema.toPrettyString();  // JSON serialization
}

Performance Impact

Typical scenario:

@RestController 
public class ChatController {
    @PostMapping("/chat")
    public String chat(@RequestBody String message) {
        // Same schemas regenerated on every HTTP request
        return chatClient.prompt(message)
            .tools(new WeatherTools())  // 2 @Tool methods = 2 schema generations
            .call().content();
    }
}

Current behavior creates repeated work:

• Same method signatures generate identical schemas multiple times
• No reuse of schema generation results across requests
• Optimization opportunity exists for applications with repeated tool usage

[ilayaperumalg]

@Seol-JY Thanks for reporting the issue and the interest in contributing . Please submit a PR with your proposal. A small write up on your proposed changes can help review from the beginning. Thanks!