Error in Spring AI 2.0.0-M6 while using Docker Model runner

[eazybytes]

Bug description
Getting below error when integrating Spring AI with Docker Model runner based LLM model. The same code works fine with Spring AI 1.x version

20:30:05.156 ERROR [http-nio-8080-exec-2] o.a.c.c.C.[.[.[.[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: com.openai.errors.NotFoundException: 404: null] with root cause
com.openai.errors.NotFoundException: 404: null
at com.openai.errors.NotFoundException$Builder.build(NotFoundException.kt:88) ~[openai-java-core-4.34.0.jar:4.34.0]
at com.openai.core.handlers.ErrorHandler$errorHandler$1.handle(ErrorHandler.kt:63) ~[openai-java-core-4.34.0.jar:4.34.0]
at com.openai.core.handlers.ErrorHandler$errorHandler$1.handle(ErrorHandler.kt:40) ~[openai-java-core-4.34.0.jar:4.34.0]
at com.openai.services.blocking.chat.ChatCompletionServiceImpl$WithRawResponseImpl.create(ChatCompletionServiceImpl.kt:141) ~[openai-java-core-4.34.0.jar:4.34.0]
at com.openai.services.blocking.chat.ChatCompletionServiceImpl.create(ChatCompletionServiceImpl.kt:63) ~[openai-java-core-4.34.0.jar:4.34.0]
at com.openai.services.blocking.chat.ChatCompletionService.create(ChatCompletionService.kt:64) ~[openai-java-core-4.34.0.jar:4.34.0]
at org.springframework.ai.openai.OpenAiChatModel.lambda$internalCall$4(OpenAiChatModel.java:209) ~[spring-ai-openai-2.0.0-M6.jar:2.0.0-M6]
at io.micrometer.observation.Observation.observe(Observation.java:634) ~[micrometer-observation-1.16.5.jar:1.16.5]
at org.springframework.ai.openai.OpenAiChatModel.internalCall(OpenAiChatModel.java:207) ~[spring-ai-openai-2.0.0-M6.jar:2.0.0-M6]
at org.springframework.ai.openai.OpenAiChatModel.call(OpenAiChatModel.java:186) ~[spring-ai-openai-2.0.0-M6.jar:2.0.0-M6]
at org.springframework.ai.chat.client.advisor.ChatModelCallAdvisor.adviseCall(ChatModelCallAdvisor.java:58) ~[spring-ai-client-chat-2.0.0-M6.jar:2.0.0-M6]
at org.springframework.ai.chat.client.advisor.DefaultAroundAdvisorChain.lambda$nextCall$1(DefaultAroundAdvisorChain.java:115) ~[spring-ai-client-chat-2.0.0-M6.jar:2.0.0-M6]
at io.micrometer.observation.Observation.observe(Observation.java:634) ~[micrometer-observation-1.16.5.jar:1.16.5]
at org.springframework.ai.chat.client.advisor.DefaultAroundAdvisorChain.nextCall(DefaultAroundAdvisorChain.java:114) ~[spring-ai-client-chat-2.0.0-M6.jar:2.0.0-M6]
at org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.lambda$doGetObservableChatClientResponse$1(DefaultChatClient.java:524) ~[spring-ai-client-chat-2.0.0-M6.jar:2.0.0-M6]
at io.micrometer.observation.Observation.observe(Observation.java:634) ~[micrometer-observation-1.16.5.jar:1.16.5]
at org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.doGetObservableChatClientResponse(DefaultChatClient.java:522) ~[spring-ai-client-chat-2.0.0-M6.jar:2.0.0-M6]
at org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.content(DefaultChatClient.java:502) ~[spring-ai-client-chat-2.0.0-M6.jar:2.0.0-M6]
at com.eazybytes.dockermodel.controller.ChatController.chat(ChatController.java:22) ~[classes/:na]
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:565) ~[na:na]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:252) ~[spring-web-7.0.7.jar:7.0.7]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:184) ~[spring-web-7.0.7.jar:7.0.7]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) ~[spring-webmvc-7.0.7.jar:7.0.7]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:934) ~[spring-webmvc-7.0.7.jar:7.0.7]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:853) ~[spring-webmvc-7.0.7.jar:7.0.7]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:86) ~[spring-webmvc-7.0.7.jar:7.0.7]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) ~[spring-webmvc-7.0.7.jar:7.0.7]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:866) ~[spring-webmvc-7.0.7.jar:7.0.7]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1000) ~[spring-webmvc-7.0.7.jar:7.0.7]
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:892) ~[spring-webmvc-7.0.7.jar:7.0.7]
at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:622) ~[tomcat-embed-core-11.0.21.jar:6.1]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:874) ~[spring-webmvc-7.0.7.jar:7.0.7]
at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:710) ~[tomcat-embed-core-11.0.21.jar:6.1]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:128) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) ~[tomcat-embed-websocket-11.0.21.jar:11.0.21]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:107) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-7.0.7.jar:7.0.7]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-7.0.7.jar:7.0.7]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:107) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-7.0.7.jar:7.0.7]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-7.0.7.jar:7.0.7]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:107) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:199) ~[spring-web-7.0.7.jar:7.0.7]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-7.0.7.jar:7.0.7]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:107) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:165) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:77) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:113) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:83) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:72) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:341) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:397) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:903) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1801) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:946) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:480) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:57) ~[tomcat-embed-core-11.0.21.jar:11.0.21]
at java.base/java.lang.Thread.run(Thread.java:1474) ~[na:na]

Environment
Spring AI version - 2.0.0-M6, Java version - 25, LLM model - gemma4

Steps to reproduce

@RestController
@RequestMapping("/api")
public class ChatController {

private final ChatClient chatClient;

public ChatController(ChatClient.Builder chatClientBuilder) {
    this.chatClient = chatClientBuilder.build();
}


@GetMapping("/chat")
public String chat(@RequestParam("message") String message) {
    return chatClient.prompt(message).call().content();
}

}

spring.ai.openai.chat.model=ai/gemma4
spring.ai.openai.api-key=dummy
spring.ai.openai.chat.base-url=http://localhost:12434/engines

Expected behavior
Should get successful response from model running behind docker model runner.

[jewoodev]

Hi, I can take a look at this.

Quick diagnosis:
This looks like a base-URL path change from the openai-java SDK migration in M5. In 1.x, Spring AI appended /v1/chat/completions via completionsPath, so http://localhost:12434/engines became …/engines/v1/chat/completions. With the new SDK, the client expects the base URL to already include /v1 and appends only chat/completions, so the same config resolves to …/engines/chat/completions and returns 404.

Workaround:
spring.ai.openai.chat.base-url=http://localhost:12434/engines/v1

If using spring.ai.openai.base-url instead, the existing spring.ai.openai.chat.base-url override should be removed because the chat-specific value takes precedence.

This also looks like a small docs drift: openai-chat.adoc still documents spring.ai.openai.chat.completions-path, but that property no longer exists after the SDK migration. Happy to send a small focused PR to update the OpenAI chat docs / M5 upgrade notes and add a Docker Model Runner note, unless you prefer restoring a completions-path-like escape hatch.

[eazybytes]

The workaround solves the problem. Both the code and documentation should be sync.

[sdeleuze]

We will upgrade documentation accordingly.