fix(distributed): reject wrong-model requests on the remaining modalities

[localai-bot]

Problem

#10970 fixed silent wrong-model answers for the four PredictOptions RPCs and explicitly deferred the rest. The exposure is not specific to those four: the controller caches a NodeModel row naming a backend's host:port, a worker can recycle a stopped backend's port for a different model's backend, and probeHealth verifies liveness, not identity. Any modality routed through that cache can be answered by the wrong model, silently.

Scope

Enumerated from grpc.InferenceBackend (pkg/grpc/backend.go:69-98) rather than the proto service — that interface is exactly the set of RPCs InFlightTrackingClient wraps with reconcile, i.e. the ones reachable through a cached NodeModel row.

#10970 covered 4. This covers the remaining 21 request messages / 24 methods: GenerateImageRequest, GenerateVideoRequest, TranscriptRequest (×2), TTSRequest (×2), SoundGenerationRequest, DetectOptions, DepthRequest, FaceVerifyRequest, FaceAnalyzeRequest, VoiceVerifyRequest, VoiceAnalyzeRequest, VoiceEmbedRequest, RerankRequest, TokenClassifyRequest, ScoreRequest, VADRequest, DiarizeRequest, SoundDetectionRequest, AudioTransformRequest.

Approach

The same mechanism as #10970, not a parallel one:

• proto: ModelIdentity on each message.
• controller: populated from ModelConfig.Model at the call site that also builds ModelOptions, so the load-time and request-time values are equal by construction. Rerank and Depth receive caller-built requests (from jina/rerank.go and localai/depth.go, neither of which holds a ModelConfig), so they are stamped inside backend.Rerank/backend.Depth; the three toProto helpers take identity as a parameter for the same reason.
• backends: pkg/grpc/server.go takes a GetModelIdentity() string interface so one guard covers every message (27 Go backends); _GUARDED_METHODS in backend/python/common (36 Python backends); llama-cpp's guard becomes a template and gains AudioTranscription/Stream, Rerank, Score; privacy-filter gets the guard for TokenClassify.
• router: no change — reconcile already drops the stale row on IsModelMismatch.

Verified FileStagingClient rewrites only Src/Dst/Voice/Model/StartImage/EndImage/Audio — never ModelIdentity.

TTS and SoundGeneration get a separate field

Both already carry a model string, and it is not usable for identity: FileStagingClient.TTS/.TTSStream rewrite it into a worker-local absolute path while the load-time value is untranslated, so in distributed mode the two already differ. Comparing them would reject valid requests in exactly the configuration this guards.

They therefore get a distinct ModelIdentity carrying the untranslated value. newTTSRequest now takes both, with specs pinning that they are distinct inputs.

Deliberately excluded, with reasons

AudioEncode / AudioDecode. The opus codec backend is loaded at core/http/endpoints/openai/realtime_webrtc.go:98 with a literal model.WithModel("opus") and no ModelConfig anywhere in the path; the requests are built in a different type and file. No value there carries the structural equality guarantee that makes this safe — an identity would be a duplicated literal, i.e. convention rather than construction, which is the false-rejection risk #10970 refused. Pinned by spec in both Go and Python so a future completeness pass must decide deliberately.

AudioTranscriptionLive, AudioTransformStream, AudioToAudioStream, Forward. ControlBackend members that bypass reconcile entirely; guarding them needs identity on a config message plus first-message inspection in each handler. Noted in the proto and Python comments.

Transparent retry: assessed, not implemented

reconcile receives only an error. Retrying would need each wrapper to pass a re-invocable closure, InFlightTrackingClient to hold a back-reference to SmartRouter plus all five Route parameters it does not currently know, ownership of a second Release the outer caller cannot see, and a guard against Route step 2 — which detaches from the caller's context and can stage multiple GB under a distributed lock, inside a request that has already spent its budget. That is a routing-lifecycle change, not a retry.

Behaviour is unchanged: the first request after a misroute fails cleanly, the second succeeds.

Compatibility

Empty means skip on both sides. Old controller → new backend sends nothing; a backend loaded by an old controller has recorded nothing. The 6 bare-request-struct sites in tests/e2e-backends/backend_test.go are unaffected, and specs in Go and Python pin the empty case so a future tightening fails there first.

Testing

Red first and functionally in all three languages:

• Go pkg/grpc: "VoiceEmbed must reject a wrong-model request: Expected an error to have occurred. Got: nil"
• Go core/backend: 17 specs, "must send ModelConfig.Model … Expected <string>: (empty)"
• Python: "AssertionError: Lists differ: 21 additional elements … no request may reach the model"

The TTS spec was rewritten to fail on behaviour after it was found failing on arity.

make protogen-go clean; go build ./core/... ./pkg/... ./tests/... OK; go test -count=1 green on pkg/grpc, core/backend, core/services/nodes (188s), core/http/endpoints/...; Python 20/20 including the auth-off default path (get_auth_interceptors() with LOCALAI_GRPC_AUTH_TOKEN unset — the trap #10970 hit, where anything wired after the early return silently does nothing across all 36 backends); make lint 0 issues.

Not verified — needs CI

The C++ changes do not compile in the development environment (no protobuf/gRPC headers in the worktree; backend/cpp/run-unit-tests.sh only covers stdlib-only tests). The llama-cpp guard is now a template over the request type with 4 new call sites, and privacy-filter gets the guard from scratch including a new g_loaded_model_identity global. Both need a CI build before merge. #10970 shipped its C++ half the same way, but this is stated explicitly rather than assumed to be at parity.

---

🤖 Generated with Claude Code