feat: Vertex AI integration (#128)

* Add Vertex AI backend support

Introduce Vertex AI (Google) backend integration: add Vertex-specific settings, models, and services to enable using Vertex AI models.

Key changes:
- Add VertexInferenceParams and GoogleServiceAccountAuth to domain configuration; register BackendType.Vertex and include GoogleServiceAccountAuth in MaINSettings.
- Add Vertex model constants and cloud model records (Gemini/Veo) and register Vertex in LLMApiRegistry.
- Implement VertexService (LLM service) with request parameter mapping, model/endpoint URL construction, and validation of service-account config.
- Add VertexTokenProvider to create and exchange JWT for OAuth2 access tokens using a service account private key.
- Wire Vertex into factory, bootstrapper, HTTP clients, and service constants so it can be resolved and used.
- Add ChatExampleVertex example and register it in the examples program.
- Small ChatService fix: ensure BackendParams are created from model backend and message types set using resolved backend.
- Add local claude settings file used by the infer page tooling.

This commit enables selecting BackendType.Vertex and calling Vertex-hosted models via service-account authentication and token exchange.

* Add shared token cache and refresh lock for Vertex token

Introduce a static ConcurrentDictionary cache keyed by ClientEmail and a SemaphoreSlim to prevent concurrent token refreshes in VertexTokenProvider. Replace per-instance _cachedToken/_tokenExpiry with a CachedToken record, double-check cache after acquiring the lock, and store refreshed tokens with a buffered expiry. Also apply minor formatting/inline null-coalescing cleanup in VertexService getters.

* Add Vertex AI backend support and UI

Introduce Vertex AI support across the InferPage UI and settings plumbing. Changes include:

- Add a new BackendOption (id 9) for Vertex AI and corresponding selection/lookup logic in Settings.razor and Program.cs; Program warns that Vertex requires service account credentials.
- Extend the settings UI to collect Vertex auth fields (Project ID, Client Email, Private Key, Location), with toggleable key visibility and required-field validation to enable saving.
- Persist Vertex auth separately via SettingsService.SaveVertexAuthAsync/GetVertexAuthAsync and a VertexAuthStorage record; private_key is stored outside general settings. Also add VertexLocation to InferPageSettings.
- Pass a GoogleServiceAccountConfig (vertexAuth) through Utils.ApplySettings so MaINSettings.GoogleServiceAccountAuth can be set when selecting Vertex.
- Add model identifier constant for gemini-2.5-pro and fix a CloudModel reference to use Models.Gemini.Gemini2_5Pro.

These changes wire up UI, storage, and application settings so Vertex can be configured from the Settings page and applied at runtime.

* Fix vertex file send for pdf files (without ocr)

Expose CreateMemoryWithVertex on IMemoryFactory and implement it in MemoryFactory to build a KernelMemory configured for Vertex (Gemini) text generation and embeddings using a bearer token provider, location and projectId. Add PDF MIME detection in ChatHelper. Extend VertexService.AskMemory to bypass KernelMemory for multimodal requests: collect inline images/PDFs, convert non-native files to text via DocumentProcessor, aggregate text/context/memory, inject optional grammar JSON, and send combined content to Gemini; includes helper methods for file/stream processing and a list of Gemini-native extensions. Also add necessary usings and temporary file handling for stream processing.

* fix: Use model backend when creating BackendParams

Assign the backend from the chat model before creating BackendParams and use it to initialize BackendParams and to set message types. This ensures message Type mapping is based on model.Backend (and avoids relying on BackendParams.Backend prior to initialization), clarifying initialization order and avoiding potential mismatches.

* final improvements

Add a VertexExample.Setup helper and update examples to call it for Google service account configuration.

Refactor GoogleServiceAccountTokenProvider: use a per-client-email ConcurrentDictionary cache and per-email SemaphoreSlim locks, parse token responses via HttpClient.Json APIs, target oauth2.googleapis.com/token, add IDisposable to dispose RSA, improve error messages and token expiry handling.

Refactor VertexService: centralize GoogleServiceAccountConfig access via Auth property, build endpoints from Auth.ProjectId, streamline GetApiKey/GetApiName/ValidateApiKey, and reorganize multimodal file processing by introducing AppendDocument, BuildQuery and MergeInlineContent helpers, renaming native extensions and cleaning up temp file handling. Small logging and clarity improvements throughout.

* Add Vertex Imagen image generation service

Introduce support for Google Vertex Imagen image generation: add VertexImageGenService to call Vertex AI predict endpoints (uses service account token, default model/location, decodes base64 image into ChatResult), register BackendType.Vertex in ImageGenServiceFactory, add model constant (Imagen4_0_Generate) and CloudModel record (VertexImagen4_0Generate). Also adjust Home.razor injections and minor backendKey formatting. Includes basic response/error handling and model name extraction for publisher prefixes.

* Refactor: replace hardcoded image gen model constants with central ModelRegistry

Introduce new image-generation models and centralize image-gen logic: added Models constants (Imagen4_0_FastGenerate, Flux1Shnell) and new Cloud/Local model records for those models. Replace ad-hoc FLUX checks with ModelRegistry.TryGetById(...).HasImageGeneration across ChatMapper, AgentService, ChatService, and AgentStateManager. Update image generation services (Gemini, OpenAI, Vertex, Xai) to resolve default model IDs from Models, pass the resolved model through to ChatResult, and remove per-service hardcoded model constants. Also add necessary using/import adjustments and improve chat model availability handling in ChatService.

* Add MCP for vertex

Add support for Vertex AI Gemini chat completions and expose location on Mcp. Mcp.cs: introduce a new Location property with default "us-central1". McpService.cs: refactor InitializeChatCompletions to accept the Mcp config (and derive backend and model from it), add a using for LLMService.Auth, and wire up a Vertex backend case that creates a Google service-account token provider, builds a bearer token delegate, normalizes the model name, and registers the VertexAIGemini chat completion with the kernel using config.Location and the project ID. Also adjust the promptSettings call site to pass the config. Includes an auth presence check that throws if Vertex credentials are not configured.

* versioning

* Add new cloud models and XAI reasoning support

Introduce multiple new cloud model definitions and update Groq model constants, plus implement streaming parsing in XaiService.

Changes include:
- Added Gemini NanoBanana and several xAI models (Grok 4.20/4.1 variants, GrokImagine image/pro) and new Groq models (Llama3_3_70b, GptOss120b). Renamed constant Llama3_1_8bInstant -> Llama3_1_8b and added DeepSeek Chat; increased DeepSeek Reasoner token limit.
- Updated example and integration tests to use the renamed Groq model constant.
- Added ProcessChatCompletionChunk in XaiService to handle streaming content, incremental reasoning deltas, and encrypted reasoning blobs; added JSON helper types and System.Text.Json imports to support parsing.

These changes add support for new backends/models and improve handling of xAI streaming/ reasoning responses.

Author: Madzionator

Examples/Examples/Chat/ChatExampleGroqCloud.cs

@@ -12,7 +12,7 @@ public async Task Start()
         Console.WriteLine("(GroqCloud) ChatExample is running!");
 
         await AIHub.Chat()
-            .WithModel(Models.Groq.Llama3_1_8bInstant)
+            .WithModel(Models.Groq.Llama3_1_8b)
             .WithMessage("Which color do people like the most?")
             .CompleteAsync(interactive: true);
     }

Examples/Examples/Chat/ChatExampleVertex.cs

@@ -0,0 +1,24 @@
+using Examples.Utils;
+using MaIN.Core.Hub;
+using MaIN.Domain.Configuration.BackendInferenceParams;
+using MaIN.Domain.Models;
+
+namespace Examples.Chat;
+
+public class ChatExampleVertex : IExample
+{
+    public async Task Start()
+    {
+        VertexExample.Setup(); //We need to provide Google service account config 
+        Console.WriteLine("(Vertex AI) ChatExample is running!");
+
+        await AIHub.Chat()
+            .WithModel(Models.Vertex.Gemini2_5Pro)
+            .WithMessage("Is the killer whale the smartest animal?")
+            .WithInferenceParams(new VertexInferenceParams
+            {
+                Location = "europe-central2"
+            })
+            .CompleteAsync(interactive: true);
+    }
+}

Examples/Examples/Program.cs

@@ -73,6 +73,7 @@ static void RegisterExamples(IServiceCollection services)
     services.AddTransient<ChatGrammarExampleGemini>();
     services.AddTransient<ChatWithImageGenGeminiExample>();
     services.AddTransient<ChatWithFilesExampleGemini>();
+    services.AddTransient<ChatExampleVertex>();
     services.AddTransient<ChatWithReasoningDeepSeekExample>();
     services.AddTransient<ChatWithTextToSpeechExample>();
     services.AddTransient<ChatExampleGroqCloud>();
@@ -186,6 +187,7 @@ public class ExampleRegistry(IServiceProvider serviceProvider)
                 ("\u25a0 Gemini Chat with grammar", serviceProvider.GetRequiredService<ChatGrammarExampleGemini>()),
                 ("\u25a0 Gemini Chat with image", serviceProvider.GetRequiredService<ChatWithImageGenGeminiExample>()),
                 ("\u25a0 Gemini Chat with files", serviceProvider.GetRequiredService<ChatWithFilesExampleGemini>()),
+                ("\u25a0 Vertex Chat", serviceProvider.GetRequiredService<ChatExampleVertex>()),
                 ("\u25a0 DeepSeek Chat with reasoning", serviceProvider.GetRequiredService<ChatWithReasoningDeepSeekExample>()),
                 ("\u25a0 GroqCloud Chat", serviceProvider.GetRequiredService<ChatExampleGroqCloud>()),
                 ("\u25a0 Anthropic Chat", serviceProvider.GetRequiredService<ChatExampleAnthropic>()),

Examples/Examples/Utils/VertexExample.cs

@@ -0,0 +1,22 @@
+using MaIN.Core;
+using MaIN.Domain.Configuration;
+using MaIN.Domain.Configuration.Vertex;
+
+namespace Examples.Utils;
+
+public class VertexExample
+{
+    public static void Setup()
+    {
+        MaINBootstrapper.Initialize(configureSettings: options =>
+        {
+            options.BackendType = BackendType.Vertex;
+            options.GoogleServiceAccountAuth = new GoogleServiceAccountConfig
+            {
+                ProjectId = "<YOUR_GCP_PROJECT_ID>",
+                ClientEmail = "<YOUR_SERVICE_ACCOUNT_EMAIL>",
+                PrivateKey = @"<YOUR_PRIVATE_KEY>"
+            };
+        });
+    }
+}

MaIN.Core.E2ETests/BackendParamsTests.cs

@@ -107,7 +107,7 @@ public async Task GroqCloud_Should_RespondWithParams()
         SkipIfMissingKey(LLMApiRegistry.GetEntry(BackendType.GroqCloud)?.ApiKeyEnvName!);
 
         var result = await AIHub.Chat()
-            .WithModel(Models.Groq.Llama3_1_8bInstant)
+            .WithModel(Models.Groq.Llama3_1_8b)
             .WithMessage(TestQuestion)
             .WithInferenceParams(new GroqCloudInferenceParams
             {
@@ -279,7 +279,7 @@ public async Task GroqCloud_Should_ThrowWhenGivenWrongParams()
     {
         await Assert.ThrowsAsync<InvalidBackendParamsException>(() =>
             AIHub.Chat()
-                .WithModel(Models.Groq.Llama3_1_8bInstant)
+                .WithModel(Models.Groq.Llama3_1_8b)
                 .WithMessage(TestQuestion)
                 .WithInferenceParams(new OpenAiInferenceParams())
                 .CompleteAsync());

Releases/0.10.4.md

@@ -0,0 +1,3 @@
+# 0.10.4 release
+
+Adds Google Vertex AI as a backend with authentication, MCP support, and new models including image generation, along with UI configuration and example usage.

src/MaIN.Core/.nuspec

@@ -2,7 +2,7 @@
 <package>
   <metadata>
     <id>MaIN.NET</id>
-    <version>0.10.3</version>
+    <version>0.10.4</version>
     <authors>Wisedev</authors>
     <owners>Wisedev</owners>
     <icon>favicon.png</icon>

src/MaIN.Domain/Configuration/BackendInferenceParams/BackendParamsFactory.cs

@@ -14,6 +14,7 @@ public static class BackendParamsFactory
         BackendType.Gemini => new GeminiInferenceParams(),
         BackendType.Anthropic => new AnthropicInferenceParams(),
         BackendType.Ollama => new OllamaInferenceParams(),
+        BackendType.Vertex => new VertexInferenceParams(),
         _ => new LocalInferenceParams()
     };
 }

src/MaIN.Domain/Configuration/BackendInferenceParams/VertexInferenceParams.cs

@@ -0,0 +1,18 @@
+using MaIN.Domain.Entities;
+using Grammar = MaIN.Domain.Models.Grammar;
+
+namespace MaIN.Domain.Configuration.BackendInferenceParams;
+
+public class VertexInferenceParams : IBackendInferenceParams
+{
+    public BackendType Backend => BackendType.Vertex;
+
+    public string Location { get; init; } = "us-central1";
+
+    public float? Temperature { get; init; }
+    public int? MaxTokens { get; init; }
+    public float? TopP { get; init; }
+    public string[]? StopSequences { get; init; }
+    public Grammar? Grammar { get; set; }
+    public Dictionary<string, object>? AdditionalParams { get; init; }
+}

src/MaIN.Domain/Configuration/MaINSettings.cs

@@ -1,3 +1,4 @@
+using MaIN.Domain.Configuration.Vertex;
 
 namespace MaIN.Domain.Configuration;
 
@@ -18,6 +19,7 @@ public class MaINSettings
     public SqliteSettings? SqliteSettings { get; set; }
     public SqlSettings? SqlSettings { get; set; }
     public string? VoicesPath { get; set; }
+    public GoogleServiceAccountConfig? GoogleServiceAccountAuth { get; set; }
 }
 
 public enum BackendType
@@ -30,4 +32,5 @@ public enum BackendType
     Anthropic = 5,
     Xai = 6,
     Ollama = 7,
+    Vertex = 8,
 }