feat: InferPage settings (#124)

* Add settings UI for InferPage

Introduce a browser-based settings experience and improve model path resolution. Adds a Settings panel (Settings.razor) with styles and JS (settings.css, settings.js) and services to persist settings and API keys (SettingsService, InferPageSettings, SettingsStateService). Wire the settings UX into NavBar and Home to request/show/save settings and apply them at runtime via Utils.ApplySettings; Program.cs now distinguishes CLI-provided config vs browser configuration (Utils.NeedsConfiguration). Update Utils to support manual capability overrides and sanitize model paths, and tweak app CSS and script includes. Improve LLMService local model resolution and path handling (fallback GenericLocalModel, ResolvePath) so unregistered or absolute model paths load correctly.

* Refactor settings, utils and startup logic

Several refactors and behavior fixes across the app:

- NavBar: call SettingsState.RequestSettings directly from the settings button, simplify theme toggle and reload helpers, remove unused flag/methods.
- Home: prevent adding image inputs when the selected model doesn't support vision and display an error.
- Settings page: build backend options with Prepend for simpler ordering.
- Program.cs: validate Self backend model at startup and exit if unsupported; adjust AddMaIN registration flow based on NeedsConfiguration and BackendType.
- SettingsService: convert to constructor-injected style, consolidate load/save into generic dict helpers, and add typed methods for saving/getting API keys and model history.
- SettingsStateService: shorten comment describing the event bus.
- Utils: unify capability checks with a generic GetCapability<T>, make Reason mutually exclusive with ImageGen, and streamline environment variable handling when setting backend API keys.

These changes simplify codepaths, centralize settings persistence logic, and enforce model capability checks earlier and at the UI level.

* Persist backend profiles with capabilities

Add per-backend profile storage (model + capability flags) and use it in settings UI. SettingsService: introduce BackendProfile record and SaveProfileForBackendAsync/GetProfileForBackendAsync using a new storage key. Settings.razor: load profile on backend select, restore manual vision/reasoning/imagegen flags only for unregistered models, fallback to legacy model-history when no profile exists, and save the full profile instead of just the model. This enables preserving capability information per backend while remaining compatible with older model-only entries.

* Support MMProj files & extract image uploads

Add support for multimodal projector (.mmproj) names across the stack and ensure uploaded image files are treated as images. Implemented MMProjectName on several local vision models and added MmProjName property to InferPage settings, backend profiles, Utils, and ServiceConstants. Settings UI now exposes an MMProj File input for local unregistered vision models and saves/loads it per backend. LLMService now extracts image files from message.Files early (ChatHelper.ExtractImageFromFiles), resolves mmproj name from the model or chat properties to load LLava weights, and requires loaded weights before processing image messages. ChatHelper.ExtractImageFromFiles moves image files into message.Images and cleans up Files so they aren't misrouted to RAG/memory.

* Fix vision for local models

* Support custom model path and MMProj hints

Home: when a custom Model Path is set for Self backend, bake the resolved full file path into a generic local model instance so the LLMService loads from the correct location (creates GenericLocal[Vision][Reasoning]Model variants with FileName set to the resolved path). This fixes cases where Chat.Properties don't reach the service and the model file must be embedded in the model object.

Settings: add RegisteredMmProjPathHint to display the expected MMProj file path for registered local vision models (shows a "MMProj file: ..." hint derived from ResolvedModelPathPreview or fallback model directory), keeping the hint in sync with the "Will load:" preview.

LLMService: when resolving an mmproj for image models, prefer the directory of a fully-qualified modelKey (custom model file path) and fall back to the configured models path; this ensures mmproj files are located next to custom model files.

* CR 1

* versioning

* CR 2 - Make settings dialog responsive

Update settings dialog sizing in src/MaIN.InferPage/wwwroot/settings.css to use viewport-based width and constraints: set width to 40vw with a min-width of 480px and max-width 90vw, and increase max-height from 85vh to 95vh. This improves layout and usability across different screen sizes.

* Fix: Per-call LLama embedder; update Nomic model

Switch LLama embedding implementation to create and dispose contexts per call (aligns with LLamaSharp 0.26.0), removing the long-lived Context field and related state. Read EmbeddingSize from a temporary context at construction, call llama_set_embeddings on each per-call context, and normalize embeddings as before. Update LLamaSharpTextEmbedding defaults: use model-default context (ContextSize=0), enable Embeddings, reduce Batch/UBatch sizes, disable FlashAttention, and set pooling + metadata override for older Nomic GGUFs. Update KnownModels and LocalModels entries for the Nomic embedding model (filename, download URL, display name) and change its embedding dimension to 2048. Remove MemoryService's pre/post import context management and adjust MemoryFactory to load weights, inject pooling metadata, and return an embedding config that uses the model's native context.

* Fix: Preserve images with file attachments across providers

Detect when the latest message contains images and, instead of clearing them, run a memory/kernel search to build contextual text for the LLM. GeminiService and LLMService now: perform SearchAsync, delete the temporary index, aggregate citation text into a context block, inject that context into the message content (nulling Files), call Send, then restore the original message content and return the result. LLMService also performs additional model/resource cleanup when disableCache is true (dispose models/generator and embedder weights, remove model from loader). DeepSeekService and OllamaService no longer null out Images on the message so image data is preserved for the new image-handling flow.

* Use mxbai embedding model and add support

Replace the default embedding model from Nomic to MXBAI across the codebase. Added a new Mxbai_Embedding LocalModel (mxbai-embedding) implementing IEmbeddingModel with a 1024-dimension embedding and updated metadata/file URL. Kept Nomic_Embedding as a separate local model record. Removed the old KnownModels.GetEmbeddingModel helper and updated MemoryFactory to obtain the embedding model from ModelRegistry ("mxbai-embedding") and adjusted imports accordingly.

* Remove unused using directives

Clean up unused using statements across several services to reduce compiler warnings and tidy imports.

Author: Madzionator

Releases/0.10.1.md

@@ -0,0 +1,5 @@
+# 0.10.1 release
+
+- Added InferPage Settings with backend selection, model configuration, API keys, and per-backend profiles.
+- Added custom model paths and multimodal (.mmproj) support for local vision models.
+- Refactored settings logic and upgraded LLamaSharp to 0.26.0 for improved vision reliability.

src/MaIN.Core/.nuspec

@@ -2,7 +2,7 @@
 <package>
   <metadata>
     <id>MaIN.NET</id>
-    <version>0.10.0</version>
+    <version>0.10.1</version>
     <authors>Wisedev</authors>
     <owners>Wisedev</owners>
     <icon>favicon.png</icon>
@@ -13,9 +13,9 @@
     <dependencies>
       <dependency id="GTranslate" version="2.3.1" />
       <dependency id="HtmlAgilityPack" version="1.12.4" />
-      <dependency id="LLamaSharp" version="0.25.0" />
-      <dependency id="LLamaSharp.Backend.Cuda12" version="0.25.0" />
-      <dependency id="LLamaSharp.kernel-memory" version="0.25.0" />
+      <dependency id="LLamaSharp" version="0.26.0" />
+      <dependency id="LLamaSharp.Backend.Cuda12" version="0.26.0" />
+      <dependency id="LLamaSharp.kernel-memory" version="0.26.0" />
       <dependency id="ModelContextProtocol.AspNetCore" version="0.2.0-preview.1" />
       <dependency id="Microsoft.SemanticKernel" version="1.68.0" />
       <dependency id="Microsoft.SemanticKernel.Connectors.Google" version="1.64.0-alpha" />

src/MaIN.Core/Hub/Contexts/ModelContext.cs

@@ -35,7 +35,7 @@ internal ModelContext(MaINSettings settings, IHttpClientFactory httpClientFactor
 
     public AIModel GetModel(string modelId) => ModelRegistry.GetById(modelId);
 
-    public AIModel GetEmbeddingModel() => new Nomic_Embedding();
+    public AIModel GetEmbeddingModel() => new Mxbai_Embedding();
 
     public bool Exists(string modelId)
     {

src/MaIN.Core/MaIN.Core.csproj

@@ -8,7 +8,7 @@
 
     <ItemGroup>
         <PackageReference Include="AsyncKeyedLock" Version="8.0.2" />
-        <PackageReference Include="LLamaSharp.Backend.Cuda12" Version="0.25.0" />
+        <PackageReference Include="LLamaSharp.Backend.Cuda12" Version="0.26.0" />
         <PackageReference Include="Tesseract.Data.English" Version="4.0.0" />
     </ItemGroup>
 

src/MaIN.Domain/MaIN.Domain.csproj

@@ -8,7 +8,7 @@
     </PropertyGroup>
 
     <ItemGroup>
-      <PackageReference Include="LLamaSharp" Version="0.25.0" />
+      <PackageReference Include="LLamaSharp" Version="0.26.0" />
     </ItemGroup>
 
 </Project>

src/MaIN.Domain/Models/Concrete/LocalModels.cs

@@ -18,23 +18,32 @@ public sealed record Gemma3_4b() : LocalModel(
     new Uri("https://huggingface.co/Inza124/Gemma3-4b/resolve/main/gemma3-4b.gguf?download=true"),
     "Gemma3 4B",
     8192,
-    "Balanced 4B model for writing, analysis, and mathematical reasoning");
+    "Balanced 4B model for writing, analysis, and mathematical reasoning"), IVisionModel
+{
+    public string MMProjectName => "mmproj-model-gemma3-4b.gguf";
+}
 
 public sealed record Gemma3_12b() : LocalModel(
     "gemma3-12b",
     "Gemma3-12b.gguf",
     new Uri("https://huggingface.co/Inza124/Gemma3-12b/resolve/main/gemma3-12b.gguf?download=true"),
     "Gemma3 12B",
     8192,
-    "Large 12B model for complex analysis, research, and creative writing");
+    "Large 12B model for complex analysis, research, and creative writing"), IVisionModel
+{
+    public string MMProjectName => "mmproj-model-gemma3-12b.gguf";
+}
 
 public sealed record Gemma3n_e4b() : LocalModel(
     "gemma3n-e4b",
     "Gemma3n-e4b.gguf",
     new Uri("https://huggingface.co/Inza124/Gemma-3n-e4b/resolve/main/gemma-3n-e4b.gguf?download=true"),
     "Gemma3n E4B",
     8192,
-    "Compact 4B model optimized for efficient reasoning and general-purpose tasks");
+    "Compact 4B model optimized for efficient reasoning and general-purpose tasks"), IVisionModel
+{
+    public string MMProjectName => "mmproj-model-gemma3n-e4b.gguf";
+}
 
 // ===== Llama Family =====
 
@@ -286,6 +295,17 @@ public sealed record Olmo2_7b() : LocalModel(
 
 // ===== Embedding Model =====
 
+public sealed record Mxbai_Embedding() : LocalModel(
+    "mxbai-embedding",
+    "mxbai-embed-large-v1.Q4_K_M.gguf",
+    new Uri("https://huggingface.co/ChristianAzinn/mxbai-embed-large-v1-gguf/resolve/main/mxbai-embed-large-v1.Q4_K_M.gguf?download=true"),
+    "mxbai-embed-large v1",
+    512,
+    "Model used to generate embeddings with superior knowledge search recall"), IEmbeddingModel
+{
+    public int EmbeddingDimension => 1024;
+}
+
 public sealed record Nomic_Embedding() : LocalModel(
     "nomic-embedding",
     "nomicv2.gguf",

src/MaIN.Domain/Models/SupportedModels.cs

@@ -246,15 +246,6 @@ public static class KnownModels
         }
     ];
 
-    public static Model GetEmbeddingModel() =>
-        new()
-        {
-            Name = KnownModelNames.Nomic_Embedding,
-            FileName = "nomicv2.gguf",
-            Description = "Model used to generate embeddings.",
-            DownloadUrl = "https://huggingface.co/Inza124/Nomic/resolve/main/nomicv2.gguf?download=true",
-        };
-
     public static bool IsModelSupported(string name) =>
         Models.Any(x => x.Name.Equals(name, StringComparison.InvariantCultureIgnoreCase)
                         || x.Name.Replace(':', '-').Equals(name,

src/MaIN.InferPage/Components/App.razor

@@ -27,6 +27,7 @@
 <script src="_framework/blazor.web.js"></script>
 <script src="scroll.js"></script>
 <script src="editor.js"></script>
+<script src="settings.js"></script>
 <script>
     window.themeManager = {
         save: function (theme) { localStorage.setItem('theme', theme); },

src/MaIN.InferPage/Components/Layout/MainLayout.razor

@@ -1,4 +1,4 @@
-@inherits LayoutComponentBase
+@inherits LayoutComponentBase
 <NavBar/>
 <div class="content">
 
@@ -9,4 +9,4 @@
     An unhandled error has occurred.
     <a href="." class="reload">Reload</a>
     <span class="dismiss">🗙</span>
-</div>
+</div>

src/MaIN.InferPage/Components/Layout/NavBar.razor

@@ -1,7 +1,10 @@
 @using Microsoft.FluentUI.AspNetCore.Components.Icons.Regular
 @using MaIN.Domain.Configuration
+@using MaIN.InferPage.Services
 @inject NavigationManager _navigationManager
 @inject IJSRuntime JS
+@inject SettingsStateService SettingsState
+@implements IDisposable
 @rendermode @(new InteractiveServerRenderMode(prerender: false))
 
 <FluentDesignTheme @bind-Mode="@Mode"
@@ -48,6 +51,11 @@
             Style="margin-left: 10px">Vision 👁️</FluentBadge>
     }
     <div  style="margin-left: auto; align-self: flex-end;">
+        <FluentButton Style="background-color: transparent;"
+                      BackgroundColor="rgba(0, 0, 0, 0)"
+                      Appearance="Appearance.Lightweight"
+                      OnClick="@(() => SettingsState.RequestSettings())" IconStart="@(new Icons.Regular.Size24.Settings().WithColor(AccentColor))">
+        </FluentButton>
         <FluentButton Style="padding: 10px; background-color: transparent;"
                       BackgroundColor="rgba(0, 0, 0, 0)"
                       Appearance="Appearance.Lightweight"
@@ -64,30 +72,28 @@
 @code {
     private DesignThemeModes Mode { get; set; }
     private string AccentColor => Mode == DesignThemeModes.Dark ? "#00ffcc" : "#00cca3";
-    private bool _isChangingTheme = false;
 
     protected override async Task OnAfterRenderAsync(bool firstRender)
     {
         if (firstRender)
         {
             var stored = await JS.InvokeAsync<string>("themeManager.load");
             Mode = stored == "dark" ? DesignThemeModes.Dark : DesignThemeModes.Light;
+            SettingsState.OnSettingsApplied += OnSettingsChanged;
             StateHasChanged();
         }
     }
 
-    private void SetTheme()
+    private void OnSettingsChanged()
     {
-        if (_isChangingTheme) return;
-        _isChangingTheme = true;
-        Mode = Mode == DesignThemeModes.Dark ? DesignThemeModes.Light : DesignThemeModes.Dark;
-        _isChangingTheme = false;
+        InvokeAsync(StateHasChanged);
     }
 
-    private void Reload(MouseEventArgs obj)
-    {
-        _navigationManager.Refresh(true);
-    }
+    private void SetTheme()
+        => Mode = Mode == DesignThemeModes.Dark ? DesignThemeModes.Light : DesignThemeModes.Dark;
+
+    private void Reload()
+        => _navigationManager.Refresh(true);
 
     private string GetBackendColor()
     {
@@ -103,4 +109,9 @@
             _ => Utils.BackendType.ToString()
         };
     }
+
+    public void Dispose()
+    {
+        SettingsState.OnSettingsApplied -= OnSettingsChanged;
+    }
 }