Skip to content

Commit 76b0ba2

Browse files
tisniktisnik
authored
Merge pull request #1993 from tisnik/lcore-1404-cfg
LCORE-1404: Updated config doc
2 parents f82d653 + 583305e commit 76b0ba2

6 files changed

Lines changed: 1205 additions & 715 deletions

File tree

‎docs/config.html‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1023,6 +1023,17 @@ <h2 id="inferenceconfiguration">InferenceConfiguration</h2>
10231023
window &#x2014; callers fall back to their own default or skip the token-based
10241024
trigger.</td>
10251025
</tr>
1026+
<tr class="even">
1027+
<td>providers</td>
1028+
<td>array</td>
1029+
<td>Unified-mode synthesis input (Decision S5): a high-level,
1030+
backend-agnostic list of inference providers the synthesizer expands
1031+
into Llama Stack provider entries. Lives at the configuration root so it
1032+
survives a future backend change. A non-empty list signals unified mode.
1033+
Empty (the default) leaves legacy/remote modes unaffected. The sibling
1034+
default_model / default_provider keep their query-time routing meaning
1035+
and are independent of this list.</td>
1036+
</tr>
10261037
</tbody>
10271038
</table>
10281039
<h2 id="jsonpathoperator">JsonPathOperator</h2>
@@ -1253,6 +1264,18 @@ <h2 id="llamastackconfiguration">LlamaStackConfiguration</h2>
12531264
<td>If enabled, Lightspeed Core can be started even when Llama Stack is
12541265
not accessible (valid for server mode only)</td>
12551266
</tr>
1267+
<tr class="odd">
1268+
<td>config</td>
1269+
<td/>
1270+
<td>Backend-specific knobs for unified mode, where LCORE synthesizes the
1271+
Llama Stack run.yaml instead of reading an external file. Holds the
1272+
baseline selector, an optional profile path, and a raw native_override
1273+
escape hatch. Backend-agnostic high-level sections
1274+
(e.g.&#xA0;inference.providers) live at the configuration root, not here.
1275+
Mutually exclusive with library_client_config_path; that cross-field
1276+
check lives on the root Configuration model. When set in library mode,
1277+
library_client_config_path is not required.</td>
1278+
</tr>
12561279
</tbody>
12571280
</table>
12581281
<h2 id="modelcontextprotocolserver">ModelContextProtocolServer</h2>
@@ -2054,6 +2077,117 @@ <h2 id="trustedproxyserviceaccount">TrustedProxyServiceAccount</h2>
20542077
</tr>
20552078
</tbody>
20562079
</table>
2080+
<h2 id="unifiedinferenceprovider">UnifiedInferenceProvider</h2>
2081+
<p>A high-level inference provider entry for unified-mode synthesis.</p>
2082+
<p>Operators describe inference providers at this high level
2083+
(backend-agnostic vocabulary) instead of authoring raw Llama Stack
2084+
provider blocks. The synthesizer
2085+
(<code>apply_high_level_inference</code>) expands each entry into a
2086+
Llama Stack <code>providers.inference</code> entry, mapping
2087+
<code>type</code> to a <code>provider_type</code> and emitting
2088+
<code>${env.&lt;VAR&gt;}</code> references for secrets (never literal
2089+
values).</p>
2090+
<p>Attributes: type: Canonical provider identifier. Vendor-neutral so it
2091+
survives a future backend change; each backend-specific synthesizer maps
2092+
it to its own provider vocabulary. api_key_env: Name of the environment
2093+
variable holding the provider API key. Emitted verbatim as
2094+
<code>${env.&lt;name&gt;}</code> so the secret never lands on disk
2095+
resolved. allowed_models: Optional allow-list of model identifiers
2096+
passed through to the synthesized provider config. extra: Additional
2097+
provider-config keys merged verbatim into the synthesized provider&#x2019;s
2098+
<code>config</code> block &#x2014; an escape hatch for provider-specific knobs
2099+
not modeled here.</p>
2100+
<table>
2101+
<colgroup>
2102+
<col style="width: 26%"/>
2103+
<col style="width: 23%"/>
2104+
<col style="width: 50%"/>
2105+
</colgroup>
2106+
<thead>
2107+
<tr class="header">
2108+
<th>Field</th>
2109+
<th>Type</th>
2110+
<th>Description</th>
2111+
</tr>
2112+
</thead>
2113+
<tbody>
2114+
<tr class="odd">
2115+
<td>type</td>
2116+
<td>string</td>
2117+
<td>Canonical, backend-agnostic provider identifier mapped to a Llama
2118+
Stack provider_type by the synthesizer.</td>
2119+
</tr>
2120+
<tr class="even">
2121+
<td>api_key_env</td>
2122+
<td>string</td>
2123+
<td>Name of the environment variable holding the provider API key.
2124+
Emitted as a ${env.name} reference so the secret is never written to
2125+
disk in resolved form.</td>
2126+
</tr>
2127+
<tr class="odd">
2128+
<td>allowed_models</td>
2129+
<td>array</td>
2130+
<td>Optional allow-list of model identifiers for this provider.</td>
2131+
</tr>
2132+
<tr class="even">
2133+
<td>extra</td>
2134+
<td>object</td>
2135+
<td>Additional provider-config keys merged verbatim into the synthesized
2136+
provider&#x2019;s config block.</td>
2137+
</tr>
2138+
</tbody>
2139+
</table>
2140+
<h2 id="unifiedllamastackconfig">UnifiedLlamaStackConfig</h2>
2141+
<p>Backend-specific knobs for unified-mode Llama Stack synthesis.</p>
2142+
<p>Per Decision S5 of the design spike, backend-agnostic high-level
2143+
sections (inference, &#x2026;) live at the configuration root, not here. This
2144+
block holds only the Llama-Stack-specific synthesis controls: which
2145+
baseline to start from, an optional profile file, and a raw
2146+
native_override escape hatch.</p>
2147+
<p>Attributes: baseline: Synthesis starting point. &#x201C;default&#x201D; begins from
2148+
LCORE&#x2019;s built-in baseline (src/data/default_run.yaml); &#x201C;empty&#x201D; begins
2149+
from an empty dict (used by the migration tool for an exact round-trip).
2150+
Ignored when <code>profile</code> is set. profile: Optional path to a
2151+
user-authored run.yaml-shaped file used as the synthesis baseline.
2152+
Relative paths resolve against the directory of the loaded
2153+
lightspeed-stack.yaml. native_override: Raw Llama Stack schema
2154+
deep-merged last (maps merge recursively, lists and scalars replace).
2155+
The escape hatch for anything the high-level sections do not
2156+
express.</p>
2157+
<table>
2158+
<colgroup>
2159+
<col style="width: 26%"/>
2160+
<col style="width: 23%"/>
2161+
<col style="width: 50%"/>
2162+
</colgroup>
2163+
<thead>
2164+
<tr class="header">
2165+
<th>Field</th>
2166+
<th>Type</th>
2167+
<th>Description</th>
2168+
</tr>
2169+
</thead>
2170+
<tbody>
2171+
<tr class="odd">
2172+
<td>baseline</td>
2173+
<td>string</td>
2174+
<td>Synthesis starting point: &#x2018;default&#x2019; uses LCORE&#x2019;s built-in baseline,
2175+
&#x2018;empty&#x2019; starts from {}. Ignored when &#x2018;profile&#x2019; is set.</td>
2176+
</tr>
2177+
<tr class="even">
2178+
<td>profile</td>
2179+
<td>string</td>
2180+
<td>Path to a run.yaml-shaped baseline file. Relative paths resolve
2181+
against the directory of the loaded lightspeed-stack.yaml.</td>
2182+
</tr>
2183+
<tr class="odd">
2184+
<td>native_override</td>
2185+
<td>object</td>
2186+
<td>Raw Llama Stack schema deep-merged last (maps merge recursively;
2187+
lists and scalars replace).</td>
2188+
</tr>
2189+
</tbody>
2190+
</table>
20572191
<h2 id="userdatacollection">UserDataCollection</h2>
20582192
<p>User data collection configuration.</p>
20592193
<table>

‎docs/config.json‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -888,6 +888,14 @@
888888
"description": "Map of fully-qualified model identifier (e.g., \"openai/gpt-4o-mini\") to context window size in tokens. Used by the conversation compaction trigger to decide when older turns must be summarized before the input exceeds the window. Models absent from this map have no registered window — callers fall back to their own default or skip the token-based trigger.",
889889
"title": "Per-model context window sizes (tokens)",
890890
"type": "object"
891+
},
892+
"providers": {
893+
"description": "Unified-mode synthesis input (Decision S5): a high-level, backend-agnostic list of inference providers the synthesizer expands into Llama Stack provider entries. Lives at the configuration root so it survives a future backend change. A non-empty list signals unified mode. Empty (the default) leaves legacy/remote modes unaffected. The sibling default_model / default_provider keep their query-time routing meaning and are independent of this list.",
894+
"items": {
895+
"$ref": "#/components/schemas/UnifiedInferenceProvider"
896+
},
897+
"title": "High-level inference providers",
898+
"type": "array"
891899
}
892900
},
893901
"title": "InferenceConfiguration",
@@ -1056,6 +1064,19 @@
10561064
"default": false,
10571065
"description": "If enabled, Lightspeed Core can be started even when Llama Stack is not accessible (valid for server mode only)",
10581066
"title": "Allow degraded mode"
1067+
},
1068+
"config": {
1069+
"anyOf": [
1070+
{
1071+
"$ref": "#/components/schemas/UnifiedLlamaStackConfig"
1072+
},
1073+
{
1074+
"type": "null"
1075+
}
1076+
],
1077+
"default": null,
1078+
"description": "Backend-specific knobs for unified mode, where LCORE synthesizes the Llama Stack run.yaml instead of reading an external file. Holds the baseline selector, an optional profile path, and a raw native_override escape hatch. Backend-agnostic high-level sections (e.g. inference.providers) live at the configuration root, not here. Mutually exclusive with library_client_config_path; that cross-field check lives on the root Configuration model. When set in library mode, library_client_config_path is not required.",
1079+
"title": "Unified Llama Stack configuration"
10591080
}
10601081
},
10611082
"title": "LlamaStackConfiguration",
@@ -1682,6 +1703,82 @@
16821703
"title": "TrustedProxyServiceAccount",
16831704
"type": "object"
16841705
},
1706+
"UnifiedInferenceProvider": {
1707+
"additionalProperties": false,
1708+
"description": "A high-level inference provider entry for unified-mode synthesis.\n\nOperators describe inference providers at this high level (backend-agnostic\nvocabulary) instead of authoring raw Llama Stack provider blocks. The\nsynthesizer (`apply_high_level_inference`) expands each entry into a Llama\nStack `providers.inference` entry, mapping `type` to a `provider_type` and\nemitting `${env.<VAR>}` references for secrets (never literal values).\n\nAttributes:\n type: Canonical provider identifier. Vendor-neutral so it survives a\n future backend change; each backend-specific synthesizer maps it to\n its own provider vocabulary.\n api_key_env: Name of the environment variable holding the provider API\n key. Emitted verbatim as `${env.<name>}` so the secret never lands\n on disk resolved.\n allowed_models: Optional allow-list of model identifiers passed through\n to the synthesized provider config.\n extra: Additional provider-config keys merged verbatim into the\n synthesized provider's `config` block — an escape hatch for\n provider-specific knobs not modeled here.",
1709+
"properties": {
1710+
"type": {
1711+
"description": "Canonical, backend-agnostic provider identifier mapped to a Llama Stack provider_type by the synthesizer.",
1712+
"enum": [
1713+
"openai",
1714+
"sentence_transformers",
1715+
"azure",
1716+
"vertexai",
1717+
"watsonx",
1718+
"vllm_rhaiis",
1719+
"vllm_rhel_ai"
1720+
],
1721+
"title": "Provider type",
1722+
"type": "string"
1723+
},
1724+
"api_key_env": {
1725+
"type": "string",
1726+
"nullable": true,
1727+
"default": null,
1728+
"description": "Name of the environment variable holding the provider API key. Emitted as a ${env.<name>} reference so the secret is never written to disk in resolved form.",
1729+
"title": "API key environment variable"
1730+
},
1731+
"allowed_models": {
1732+
"type": "array",
1733+
"nullable": true,
1734+
"default": null,
1735+
"description": "Optional allow-list of model identifiers for this provider.",
1736+
"title": "Allowed models"
1737+
},
1738+
"extra": {
1739+
"additionalProperties": true,
1740+
"description": "Additional provider-config keys merged verbatim into the synthesized provider's config block.",
1741+
"title": "Extra provider config",
1742+
"type": "object"
1743+
}
1744+
},
1745+
"required": [
1746+
"type"
1747+
],
1748+
"title": "UnifiedInferenceProvider",
1749+
"type": "object"
1750+
},
1751+
"UnifiedLlamaStackConfig": {
1752+
"additionalProperties": false,
1753+
"description": "Backend-specific knobs for unified-mode Llama Stack synthesis.\n\nPer Decision S5 of the design spike, backend-agnostic high-level sections\n(inference, ...) live at the configuration root, not here. This block holds\nonly the Llama-Stack-specific synthesis controls: which baseline to start\nfrom, an optional profile file, and a raw native_override escape hatch.\n\nAttributes:\n baseline: Synthesis starting point. \"default\" begins from LCORE's\n built-in baseline (src/data/default_run.yaml); \"empty\" begins from\n an empty dict (used by the migration tool for an exact round-trip).\n Ignored when `profile` is set.\n profile: Optional path to a user-authored run.yaml-shaped file used as\n the synthesis baseline. Relative paths resolve against the directory\n of the loaded lightspeed-stack.yaml.\n native_override: Raw Llama Stack schema deep-merged last (maps merge\n recursively, lists and scalars replace). The escape hatch for\n anything the high-level sections do not express.",
1754+
"properties": {
1755+
"baseline": {
1756+
"default": "default",
1757+
"description": "Synthesis starting point: 'default' uses LCORE's built-in baseline, 'empty' starts from {}. Ignored when 'profile' is set.",
1758+
"enum": [
1759+
"default",
1760+
"empty"
1761+
],
1762+
"title": "Baseline selector",
1763+
"type": "string"
1764+
},
1765+
"profile": {
1766+
"type": "string",
1767+
"nullable": true,
1768+
"default": null,
1769+
"description": "Path to a run.yaml-shaped baseline file. Relative paths resolve against the directory of the loaded lightspeed-stack.yaml.",
1770+
"title": "Profile path"
1771+
},
1772+
"native_override": {
1773+
"additionalProperties": true,
1774+
"description": "Raw Llama Stack schema deep-merged last (maps merge recursively; lists and scalars replace).",
1775+
"title": "Native override",
1776+
"type": "object"
1777+
}
1778+
},
1779+
"title": "UnifiedLlamaStackConfig",
1780+
"type": "object"
1781+
},
16851782
"UserDataCollection": {
16861783
"additionalProperties": false,
16871784
"description": "User data collection configuration.",

0 commit comments

Comments
 (0)