Skip to content

Commit fb0e2bf

Browse files
tisniktisnik
authored
Merge pull request lightspeed-core#2070 from tisnik/lcore-2861-sync-config-doc
LCORE-2861: Sync config doc
2 parents ca481ca + 75b5db5 commit fb0e2bf

3 files changed

Lines changed: 38 additions & 6 deletions

File tree

‎docs/config.html‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1034,6 +1034,22 @@ <h2 id="inferenceconfiguration">InferenceConfiguration</h2>
10341034
default_model / default_provider keep their query-time routing meaning
10351035
and are independent of this list.</td>
10361036
</tr>
1037+
<tr class="odd">
1038+
<td>max_infer_iters</td>
1039+
<td>integer</td>
1040+
<td>Server-side default for the maximum number of inference iterations a
1041+
model can perform in a single request. Prevents small models from
1042+
looping indefinitely on tool calls. Per-request values take precedence
1043+
over this default. Set to None to disable the limit.</td>
1044+
</tr>
1045+
<tr class="even">
1046+
<td>max_tool_calls</td>
1047+
<td>integer</td>
1048+
<td>Server-side default for the maximum number of tool calls allowed in
1049+
a single response. Prevents small models from exhausting the context
1050+
window with repeated tool calls. Per-request values take precedence over
1051+
this default. Set to None to disable the limit.</td>
1052+
</tr>
10371053
</tbody>
10381054
</table>
10391055
<h2 id="jsonpathoperator">JsonPathOperator</h2>

‎docs/config.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -896,6 +896,20 @@
896896
},
897897
"title": "High-level inference providers",
898898
"type": "array"
899+
},
900+
"max_infer_iters": {
901+
"type": "integer",
902+
"nullable": true,
903+
"default": 10,
904+
"description": "Server-side default for the maximum number of inference iterations a model can perform in a single request. Prevents small models from looping indefinitely on tool calls. Per-request values take precedence over this default. Set to None to disable the limit.",
905+
"title": "Default max inference iterations"
906+
},
907+
"max_tool_calls": {
908+
"type": "integer",
909+
"nullable": true,
910+
"default": 30,
911+
"description": "Server-side default for the maximum number of tool calls allowed in a single response. Prevents small models from exhausting the context window with repeated tool calls. Per-request values take precedence over this default. Set to None to disable the limit.",
912+
"title": "Default max tool calls"
899913
}
900914
},
901915
"title": "InferenceConfiguration",

‎docs/config.md‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -332,12 +332,14 @@ In-memory cache configuration.
332332
Inference configuration.
333333

334334

335-
| Field | Type | Description |
336-
|------------------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
337-
| default_model | string | Identification of default model used when no other model is specified. |
338-
| default_provider | string | Identification of default provider used when no other model is specified. |
339-
| context_windows | object | Map of fully-qualified model identifier (e.g., "openai/gpt-4o-mini") to context window size in tokens. Used by the conversation compaction trigger to decide when older turns must be summarized before the input exceeds the window. Models absent from this map have no registered window — callers fall back to their own default or skip the token-based trigger. |
340-
| providers | array | Unified-mode synthesis input (Decision S5): a high-level, backend-agnostic list of inference providers the synthesizer expands into Llama Stack provider entries. Lives at the configuration root so it survives a future backend change. A non-empty list signals unified mode. Empty (the default) leaves legacy/remote modes unaffected. The sibling default_model / default_provider keep their query-time routing meaning and are independent of this list. |
335+
| Field | Type | Description |
336+
|------------------|---------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
337+
| default_model | string | Identification of default model used when no other model is specified. |
338+
| default_provider | string | Identification of default provider used when no other model is specified. |
339+
| context_windows | object | Map of fully-qualified model identifier (e.g., "openai/gpt-4o-mini") to context window size in tokens. Used by the conversation compaction trigger to decide when older turns must be summarized before the input exceeds the window. Models absent from this map have no registered window — callers fall back to their own default or skip the token-based trigger. |
340+
| providers | array | Unified-mode synthesis input (Decision S5): a high-level, backend-agnostic list of inference providers the synthesizer expands into Llama Stack provider entries. Lives at the configuration root so it survives a future backend change. A non-empty list signals unified mode. Empty (the default) leaves legacy/remote modes unaffected. The sibling default_model / default_provider keep their query-time routing meaning and are independent of this list. |
341+
| max_infer_iters | integer | Server-side default for the maximum number of inference iterations a model can perform in a single request. Prevents small models from looping indefinitely on tool calls. Per-request values take precedence over this default. Set to None to disable the limit. |
342+
| max_tool_calls | integer | Server-side default for the maximum number of tool calls allowed in a single response. Prevents small models from exhausting the context window with repeated tool calls. Per-request values take precedence over this default. Set to None to disable the limit. |
341343

342344

343345
## JsonPathOperator

0 commit comments

Comments
 (0)