You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -40,13 +40,13 @@ The `Agent` class builds a pipeline through class composition with two main comp
40
40
**Agent** (outer component):
41
41
42
42
1.**Agent Middleware + Telemetry** - the `AgentMiddlewareLayer` and `AgentTelemetryLayer` classes handle middleware invocation and OpenTelemetry instrumentation
43
-
2.**RawAgent** - Core agent logic that invokes context providers
44
-
3.**Context Providers** - Unified `context_providers` list manages history and additional context
43
+
2.**RawAgent** - Core agent logic that invokes context providers and collects provider-added middleware
44
+
3.**Context Providers** - Unified `context_providers` list manages history, additional context, and per-run chat/function middleware
45
45
46
46
**ChatClient** (separate and interchangeable component):
47
47
48
48
1.**FunctionInvocation** - Handles tool calling loop, invoking Function Middleware + Telemetry per tool call
49
-
2.**Chat Middleware + Telemetry** - Optional middleware chain and instrumentation layers, running per model call
49
+
2.**Chat Middleware + Telemetry** - Optional middleware chain and instrumentation layers, including any chat middleware added by context providers, running per model call
50
50
3.**RawChatClient** - Provider-specific implementation (Azure OpenAI, OpenAI, Anthropic, etc.) that communicates with the LLM
51
51
52
52
When you call `run()`, your request flows through the Agent layers, then into the ChatClient pipeline for LLM communication.
@@ -144,6 +144,8 @@ agent = Agent(
144
144
)
145
145
```
146
146
147
+
Context providers can also attach chat or function middleware to a single invocation via `SessionContext.extend_middleware()`. The agent flattens those additions in provider order before entering the ChatClient pipeline.
148
+
147
149
::: zone-end
148
150
149
151
For detailed context provider patterns, see [Context Providers](./conversations/context-providers.md).
@@ -157,9 +159,10 @@ The chat client layer handles the actual communication with the LLM service.
157
159
`ChatClientAgent` uses an `IChatClient` instance, which can be decorated with additional middleware:
@@ -170,9 +173,10 @@ var agent = new ChatClientAgent(chatClient, instructions: "You are helpful.");
170
173
You can also use `AIContextProvider` as chat client middleware to enrich messages, tools, and instructions at the client level. This must be used within the context of a running `AIAgent`:
@@ -226,14 +230,14 @@ When you invoke an agent, the request flows through the pipeline:
226
230
**Agent pipeline:**
227
231
228
232
1.**Agent Middleware + Telemetry** executes middleware (if configured) and records spans
229
-
2.**RawAgent** invokes context providers to load history and add context
233
+
2.**RawAgent** invokes context providers to load history, add context, and collect provider-added chat/function middleware
230
234
3. Request is passed to the ChatClient
231
235
232
236
**ChatClient pipeline:**
233
237
234
238
4.**FunctionInvocation** manages the tool calling loop
235
-
- For each tool call, **Function Middleware + Telemetry** executes
236
-
5.**Chat Middleware + Telemetry** executes per model call (if configured)
239
+
- For each tool call, **Function Middleware + Telemetry** executes, including any function middleware added by context providers
240
+
5.**Chat Middleware + Telemetry** executes per model call (if configured), including any chat middleware added by context providers
237
241
6.**RawChatClient** handles provider-specific LLM communication
238
242
7. Response flows back through the same layers
239
243
8.**Context providers** are notified of new messages for storage
@@ -274,6 +278,16 @@ var copilotAgent = originalCopilotAgent
274
278
::: zone-end
275
279
276
280
::: zone pivot="programming-language-python"
281
+
282
+
## Other agent types
283
+
284
+
Not every Python agent uses the full `Agent` + `ChatClient` pipeline. `GitHubCopilotAgent`, for example, sends requests through the GitHub Copilot CLI instead of a local chat client.
285
+
286
+
Even so, Python `GitHubCopilotAgent` still supports agent middleware and now runs `context_providers` around each invocation. Provider-added messages and instructions are included in the prompt sent to Copilot, and providers receive the matching `after_run` callback once a response is available.
287
+
288
+
> [!NOTE]
289
+
> Because `GitHubCopilotAgent` does not use a local chat client, chat client middleware still does not apply.
290
+
277
291
::: zone-end
278
292
279
293
## Next steps
@@ -285,4 +299,4 @@ var copilotAgent = originalCopilotAgent
285
299
286
300
-[Middleware](./middleware/index.md) - Add cross-cutting behavior to your agents
287
301
-[Context Providers](./conversations/context-providers.md) - Detailed patterns for history and context injection
288
-
-[Running Agents](./running-agents.md) - How to invoke agents
302
+
-[Running Agents](./running-agents.md) - How to invoke agents
Copy file name to clipboardExpand all lines: agent-framework/agents/background-responses.md
+6-8Lines changed: 6 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -55,11 +55,10 @@ Some agents may not allow explicit control over background responses. These agen
55
55
For non-streaming scenarios, when you initially run an agent, it may or may not return a continuation token. If no continuation token is returned, it means the operation has completed. If a continuation token is returned, it indicates that the agent has initiated a background response that is still processing and will require polling to retrieve the final result:
56
56
57
57
```csharp
58
-
AIAgentagent=newAzureOpenAIClient(
59
-
newUri("https://<myresource>.openai.azure.com"),
58
+
AIAgentagent=newAIProjectClient(
59
+
newUri("<your-foundry-project-endpoint>"),
60
60
newDefaultAzureCredential())
61
-
.GetResponsesClient("<deployment-name>")
62
-
.AsAIAgent();
61
+
.AsAIAgent(model: "<deployment-name>", instructions: "You are a helpful assistant.");
In streaming scenarios, background responses work much like regular streaming responses - the agent streams all updates back to consumers in real-time. However, the key difference is that if the original stream gets interrupted, agents support stream resumption through continuation tokens. Each update includes a continuation token that captures the current state, allowing the stream to be resumed from exactly where it left off by passing this token to subsequent streaming API calls:
101
100
102
101
```csharp
103
-
AIAgentagent=newAzureOpenAIClient(
104
-
newUri("https://<myresource>.openai.azure.com"),
102
+
AIAgentagent=newAIProjectClient(
103
+
newUri("<your-foundry-project-endpoint>"),
105
104
newDefaultAzureCredential())
106
-
.GetResponsesClient("<deployment-name>")
107
-
.AsAIAgent();
105
+
.AsAIAgent(model: "<deployment-name>", instructions: "You are a helpful assistant.");
> `ContextProvider` and `HistoryProvider` are the canonical Python base classes. `BaseContextProvider` and `BaseHistoryProvider` still exist as deprecated aliases for compatibility, but new providers should inherit from the new names.
322
+
>
323
+
> Context providers can also add chat or function middleware for the current invocation by calling `context.extend_middleware(self.source_id, middleware)`. The agent flattens those additions with `context.get_middleware()` and applies them in provider order before invoking the chat client.
324
+
317
325
:::zone-end
318
326
319
327
:::zone pivot="programming-language-python"
@@ -326,10 +334,10 @@ History providers are context providers specialized for loading/storing messages
326
334
from collections.abc import Sequence
327
335
from typing import Any
328
336
329
-
from agent_framework importBaseHistoryProvider, Message
337
+
from agent_framework importHistoryProvider, Message
@@ -365,6 +373,7 @@ class DatabaseHistoryProvider(BaseHistoryProvider):
365
373
> [!IMPORTANT]
366
374
> In Python, you can configure multiple history providers, but **only one** should use `load_messages=True`.
367
375
> Use additional providers for diagnostics/evals with `load_messages=False` and `store_context_messages=True` so they capture context from other providers alongside input/output.
376
+
> If you need local history to persist around each model call in a tool loop, see [Storage](./storage.md#per-service-call-local-history-persistence).
Tool-calling runs can make multiple model calls before a single `agent.run()` completes. By default, local history providers persist once after the full run. If you want local history to mirror service-managed conversations more closely, set `require_per_service_call_history_persistence=True` so history providers run around each model call instead.
126
+
127
+
:::zone pivot="programming-language-python"
128
+
129
+
```python
130
+
from agent_framework import Agent, InMemoryHistoryProvider
131
+
from agent_framework.openai import OpenAIChatClient
> Use this mode only for framework-managed local history. If the run is already bound to a service-managed conversation (for example via `session.service_session_id` or `options={"conversation_id": ...}`), Agent Framework raises an error instead of mixing the two persistence models.
144
+
>
145
+
> This mode is especially useful when middleware can terminate immediately after a tool call: persisting per model call keeps local history aligned with what a service-managed conversation would keep.
146
+
147
+
:::zone-end
148
+
123
149
## Third-party/Custom storage pattern
124
150
125
151
For database/Redis/blob-backed history, implement a custom history provider.
Console.WriteLine(awaitagent!.RunAsync("Tell me a joke about a pirate in English."));
62
+
63
+
// Invoke the agent with streaming support.
64
+
awaitforeach (varupdateinagent!.RunStreamingAsync("Tell me a joke about a pirate in French."))
65
+
{
66
+
Console.WriteLine(update);
67
+
}
40
68
```
41
69
70
+
> [!WARNING]
71
+
> `DefaultAzureCredential` is convenient for development but requires careful consideration in production. In production, consider using a specific credential (e.g., `ManagedIdentityCredential`) to avoid latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
0 commit comments