docs: add MLflow AI Gateway cookbook example#7511
Open
PattaraS wants to merge 3 commits intomicrosoft:mainfrom
Open
docs: add MLflow AI Gateway cookbook example#7511PattaraS wants to merge 3 commits intomicrosoft:mainfrom
PattaraS wants to merge 3 commits intomicrosoft:mainfrom
Conversation
Author
|
@microsoft-github-policy-service agree company="Databricks" |
Adds a Jupyter notebook demonstrating how to use MLflow AI Gateway as an LLM backend for AutoGen agents via OpenAIChatCompletionClient with a custom base_url pointing to the gateway's OpenAI-compatible endpoint.
Add screenshots for endpoint creation, budget tracking, and usage tracing. Simplify prerequisites to UI-only setup with link to docs for advanced options.
The model_client was closed in the multi-agent cell, causing the subsequent streaming cell to fail with APIConnectionError. Moved await model_client.close() to the last code cell.
PattaraS
force-pushed
the
add-mlflow-gateway-integration
branch
from
202b7d7 to
2b379c8
Compare
Author
|
Hi @victordibia, friendly ping on this PR. We're from the MLflow team and working on adding MLflow AI Gateway integration guides across major agent frameworks. MLflow AI Gateway (MLflow >= 3.0) is an open-source LLM proxy with built-in secrets management, fallback/retry, traffic splitting, and usage tracing. Since it exposes an OpenAI-compatible API, it works with AutoGen's OpenAIChatCompletionClient via base_url. This PR adds a cookbook recipe demonstrating that, with screenshots of the gateway UI. Happy to make any changes. Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
Adds a cookbook recipe showing how to use MLflow AI Gateway as an LLM backend for AutoGen agents. MLflow AI Gateway is a database-backed LLM proxy (MLflow ≥ 3.0) that provides a unified OpenAI-compatible API across 20+ providers with encrypted secrets management, fallback/retry, traffic splitting, and usage tracing.
The notebook demonstrates using
OpenAIChatCompletionClientwith a custombase_urlpointing to the gateway — single-turn chat, multi-agent chat, and streaming examples. Includes screenshots of the gateway UI.Related issue number
N/A — new documentation
Checks
AI Disclosure
This pull request was AI-assisted by Claude. All content was reviewed and validated by a human contributor. Screenshots were captured manually from a running MLflow server.