Skip to content

Commit 1917729

Browse files
Copilotowndev
Copilot
and
owndev
authored
Add Azure Search / RAG Integration Support to Azure AI Foundry Pipeline (#49)
* Initial plan * Implement Azure Search/RAG integration in Azure AI Foundry Pipeline Co-authored-by: owndev <69784886+owndev@users.noreply.github.com> * Add .gitignore for Python cache files and finalize Azure Search implementation Co-authored-by: owndev <69784886+owndev@users.noreply.github.com> * Remove pycache * Update .gitignore * Add OpenAI OSS Models * Update Version * Fix Azure Search validation errors by removing unsupported fields Co-authored-by: owndev <69784886+owndev@users.noreply.github.com> * Revert "Fix Azure Search validation errors" - restore required fields Co-authored-by: owndev <69784886+owndev@users.noreply.github.com> * Restore version to 2.3.1 after reverting changes Co-authored-by: owndev <69784886+owndev@users.noreply.github.com> * Improvements added * Add OpenAI GPT-5 Models * Added some GPT models * fix: Streaming in azure_ai_foundry.py From PR: #55 * Update docs --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: owndev <69784886+owndev@users.noreply.github.com>
1 parent ff29c2e commit 1917729

3 files changed

Lines changed: 487 additions & 57 deletions

File tree

‎.gitignore‎

Lines changed: 206 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,208 @@
11

2-
# pixi environments
2+
# Byte-compiled / optimized / DLL files
3+
__pycache__/
4+
*.py[codz]
5+
*$py.class
6+
7+
# C extensions
8+
*.so
9+
10+
# Distribution / packaging
11+
.Python
12+
build/
13+
develop-eggs/
14+
dist/
15+
downloads/
16+
eggs/
17+
.eggs/
18+
lib/
19+
lib64/
20+
parts/
21+
sdist/
22+
var/
23+
wheels/
24+
share/python-wheels/
25+
*.egg-info/
26+
.installed.cfg
27+
*.egg
28+
MANIFEST
29+
30+
# PyInstaller
31+
# Usually these files are written by a python script from a template
32+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
33+
*.manifest
34+
*.spec
35+
36+
# Installer logs
37+
pip-log.txt
38+
pip-delete-this-directory.txt
39+
40+
# Unit test / coverage reports
41+
htmlcov/
42+
.tox/
43+
.nox/
44+
.coverage
45+
.coverage.*
46+
.cache
47+
nosetests.xml
48+
coverage.xml
49+
*.cover
50+
*.py.cover
51+
.hypothesis/
52+
.pytest_cache/
53+
cover/
54+
55+
# Translations
56+
*.mo
57+
*.pot
58+
59+
# Django stuff:
60+
*.log
61+
local_settings.py
62+
db.sqlite3
63+
db.sqlite3-journal
64+
65+
# Flask stuff:
66+
instance/
67+
.webassets-cache
68+
69+
# Scrapy stuff:
70+
.scrapy
71+
72+
# Sphinx documentation
73+
docs/_build/
74+
75+
# PyBuilder
76+
.pybuilder/
77+
target/
78+
79+
# Jupyter Notebook
80+
.ipynb_checkpoints
81+
82+
# IPython
83+
profile_default/
84+
ipython_config.py
85+
86+
# pyenv
87+
# For a library or package, you might want to ignore these files since the code is
88+
# intended to run in multiple environments; otherwise, check them in:
89+
# .python-version
90+
91+
# pipenv
92+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
93+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
94+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
95+
# install all needed dependencies.
96+
#Pipfile.lock
97+
98+
# UV
99+
# Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
100+
# This is especially recommended for binary packages to ensure reproducibility, and is more
101+
# commonly ignored for libraries.
102+
#uv.lock
103+
104+
# poetry
105+
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
106+
# This is especially recommended for binary packages to ensure reproducibility, and is more
107+
# commonly ignored for libraries.
108+
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
109+
#poetry.lock
110+
#poetry.toml
111+
112+
# pdm
113+
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
114+
# pdm recommends including project-wide configuration in pdm.toml, but excluding .pdm-python.
115+
# https://pdm-project.org/en/latest/usage/project/#working-with-version-control
116+
#pdm.lock
117+
#pdm.toml
118+
.pdm-python
119+
.pdm-build/
120+
121+
# pixi
122+
# Similar to Pipfile.lock, it is generally recommended to include pixi.lock in version control.
123+
#pixi.lock
124+
# Pixi creates a virtual environment in the .pixi directory, just like venv module creates one
125+
# in the .venv directory. It is recommended not to include this directory in version control.
3126
.pixi
4-
*.egg-info
127+
128+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
129+
__pypackages__/
130+
131+
# Celery stuff
132+
celerybeat-schedule
133+
celerybeat.pid
134+
135+
# SageMath parsed files
136+
*.sage.py
137+
138+
# Environments
139+
.env
140+
.envrc
141+
.venv
142+
env/
143+
venv/
144+
ENV/
145+
env.bak/
146+
venv.bak/
147+
148+
# Spyder project settings
149+
.spyderproject
150+
.spyproject
151+
152+
# Rope project settings
153+
.ropeproject
154+
155+
# mkdocs documentation
156+
/site
157+
158+
# mypy
159+
.mypy_cache/
160+
.dmypy.json
161+
dmypy.json
162+
163+
# Pyre type checker
164+
.pyre/
165+
166+
# pytype static type analyzer
167+
.pytype/
168+
169+
# Cython debug symbols
170+
cython_debug/
171+
172+
# PyCharm
173+
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
174+
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
175+
# and can be added to the global gitignore or merged into this file. For a more nuclear
176+
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
177+
#.idea/
178+
179+
# Abstra
180+
# Abstra is an AI-powered process automation framework.
181+
# Ignore directories containing user credentials, local state, and settings.
182+
# Learn more at https://abstra.io/docs
183+
.abstra/
184+
185+
# Visual Studio Code
186+
# Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore
187+
# that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
188+
# and can be added to the global gitignore or merged into this file. However, if you prefer,
189+
# you could uncomment the following to ignore the entire vscode folder
190+
# .vscode/
191+
192+
# Ruff stuff:
193+
.ruff_cache/
194+
195+
# PyPI configuration file
196+
.pypirc
197+
198+
# Cursor
199+
# Cursor is an AI-powered code editor. `.cursorignore` specifies files/directories to
200+
# exclude from AI features like autocomplete and code analysis. Recommended for sensitive data
201+
# refer to https://docs.cursor.com/context/ignore-files
202+
.cursorignore
203+
.cursorindexingignore
204+
205+
# Marimo
206+
marimo/_static/
207+
marimo/_lsp/
208+
__marimo__/

‎docs/azure-ai-integration.md‎

Lines changed: 116 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -1,53 +1,116 @@
1-
# Azure AI Integration
2-
3-
The repository includes functions specifically designed for **Azure AI**, supporting both **Azure OpenAI** models and general **Azure AI** services.
4-
5-
🔗 [Learn More About Azure AI](https://own.dev/azure-microsoft-com-en-us-solutions-ai)
6-
7-
8-
## Pipeline
9-
- 🧩 [Azure AI Foundry Pipeline](https://own.dev/github-owndev-open-webui-functions-azure-ai-foundry)
10-
11-
12-
### Features:
13-
14-
- **Azure OpenAI API Support**
15-
Access models like **GPT-4o, o3**, and **other fine-tuned AI models** via Azure.
16-
17-
- **Azure AI Model Deployment**
18-
Connect to **custom models** hosted on Azure AI.
19-
20-
- **Secure API Requests**
21-
Supports API key authentication and environment variable configurations.
22-
23-
24-
### Environment Variables:
25-
26-
Configure the following environment variables to enable Azure AI support:
27-
28-
```bash
29-
# API key or token for Azure AI
30-
AZURE_AI_API_KEY="your-api-key"
31-
32-
# Azure AI endpoint
33-
# Examples:
34-
# - For general Azure AI: "https://<your-endpoint>/chat/completions?api-version=2024-05-01-preview"
35-
# - For Azure OpenAI: "https://<your-endpoint>/openai/deployments/<model-name>/chat/completions?api-version=2024-08-01-preview"
36-
AZURE_AI_ENDPOINT="https://<your project>.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview"
37-
38-
# Optional: model names (if not embedded in the URL)
39-
# Supports semicolon or comma separated values: "gpt-4o;gpt-4o-mini" or "gpt-4o,gpt-4o-mini"
40-
AZURE_AI_MODEL="gpt-4o;gpt-4o-mini"
41-
42-
# If true, the model name will be included in the request body
43-
AZURE_AI_MODEL_IN_BODY=true
44-
45-
# Whether to use a predefined list of Azure AI models
46-
USE_PREDEFINED_AZURE_AI_MODELS=false
47-
48-
# If true, use "Authorization: Bearer" instead of "api-key" header
49-
AZURE_AI_USE_AUTHORIZATION_HEADER=false
50-
```
51-
52-
> [!TIP]
53-
> To use **Azure OpenAI** and other **Azure AI** models **simultaneously**, you can use the following URL: `https://<your project>.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview`
1+
# Azure AI Integration
2+
3+
The repository includes functions specifically designed for **Azure AI**, supporting both **Azure OpenAI** models and general **Azure AI** services.
4+
5+
🔗 [Learn More About Azure AI](https://azure.microsoft.com/en-us/solutions/ai)
6+
7+
## Pipeline
8+
9+
- 🧩 [Azure AI Foundry Pipeline](../pipelines/azure/azure_ai_foundry.py)
10+
11+
### Features
12+
13+
- **Azure OpenAI API Support**
14+
Access models like **GPT-4o, o3**, and **other fine-tuned AI models** via Azure.
15+
16+
- **Azure AI Model Deployment**
17+
Connect to **custom models** hosted on Azure AI.
18+
19+
- **Secure API Requests**
20+
Supports API key authentication and environment variable configurations.
21+
22+
### Environment Variables
23+
24+
Configure the following environment variables to enable Azure AI support:
25+
26+
```bash
27+
# API key or token for Azure AI
28+
AZURE_AI_API_KEY="your-api-key"
29+
30+
# Azure AI endpoint
31+
# Examples:
32+
# - For general Azure AI: "https://<your-endpoint>/chat/completions?api-version=2024-05-01-preview"
33+
# - For Azure OpenAI: "https://<your-endpoint>/openai/deployments/<model-name>/chat/completions?api-version=2024-08-01-preview"
34+
AZURE_AI_ENDPOINT="https://<your project>.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview"
35+
36+
# Optional: model names (if not embedded in the URL)
37+
# Supports semicolon or comma separated values: "gpt-4o;gpt-4o-mini" or "gpt-4o,gpt-4o-mini"
38+
AZURE_AI_MODEL="gpt-4o;gpt-4o-mini"
39+
40+
# If true, the model name will be included in the request body
41+
AZURE_AI_MODEL_IN_BODY=true
42+
43+
# Whether to use a predefined list of Azure AI models
44+
USE_PREDEFINED_AZURE_AI_MODELS=false
45+
46+
# If true, use "Authorization: Bearer" instead of "api-key" header
47+
AZURE_AI_USE_AUTHORIZATION_HEADER=false
48+
49+
# Azure Search / RAG Configuration (optional)
50+
# Azure Search endpoint for document retrieval
51+
AZURE_SEARCH_ENDPOINT="https://your-search-service.search.windows.net"
52+
53+
# Azure Search index name containing the documents
54+
AZURE_SEARCH_INDEX_NAME="your-index-name"
55+
56+
# Azure Search project resource ID (optional)
57+
AZURE_SEARCH_PROJECT_RESOURCE_ID="your-project-resource-id"
58+
59+
# Azure Search API key (if using api_key authentication)
60+
AZURE_SEARCH_KEY="your-search-api-key"
61+
62+
# Authentication type for Azure Search
63+
AZURE_SEARCH_AUTHENTICATION_TYPE="system_assigned_managed_identity"
64+
65+
# Semantic configuration name for Azure Search
66+
AZURE_SEARCH_SEMANTIC_CONFIGURATION="azureml-default"
67+
68+
# Azure Search embedding endpoint (optional)
69+
AZURE_SEARCH_EMBEDDING_ENDPOINT="your-embedding-endpoint"
70+
71+
# Azure Search embedding API key (optional)
72+
AZURE_SEARCH_EMBEDDING_KEY="your-embedding-key"
73+
74+
# Query type for Azure Search
75+
AZURE_SEARCH_QUERY_TYPE="vectorSimpleHybrid"
76+
77+
# Whether to limit search to indexed documents only
78+
AZURE_SEARCH_IN_SCOPE=false
79+
80+
# Role information for Azure Search responses
81+
AZURE_SEARCH_ROLE_INFORMATION="You are an AI assistant."
82+
83+
# Azure Search strictness level (1-5)
84+
AZURE_SEARCH_STRICTNESS=5
85+
86+
# Number of top documents to retrieve
87+
AZURE_SEARCH_TOP_N_DOCUMENTS=20
88+
```
89+
90+
### Azure Search / RAG Integration
91+
92+
The pipeline now supports **Azure Search** integration for **Retrieval-Augmented Generation (RAG)**. When configured, the pipeline will automatically include a `data_sources` field in requests to Azure AI, enabling document-based AI responses.
93+
94+
#### Configuration
95+
96+
Configure Azure Search by setting the following environment variables:
97+
98+
- **AZURE_SEARCH_ENDPOINT**: Your Azure Search service endpoint
99+
- **AZURE_SEARCH_INDEX_NAME**: Name of the search index containing your documents
100+
- **AZURE_SEARCH_AUTHENTICATION_TYPE**: Authentication method (`system_assigned_managed_identity` or `api_key`)
101+
- **AZURE_SEARCH_KEY**: API key (if using `api_key` authentication)
102+
103+
#### Optional Settings
104+
105+
- **AZURE_SEARCH_PROJECT_RESOURCE_ID**: Project resource ID
106+
- **AZURE_SEARCH_SEMANTIC_CONFIGURATION**: Semantic configuration name
107+
- **AZURE_SEARCH_EMBEDDING_ENDPOINT**: Embedding service endpoint
108+
- **AZURE_SEARCH_EMBEDDING_KEY**: Embedding service API key
109+
- **AZURE_SEARCH_QUERY_TYPE**: Query type (`vectorSimpleHybrid`, `vector`, `semantic`)
110+
- **AZURE_SEARCH_IN_SCOPE**: Limit to indexed documents only
111+
- **AZURE_SEARCH_ROLE_INFORMATION**: Role information for responses
112+
- **AZURE_SEARCH_STRICTNESS**: Strictness level (1-5)
113+
- **AZURE_SEARCH_TOP_N_DOCUMENTS**: Number of documents to retrieve
114+
115+
> [!TIP]
116+
> To use **Azure OpenAI** and other **Azure AI** models **simultaneously**, you can use the following URL: `https://<your project>.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview`

0 commit comments

Comments
 (0)