Skip to content
This repository was archived by the owner on Jul 13, 2026. It is now read-only.

Commit 7302e72

Browse files
diberryCopilot
diberry
and
Copilot
committed
Update select-algorithm samples to use local data/ folder
Each sample now expects Hotels_Vector.json in a local data/ folder instead of referencing the shared ../../data/ path. Added data/README.md placeholders with copy instructions for each sample. Path changes: - TypeScript: data/Hotels_Vector.json (joined with __dirname/..) - Python: ../data/Hotels_Vector.json (scripts run from src/) - Go: ./data/Hotels_Vector.json (runs from project root) - Java: ./data/Hotels_Vector.json (Maven runs from project root) - .NET: ./data/Hotels_Vector.json (matches appsettings.json) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 4927a9c commit 7302e72

22 files changed

Lines changed: 630 additions & 89 deletions

File tree

‎ai/select-algorithm-dotnet/Models/Configuration.cs‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
namespace SelectAlgorithm.Models;
2+
3+
public class AppConfiguration
4+
{
5+
public AzureOpenAIConfiguration AzureOpenAI { get; set; } = new();
6+
public DocumentDBConfiguration DocumentDB { get; set; } = new();
7+
public EmbeddingConfiguration Embedding { get; set; } = new();
8+
public VectorSearchConfiguration VectorSearch { get; set; } = new();
9+
public DataFilesConfiguration DataFiles { get; set; } = new();
10+
}
11+
12+
public class AzureOpenAIConfiguration
13+
{
14+
public string Endpoint { get; set; } = string.Empty;
15+
public string EmbeddingModel { get; set; } = "text-embedding-3-small";
16+
}
17+
18+
public class DocumentDBConfiguration
19+
{
20+
public string ClusterName { get; set; } = string.Empty;
21+
public string DatabaseName { get; set; } = "Hotels";
22+
public int LoadBatchSize { get; set; } = 100;
23+
}
24+
25+
public class EmbeddingConfiguration
26+
{
27+
public string EmbeddedField { get; set; } = "DescriptionVector";
28+
public int Dimensions { get; set; } = 1536;
29+
}
30+
31+
public class VectorSearchConfiguration
32+
{
33+
public string Query { get; set; } = "luxury hotel near the beach";
34+
public int TopK { get; set; } = 3;
35+
}
36+
37+
public class DataFilesConfiguration
38+
{
39+
public string WithVectors { get; set; } = "./data/Hotels_Vector.json";
40+
}

‎ai/select-algorithm-dotnet/README.md‎

Lines changed: 76 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -17,60 +17,77 @@ Demonstrates three vector index algorithms available in Azure DocumentDB (vCore)
1717

1818
## Setup
1919

20-
1. Copy the environment file and fill in your values:
20+
1. **Configure environment:**
21+
22+
The .NET sample uses `appsettings.json` for configuration. After deploying with `azd up`, you can export values:
2123

2224
```bash
23-
cp .env.example .env
25+
azd env get-values
26+
```
27+
28+
Then update `appsettings.json` with your Azure resource values.
29+
30+
2. Edit `appsettings.json` with your configuration:
31+
32+
```json
33+
{
34+
"AzureOpenAI": {
35+
"EmbeddingModel": "text-embedding-3-small",
36+
"EmbeddingEndpoint": "https://<your-resource>.openai.azure.com"
37+
},
38+
"DocumentDB": {
39+
"ClusterName": "<your-cluster-name>",
40+
"DatabaseName": "Hotels"
41+
},
42+
"Algorithm": "all",
43+
"Similarity": "COS"
44+
}
2445
```
2546

26-
2. Edit `.env` with your configuration:
47+
3. Copy the data file:
2748

28-
```env
29-
AZURE_OPENAI_EMBEDDING_MODEL=text-embedding-3-small
30-
AZURE_OPENAI_EMBEDDING_ENDPOINT=https://<your-resource>.openai.azure.com
31-
MONGO_CLUSTER_NAME=<your-cluster-name>
32-
AZURE_DOCUMENTDB_DATABASENAME=Hotels
33-
ALGORITHM=all
34-
SIMILARITY=COS
49+
Copy `Hotels_Vector.json` from the repository's `ai/data/` folder into this project's `data/` folder:
50+
51+
```bash
52+
cp ../../data/Hotels_Vector.json ./data/
3553
```
3654

37-
3. Restore packages:
55+
4. Restore packages:
3856

3957
```bash
40-
cd src
4158
dotnet restore
4259
```
4360

4461
## Usage
4562

46-
Run all algorithms:
63+
Run all 9 combinations (default):
4764

4865
```bash
49-
cd src
5066
dotnet run
5167
```
5268

5369
Run a specific algorithm:
5470

5571
```bash
56-
# Set in .env: ALGORITHM=ivf | hnsw | diskann | all
57-
dotnet run
72+
dotnet run -- ivf
73+
dotnet run -- hnsw
74+
dotnet run -- diskann
5875
```
5976

6077
## Compare All Algorithms
6178

6279
Run all 9 combinations (3 algorithms × 3 similarity metrics) in a single invocation with a formatted comparison table:
6380

6481
```bash
65-
# Set in .env: ALGORITHM=compare
66-
dotnet run
82+
dotnet run -- compare-all
6783
```
6884

6985
This mode:
70-
- Uses a **single collection** (`hotels`) with 9 vector indexes
86+
- Uses a **single collection** (`hotels`)
7187
- Generates **one embedding** for the query, reused across all searches
72-
- Runs searches **sequentially** with `Stopwatch` timing for fair comparison
73-
- Prints a formatted table with latency, top result, and scores
88+
- For each of 9 algorithm/metric combinations: creates the index → searches → drops the index
89+
- DocumentDB only allows one vector index per kind per field, so indexes are created sequentially
90+
- Prints a formatted comparison table with scores, top results, and key insights
7491

7592
**Additional environment variables for compare mode:**
7693

@@ -80,34 +97,49 @@ This mode:
8097
| `TOP_K` | `3` | Number of results per search |
8198
| `VERBOSE` | `false` | Show detailed per-result output |
8299

83-
**9 Index Combinations:**
84-
85-
| Index Name | Algorithm | Metric | Parameters |
86-
|------------|-----------|--------|------------|
87-
| `vector_ivf_cos` | IVF | COS | numLists=1 |
88-
| `vector_hnsw_cos` | HNSW | COS | m=16, efConstruction=64 |
89-
| `vector_diskann_cos` | DiskANN | COS | maxDegree=32, lBuild=50 |
90-
| `vector_ivf_l2` | IVF | L2 | numLists=1 |
91-
| `vector_hnsw_l2` | HNSW | L2 | m=16, efConstruction=64 |
92-
| `vector_diskann_l2` | DiskANN | L2 | maxDegree=32, lBuild=50 |
93-
| `vector_ivf_ip` | IVF | IP | numLists=1 |
94-
| `vector_hnsw_ip` | HNSW | IP | m=16, efConstruction=64 |
95-
| `vector_diskann_ip` | DiskANN | IP | maxDegree=32, lBuild=50 |
100+
**Output:**
101+
```
102+
====================================================================================================
103+
COMPARISON RESULTS
104+
====================================================================================================
105+
Algorithm Similarity #1 Result #1 Score #2 Result #2 Score Diff
106+
----------------------------------------------------------------------------------------------------
107+
IVF COS Ocean Water Resort &.. 0.6184 Windy Ocean Motel 0.5056 0.1128
108+
IVF L2 Ocean Water Resort &.. 0.8736 Windy Ocean Motel 0.9943 -0.1208
109+
IVF IP Ocean Water Resort &.. 0.6184 Windy Ocean Motel 0.5056 0.1128
110+
...
111+
====================================================================================================
112+
KEY INSIGHTS
113+
====================================================================================================
114+
🎯 Highest #1 score: IVF/COS (0.6184)
115+
📊 Biggest separation: IVF/COS (diff: 0.1128)
116+
🔑 All algorithms return the same top results — algorithm choice
117+
affects performance at scale, not accuracy on small datasets.
118+
📐 COS and IP produce identical scores (normalized embeddings).
119+
📏 L2 scores are distances (lower = closer), not similarities.
120+
====================================================================================================
121+
```
96122

97123
## Project Structure
98124

99125
```
100126
select-algorithm-dotnet/
101-
├── .env.example # Environment variable template
102-
├── README.md # This file
103-
└── src/
104-
├── SelectAlgorithm.csproj # Project file
105-
├── Program.cs # Entry point - dispatches by ALGORITHM env
106-
├── Utils.cs # Shared helpers (connection, embedding, search)
107-
├── IvfDemo.cs # IVF index creation and search
108-
├── HnswDemo.cs # HNSW index creation and search
109-
├── DiskannDemo.cs # DiskANN index creation and search
110-
└── CompareAll.cs # Unified 9-combination comparison runner
127+
├── .devcontainer/
128+
│ └── devcontainer.json # Dev container configuration
129+
├── Models/
130+
│ ├── Configuration.cs # App configuration model
131+
│ └── HotelData.cs # Hotel document model
132+
├── Utilities/
133+
│ └── AzureIdentityTokenHandler.cs # OIDC token handler
134+
├── output/
135+
│ └── compare_all.txt # Sample comparison output
136+
├── AlgorithmRunner.cs # Per-algorithm index + search runner
137+
├── appsettings.json # Configuration file
138+
├── CompareAll.cs # Unified 9-combination comparison runner
139+
├── Program.cs # Entry point - dispatches by ALGORITHM setting
140+
├── README.md # This file
141+
├── SelectAlgorithm.csproj # Project file
142+
└── Utils.cs # Shared helpers (connection, embedding, search)
111143
```
112144

113145
## How It Works

‎ai/select-algorithm-dotnet/data/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Data Files
2+
3+
Copy `Hotels_Vector.json` into this folder before running the sample.
4+
5+
The file is available in the repository at `ai/data/Hotels_Vector.json`.

‎ai/select-algorithm-go/.env.example‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# DocumentDB Configuration
2+
# Name of the DocumentDB cluster (used for passwordless OIDC authentication)
3+
MONGO_CLUSTER_NAME=your-cluster-name
4+
5+
# Azure OpenAI Embedding Configuration
6+
# Azure OpenAI service endpoint URL
7+
AZURE_OPENAI_EMBEDDING_ENDPOINT=https://your-resource.openai.azure.com
8+
9+
# Azure OpenAI embedding model name
10+
AZURE_OPENAI_EMBEDDING_MODEL=text-embedding-3-small
11+
12+
# Azure OpenAI API version for embeddings
13+
AZURE_OPENAI_EMBEDDING_API_VERSION=2023-05-15
14+
15+
# Database name in DocumentDB
16+
AZURE_DOCUMENTDB_DATABASENAME=Hotels
17+
18+
# Path to JSON file with generated vector embeddings
19+
DATA_FILE_WITH_VECTORS=./data/Hotels_Vector.json
20+
21+
# Name of the field where embeddings are stored
22+
EMBEDDED_FIELD=contentVector
23+
24+
# Number of dimensions in the embedding vectors (1536 for text-embedding-3-small)
25+
EMBEDDING_DIMENSIONS=1536
26+
27+
# Number of records to load per batch during data insertion
28+
LOAD_SIZE_BATCH=100
29+
30+
# Algorithm to run: "all", "ivf", "hnsw", or "diskann"
31+
ALGORITHM=all
32+
33+
# Vector similarity metric: "COS" (cosine), "L2" (Euclidean), or "IP" (inner product)
34+
SIMILARITY=COS
35+
36+
# Notes:
37+
# 1. Replace all placeholder values with your actual Azure resource information
38+
# 2. For production, use Azure Key Vault or environment variables instead of storing keys in files
39+
# 3. The EMBEDDING_DIMENSIONS must match your chosen embedding model:
40+
# - text-embedding-3-small: 1536 dimensions
41+
# - text-embedding-3-large: 3072 dimensions
42+
# 4. Adjust batch sizes based on your API rate limits and performance requirements
43+
# 5. For passwordless authentication, ensure your Azure identity has appropriate RBAC permissions

‎ai/select-algorithm-go/README.md‎

Lines changed: 27 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -38,18 +38,26 @@ This sample demonstrates how to compare different vector search algorithms (IVF,
3838
AZURE_OPENAI_EMBEDDING_ENDPOINT=https://your-resource.openai.azure.com
3939
AZURE_OPENAI_EMBEDDING_MODEL=text-embedding-3-small
4040
AZURE_DOCUMENTDB_DATABASENAME=Hotels
41-
DATA_FILE_WITH_VECTORS=../../data/Hotels_Vector.json
41+
DATA_FILE_WITH_VECTORS=./data/Hotels_Vector.json
4242
EMBEDDED_FIELD=contentVector
4343
EMBEDDING_DIMENSIONS=1536
4444
```
4545

46-
3. **Install dependencies**:
46+
3. **Copy the data file:**
47+
48+
Copy `Hotels_Vector.json` from the repository's `ai/data/` folder into this project's `data/` folder:
49+
50+
```bash
51+
cp ../../data/Hotels_Vector.json ./data/
52+
```
53+
54+
4. **Install dependencies**:
4755

4856
```bash
4957
go mod download
5058
```
5159

52-
4. **Sign in to Azure** (for passwordless authentication):
60+
5. **Sign in to Azure** (for passwordless authentication):
5361

5462
```bash
5563
az login
@@ -62,7 +70,7 @@ This sample demonstrates how to compare different vector search algorithms (IVF,
6270
Run all 9 combinations (3 algorithms × 3 similarity metrics) in a single execution:
6371

6472
```bash
65-
go run compare_all.go utils.go
73+
go run ./src/...
6674
```
6775

6876
This creates indexes sequentially (create/search/drop per combo — DocumentDB allows one vector index per kind per field) and prints a comparison table showing scores and top results.
@@ -100,22 +108,22 @@ Test a specific algorithm with cosine similarity:
100108

101109
```bash
102110
# IVF (Inverted File) — clustering-based, works on all tiers
103-
go run ivf.go utils.go
111+
go run src/ivf.go src/utils.go
104112

105113
# HNSW (Hierarchical Navigable Small World) — graph-based, higher recall
106-
go run hnsw.go utils.go
114+
go run src/hnsw.go src/utils.go
107115

108116
# DiskANN — disk-optimized, best for large datasets (requires M40+ tier)
109-
go run diskann.go utils.go
117+
go run src/diskann.go src/utils.go
110118
```
111119

112120
### On Windows (PowerShell)
113121

114122
```powershell
115-
go run compare_all.go utils.go
116-
go run ivf.go utils.go
117-
go run hnsw.go utils.go
118-
go run diskann.go utils.go
123+
go run ./src/...
124+
go run src/ivf.go src/utils.go
125+
go run src/hnsw.go src/utils.go
126+
go run src/diskann.go src/utils.go
119127
```
120128

121129
## Environment Variables
@@ -126,7 +134,7 @@ go run diskann.go utils.go
126134
| `AZURE_OPENAI_EMBEDDING_ENDPOINT` | *(required)* | Azure OpenAI endpoint |
127135
| `AZURE_OPENAI_EMBEDDING_MODEL` | `text-embedding-3-small` | Embedding model name |
128136
| `AZURE_DOCUMENTDB_DATABASENAME` | `Hotels` | Database name |
129-
| `DATA_FILE_WITH_VECTORS` | `../../data/Hotels_Vector.json` | Path to data file |
137+
| `DATA_FILE_WITH_VECTORS` | `./data/Hotels_Vector.json` | Path to data file |
130138
| `EMBEDDED_FIELD` | `contentVector` | Field containing embeddings |
131139
| `EMBEDDING_DIMENSIONS` | `1536` | Embedding vector dimensions |
132140
| `QUERY_TEXT` | `luxury hotel near the beach` | Search query |
@@ -167,12 +175,14 @@ select-algorithm-go/
167175
├── .env.example # Environment variable template
168176
├── go.mod # Go module dependencies
169177
├── go.sum # Go module checksums
178+
├── output/ # Sample output files
170179
├── README.md # This file
171-
├── utils.go # Shared config, auth, data, and search helpers
172-
├── compare_all.go # Unified 9-combination comparison runner (create/search/drop)
173-
├── ivf.go # IVF algorithm demonstration
174-
├── hnsw.go # HNSW algorithm demonstration
175-
└── diskann.go # DiskANN algorithm demonstration
180+
└── src/
181+
├── utils.go # Shared config, auth, data, and search helpers
182+
├── compare_all.go # Unified 9-combination comparison runner (create/search/drop)
183+
├── ivf.go # IVF algorithm demonstration
184+
├── hnsw.go # HNSW algorithm demonstration
185+
└── diskann.go # DiskANN algorithm demonstration
176186
```
177187

178188
## Authentication

‎ai/select-algorithm-go/data/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Data Files
2+
3+
Copy `Hotels_Vector.json` into this folder before running the sample.
4+
5+
The file is available in the repository at `ai/data/Hotels_Vector.json`.

0 commit comments

Comments
 (0)