Skip to content

Commit b9dfbdc

Browse files
committed
Update README
Co-authored-by: Isaac
1 parent c7aab93 commit b9dfbdc

1 file changed

Lines changed: 38 additions & 48 deletions

File tree

  • knowledge_base/vector_search_product_discovery

knowledge_base/vector_search_product_discovery/README.md

Lines changed: 38 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,18 @@
11
# Vector Search: Semantic Product Discovery
22

3-
A Declarative Automation Bundle demonstrating **semantic product search** using
3+
A Declarative Automation Bundle demonstrating semantic product search using
44
[Databricks Vector Search](https://docs.databricks.com/en/generative-ai/vector-search.html).
5+
It automates the full setup — the Unity Catalog schema, the Vector Search endpoint and
6+
index, and the jobs that load and query the catalog — so a single `databricks bundle deploy`
7+
gives you a working semantic-search example to explore and adapt.
58

6-
## The problem
9+
## How it works
710

811
Keyword search fails when shoppers use different words than what appears in product
9-
descriptions. A customer searching for *"something to keep my coffee hot all day"* won't
10-
match a product described as an *"insulated stainless water bottle with double-wall vacuum
11-
insulation"* — even though it's the right answer.
12-
13-
Semantic search using vector embeddings matches on **meaning**, not words.
14-
15-
## How it works
12+
descriptions. A customer searching for "something to keep my coffee hot all day" won't
13+
match a product described as an "insulated stainless water bottle with double-wall vacuum
14+
insulation" even though it's the right answer. Semantic search using vector embeddings
15+
matches on meaning, not words.
1616

1717
Product descriptions are embedded at upsert time by the setup job using
1818
[`databricks-gte-large-en`](https://docs.databricks.com/en/machine-learning/foundation-models/supported-models.html).
@@ -27,48 +27,56 @@ product_index (Direct Access Vector Search index)
2727
ranked results
2828
```
2929

30-
## Bundle resources
30+
## Project structure
3131

32-
| Resource | Type | Description |
33-
|---|---|---|
34-
| `product_search_schema` | `schemas` | Unity Catalog schema that namespaces the index |
35-
| `product_search_endpoint` | `vector_search_endpoints` | Managed ANN serving endpoint |
36-
| `product_index` | `vector_search_indexes` | Direct Access index — schema defined in `resources/index.yml` |
37-
| `product_discovery_setup` | `jobs` | Embeds product descriptions and upserts into the index |
38-
| `product_discovery_query` | `jobs` | Embeds a query and returns ranked results |
32+
```
33+
.
34+
├── databricks.yml # Bundle name, variables, and the deploy target
35+
├── data/
36+
│ └── products.json # Product catalog — synced to the workspace on deploy
37+
├── resources/
38+
│ ├── schema.yml # Unity Catalog schema that namespaces the index
39+
│ ├── vector-search-endpoint.yml # Vector Search endpoint (managed ANN serving)
40+
│ ├── vector-search-index.yml # Direct Access index — schema defined inline
41+
│ ├── setup-job.yml # Job: embed product descriptions and upsert them
42+
│ └── query-job.yml # Job: embed a query and return ranked results
43+
└── src/
44+
├── 01_upsert_products.py # Reads products.json, embeds, calls upsert_data
45+
└── 02_query_demo.py # Semantic search — runs as a job or interactively
46+
```
3947

4048
## Prerequisites
4149

4250
- Databricks workspace with Unity Catalog enabled
43-
- Databricks CLI that supports `vector_search_endpoints` / `vector_search_indexes` as bundle resources
51+
- Databricks CLI version 1.1.0 or above
4452
- An existing Unity Catalog catalog (default: `main`)
4553

46-
## Quick start
54+
## Usage
4755

48-
1. **Authenticate**
56+
1. Authenticate the CLI:
4957
```bash
5058
databricks auth login --host https://your-workspace.cloud.databricks.com
5159
```
5260

53-
2. **Configure** `databricks.yml` — set the workspace host and any variable overrides
61+
2. Configure `databricks.yml`. Set the workspace host and any variable overrides.
5462

55-
3. **Deploy**creates the schema, endpoint, index, jobs, and syncs `data/products.json`
63+
3. Deploy the bundle. This creates the schema, endpoint, index, jobs, and syncs `data/products.json`.
5664
```bash
5765
databricks bundle deploy
5866
```
5967
> Vector Search endpoint creation takes a few minutes to reach ONLINE status.
6068
61-
4. **Load the catalog**embeds all product descriptions and upserts them into the index
69+
4. Load the catalog by running the bundle. This embeds all product descriptions and upserts them into the index.
6270
```bash
6371
databricks bundle run product_discovery_setup
6472
```
6573

66-
5. **Search** — pass any natural-language query
74+
5. Pass any natural-language query to search.
6775
```bash
6876
databricks bundle run product_discovery_query --params "query=footwear for slippery wet trails"
6977
```
7078

71-
6. **Or open** `src/02_query_demo.py` in your workspace to run queries interactively
79+
6. Or open `src/02_query_demo.py` in your workspace to run queries interactively.
7280

7381
## Configuration
7482

@@ -89,15 +97,15 @@ databricks bundle deploy \
8997
| `schema` | `product_search` | Schema created by the bundle |
9098
| `endpoint_name` | `product-search-endpoint` | Vector Search endpoint name (must be unique per workspace) |
9199
| `embedding_model` | `databricks-gte-large-en` | Foundation model used for embeddings |
92-
| `embedding_dimension` | `1024` | Vector dimension — must match `embedding_dimension` in `resources/index.yml` |
100+
| `embedding_dimension` | `1024` | Vector dimension — must match `embedding_dimension` in `resources/vector-search-index.yml` |
93101

94-
> **Note:** `embedding_dimension` in `resources/index.yml` is hardcoded to `1024` because
102+
> **Note:** `embedding_dimension` in `resources/vector-search-index.yml` is hardcoded to `1024` because
95103
> it is immutable after index creation. If you need a different dimension, change the value
96-
> in `index.yml` before the first deploy.
104+
> in `vector-search-index.yml` before the first deploy.
97105
98106
## Index schema
99107

100-
The index schema lives entirely in `resources/index.yml`:
108+
The index schema lives entirely in `resources/vector-search-index.yml`:
101109

102110
```yaml
103111
direct_access_index_spec:
@@ -130,25 +138,7 @@ records enter the index via `upsert_data`. If you already have a pipeline writin
130138
Delta table, a **Delta Sync** index is often simpler — you point the index at the source
131139
table and it keeps itself up to date. Replace `index_type: DIRECT_ACCESS` and
132140
`direct_access_index_spec` with `index_type: DELTA_SYNC` and `delta_sync_index_spec` in
133-
`resources/index.yml`, and remove the upsert job.
134-
135-
## Project structure
136-
137-
```
138-
.
139-
├── databricks.yml
140-
├── data/
141-
│ └── products.json # Product catalog — synced to workspace on deploy
142-
├── resources/
143-
│ ├── schema.yml # Unity Catalog schema
144-
│ ├── endpoint.yml # Vector Search endpoint
145-
│ ├── index.yml # Direct Access index
146-
│ ├── setup_job.yml # Embed + upsert job
147-
│ └── query_demo.yml # Query job (--params "query=...")
148-
└── src/
149-
├── 01_upsert_products.py # Reads products.json, embeds, calls upsert_data
150-
└── 02_query_demo.py # Semantic search — runs as job or interactively
151-
```
141+
`resources/vector-search-index.yml`, and remove the upsert job.
152142

153143
## Resources
154144

0 commit comments

Comments
 (0)