Skip to content

Commit 2f79660

Browse files
authored
Merge branch 'feast-dev:master' into feast-mlflow
2 parents 4aa9d50 + 9b185ee commit 2f79660

68 files changed

Lines changed: 5492 additions & 325 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

docs/SUMMARY.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@
5555
* [Retrieval Augmented Generation (RAG) with Feast](tutorials/rag-with-docling.md)
5656
* [RAG Fine Tuning with Feast and Milvus](../examples/rag-retriever/README.md)
5757
* [MCP - AI Agent Example](../examples/mcp_feature_store/README.md)
58+
* [Feast-Powered AI Agent](../examples/agent_feature_store/README.md)
5859

5960
## How-to Guides
6061

@@ -70,6 +71,7 @@
7071
* [Multi-Team Feature Store Setup](how-to-guides/federated-feature-store.md)
7172
* [Running Feast in production (e.g. on Kubernetes)](how-to-guides/running-feast-in-production.md)
7273
* [Feast on Kubernetes](how-to-guides/feast-on-kubernetes.md)
74+
* [Feast Production Deployment Topologies](how-to-guides/production-deployment-topologies.md)
7375
* [Online Server Performance Tuning](how-to-guides/online-server-performance-tuning.md)
7476
* [Customizing Feast](how-to-guides/customizing-feast/README.md)
7577
* [Adding a custom batch materialization engine](how-to-guides/customizing-feast/creating-a-custom-materialization-engine.md)

docs/getting-started/genai.md

Lines changed: 64 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,53 @@ The transformation workflow typically involves:
5656
3. **Chunking**: Split documents into smaller, semantically meaningful chunks
5757
4. **Embedding Generation**: Convert text chunks into vector embeddings
5858
5. **Storage**: Store embeddings and metadata in Feast's feature store
59+
60+
### DocEmbedder: End-to-End Document Ingestion Pipeline
61+
62+
The `DocEmbedder` class provides an end-to-end pipeline for ingesting documents into Feast's online vector store. It handles chunking, embedding generation, and writing results -- all in a single step.
63+
64+
#### Key Components
65+
66+
* **`DocEmbedder`**: High-level orchestrator that runs the full pipeline: chunk → embed → schema transform → write to online store
67+
* **`BaseChunker` / `TextChunker`**: Pluggable chunking layer. `TextChunker` splits text by word count with configurable `chunk_size`, `chunk_overlap`, `min_chunk_size`, and `max_chunk_chars`
68+
* **`BaseEmbedder` / `MultiModalEmbedder`**: Pluggable embedding layer with modality routing. `MultiModalEmbedder` supports text (via sentence-transformers) and image (via CLIP) with lazy model loading
69+
* **`SchemaTransformFn`**: A user-defined function that transforms the chunked + embedded DataFrame into the format expected by the FeatureView schema
70+
71+
#### Quick Example
72+
73+
```python
74+
from feast import DocEmbedder
75+
import pandas as pd
76+
77+
# Prepare your documents
78+
df = pd.DataFrame({
79+
"id": ["doc1", "doc2"],
80+
"text": ["First document content...", "Second document content..."],
81+
})
82+
83+
# Create DocEmbedder -- automatically generates a FeatureView and applies the repo
84+
embedder = DocEmbedder(
85+
repo_path="feature_repo/",
86+
feature_view_name="text_feature_view",
87+
)
88+
89+
# Embed and ingest documents in one step
90+
result = embedder.embed_documents(
91+
documents=df,
92+
id_column="id",
93+
source_column="text",
94+
column_mapping=("text", "text_embedding"),
95+
)
96+
```
97+
98+
#### Features
99+
100+
* **Auto-generates FeatureView**: Creates a Python file with Entity and FeatureView definitions compatible with `feast apply`
101+
* **Auto-applies repo**: Registers the generated FeatureView in the registry automatically
102+
* **Custom schema transform**: Provide your own `SchemaTransformFn` to control how chunked + embedded data maps to your FeatureView schema
103+
* **Extensible**: Subclass `BaseChunker` or `BaseEmbedder` to plug in your own chunking or embedding strategies
104+
105+
For a complete walkthrough, see the [DocEmbedder tutorial notebook](../../examples/rag-retriever/rag_feast_docembedder.ipynb).
59106
### Feature Transformation for LLMs
60107

61108
Feast supports transformations that can be used to:
@@ -89,6 +136,17 @@ Implement semantic search by:
89136
2. Converting search queries to embeddings
90137
3. Finding semantically similar documents using vector search
91138

139+
### AI Agents with Context and Memory
140+
141+
Feast can serve as both the **context provider** and **persistent memory layer** for AI agents. Unlike stateless RAG pipelines, agents make autonomous decisions about which tools to call and can write state back to the feature store:
142+
143+
1. **Structured context**: Retrieve customer profiles, account data, and other entity-keyed features
144+
2. **Knowledge retrieval**: Search vector embeddings for relevant documents
145+
3. **Persistent memory**: Store and recall per-entity interaction history (last topic, resolution, preferences) using `write_to_online_store`
146+
4. **Governed access**: All reads and writes are subject to the same RBAC, TTL, and audit policies as any other feature
147+
148+
With MCP enabled, agents built with any framework (LangChain, LlamaIndex, CrewAI, AutoGen, or custom) can discover and call Feast tools dynamically. See the [Feast-Powered AI Agent example](../../examples/agent_feature_store/) and the blog post [Building AI Agents with Feast](https://feast.dev/blog/feast-agents-mcp/) for a complete walkthrough.
149+
92150
### Scaling with Spark Integration
93151

94152
Feast integrates with Apache Spark to enable large-scale processing of unstructured data for GenAI applications:
@@ -167,20 +225,25 @@ The MCP integration uses the `fastapi_mcp` library to automatically transform yo
167225
The fastapi_mcp integration automatically exposes your Feast feature server's FastAPI endpoints as MCP tools. This means AI assistants can:
168226

169227
* **Call `/get-online-features`** to retrieve features from the feature store
228+
* **Call `/retrieve-online-documents`** to perform vector similarity search
229+
* **Call `/write-to-online-store`** to persist agent state (memory, notes, interaction history)
170230
* **Use `/health`** to check server status
171231

172-
For a complete example, see the [MCP Feature Store Example](../../examples/mcp_feature_store/).
232+
For a basic MCP example, see the [MCP Feature Store Example](../../examples/mcp_feature_store/). For a full agent with persistent memory, see the [Feast-Powered AI Agent Example](../../examples/agent_feature_store/).
173233

174234
## Learn More
175235

176236
For more detailed information and examples:
177237

178238
* [Vector Database Reference](../reference/alpha-vector-database.md)
179239
* [RAG Tutorial with Docling](../tutorials/rag-with-docling.md)
240+
* [DocEmbedder Tutorial Notebook](../../examples/rag-retriever/rag_feast_docembedder.ipynb)
180241
* [RAG Fine Tuning with Feast and Milvus](../../examples/rag-retriever/README.md)
181242
* [Milvus Quickstart Example](https://github.com/feast-dev/feast/tree/master/examples/rag/milvus-quickstart.ipynb)
182243
* [Feast + Ray: Distributed Processing for RAG Applications](https://feast.dev/blog/feast-ray-distributed-processing/)
183244
* [MCP Feature Store Example](../../examples/mcp_feature_store/)
245+
* [Feast-Powered AI Agent Example (with Memory)](../../examples/agent_feature_store/)
246+
* [Blog: Building AI Agents with Feast](https://feast.dev/blog/feast-agents-mcp/)
184247
* [MCP Feature Server Reference](../reference/feature-servers/mcp-feature-server.md)
185248
* [Spark Data Source](../reference/data-sources/spark.md)
186249
* [Spark Offline Store](../reference/offline-stores/spark.md)

docs/how-to-guides/feast-on-kubernetes.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,10 @@ Kubernetes is a common target environment for running Feast in production. You c
1010
2. Run scheduled and ad-hoc jobs (e.g. materialization jobs) as Kubernetes Jobs.
1111
3. Operate Feast components using Kubernetes-native primitives.
1212

13+
{% hint style="info" %}
14+
**Planning a production deployment?** See the [Feast Production Deployment Topologies](./production-deployment-topologies.md) guide for architecture diagrams, sample FeatureStore CRs, RBAC policies, infrastructure recommendations, and scaling best practices across Minimal, Standard, and Enterprise topologies.
15+
{% endhint %}
16+
1317
## Feast Operator
1418

1519
To deploy Feast components on Kubernetes, use the included [feast-operator](../../infra/feast-operator).

0 commit comments

Comments
 (0)