Skip to content

Commit 2d86d27

Browse files
authored
Merge pull request #613 from Portkey-AI/mintlify/semantic-caching-vector-db-note-24954
Add vector database requirement note to semantic caching docs
2 parents dbd0800 + 0e2f1e4 commit 2d86d27

1 file changed

Lines changed: 5 additions & 1 deletion

File tree

product/ai-gateway/cache-simple-and-semantic.mdx

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: "Cache (Simple & Semantic)"
44

55
<Info>
66
**Simple** caching is available for all plans.<br />
7-
**Semantic** caching is available for [**Production**](https://portkey.ai/pricing) and [**Enterprise**](https://portkey.ai/docs/product/enterprise-offering) users.
7+
**Semantic** caching requires a vector database and is only available on select Enterprise plans. [Contact us](https://portkey.ai/docs/support/contact-us) to learn more about enabling this feature.
88
</Info>
99

1010
Speed up and save money on your LLM requests by storing past responses in the Portkey cache. There are 2 cache modes:
@@ -42,6 +42,10 @@ Simple cache performs an exact match on the input prompts. If the exact same req
4242
"cache": { "mode": "semantic" }
4343
```
4444
45+
<Note>
46+
Semantic caching requires a vector database and is only available on select Enterprise plans. [Contact us](https://portkey.ai/docs/support/contact-us) to learn more about enabling this feature.
47+
</Note>
48+
4549
### How it Works
4650
4751
Semantic cache considers the contextual similarity between input requests. It uses cosine similarity to ascertain if the similarity between the input and a cached request exceeds a specific threshold. If the similarity threshold is met, Portkey retrieves the response from the cache, saving model execution time. Check out this [blog](https://portkey.ai/blog/reducing-llm-costs-and-latency-semantic-cache/) for more details on how we do this.

0 commit comments

Comments
 (0)