Skip to content

Commit ba7503f

Browse files
committed
Clarify latency reduction benefit for self-hosting in README prompt caching note
1 parent f1915d5 commit ba7503f

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ Got tired with copy and paste? Autofill prompts with the browser extension:
7777
From code editing to voice input and commit messages, CWC supports your software development efforts on every step. Bring Your Own Key for a remote model provider of choice. Self-host with [Ollama](https://ollama.com/search).
7878

7979
> [!NOTE]
80-
> Thanks to smart context files ordering, CWC heavily utilizes [prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching/). This reduces input token costs by up to 90% and latency by up to 80%.
80+
> Thanks to smart context files ordering, CWC heavily utilizes [prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching/). This reduces input token costs by up to 90% and, especially when self-hosting, latency by up to 80%.
8181
8282
> [!IMPORTANT]
8383
> Not all providers equally offer discounted input tokens. [Google](https://aistudio.google.com/api-keys) and [OpenAI](https://platform.openai.com/settings/organization/api-keys) are your best bets.

0 commit comments

Comments
 (0)