Update needed on Anthropic prompt caching support

[anakin87]

While working on #1940, I found that the support for prompt caching in Anthropic is outdated and does not work as expected.

• When introduced, this feature was in beta and our implementation is based on this aspect (e.g. if specific beta headers are not found, cache support is disabled)
• The prompt caching example does not work.
• Our implementation seems to apply to system messages only, and our successful tests focus on this aspect.

The best solution would be to update our Chat Generator; this would require some effort. Alternatively, rather than keeping a broken or misleading feature, it would be better to (temporarily) remove it altogether.

[vblagoje]

@gulbaki take this one on then 😄

[gulbaki]

@vblagoje Thank you I am taking this😄

[gulbaki]

Hey, @vblagoje @anakin87
I’ve started working on prompt caching and I’m updating it based on the new documentation. By default the TTL is 5 minutes but there’s a beta option to use 1 hour. Should I add support for the 1-hour beta TTL or should I just remove the beta part and keep it working with the current setup?
wdyt?
https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching#1-hour-cache-duration

[vblagoje]

@gulbaki I'd say don't hardcode any of these beta headers but enable user to pass them, @anakin87 ?