Cost optimization

[poissoncorp]

Does Hindsight currently provide any guidance on how to tune the “thinking valve” to control token usage and cost?

I’d love to better understand the trade-offs between reasoning depth, performance, and tokens spent. Some practical recommendations, example configurations, or cost-vs-quality benchmark charts would be really valuable.

Awesome work on the tech, by the way ⚡

[nicoloboschi]

• use a relative small model on retain/consolidation
• use mental models instead of reflect for repetitive queries
• use mental models with update mode=delta
• on client side, tune the max_tokens on recall - 4096 usually has already a lot of information - don't include chunks if you don't need specific quotes
• retain documents in batch to optimize consolidation performances