Skip to content

Model-aware sliding window context #108

Description

@nrfulton

Contexts are typically used via a MelleaSession. In that setting, contexts are "bound" to a specific model_id. When we know the model_id, we can lookup the max context length and then use this context length to determine when the Context's sliding window needs to move. We should implement a DynamicLinearContext that provides this behavior, or maybe even just change the LinearContext to do this dynamic window management based upon models' max context length.

Metadata

Metadata

Labels

area/contextChatContext, message history, context windowing, serializationarea/stdlibCore abstractions: Context, MOT, SamplingStrategy, formatters, serializationenhancementNew feature or requestgood first issueGood for newcomersp2Medium/low: minor bugs, niche features, polish, docs, tests, cleanup. Scoped, lower urgency.

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions