This repository was archived by the owner on Dec 1, 2024. It is now read-only.

This repository was archived by the owner on Dec 1, 2024. It is now read-only.

How to split execution of prefill and decode for Flexgen? #138

Open

opened

I was wondering what would be the best way to split the execution of flexgen to prefill and decode only.

How should I save the values from prefill and how should I load them when I am running flexgen again for decode only.

Thanks

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests