This repository was archived by the owner on Dec 1, 2024. It is now read-only.

This repository was archived by the owner on Dec 1, 2024. It is now read-only.

How to split execution of prefill and decode for Flexgen? #138

Open

opened

I was wondering what would be the best way to split the execution of flexgen to prefill and decode only.

How should I save the values from prefill and how should I load them when I am running flexgen again for decode only.

Thanks

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests