Entity
KV Cache
Key-Value caching mechanism in transformer models that stores previous attention computations to accelerate sequential generation
术语属性
- type：Optimization
- purpose：Inference acceleration
- impact：Memory efficiency
Entity
Key-Value caching mechanism in transformer models that stores previous attention computations to accelerate sequential generation