books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

Remove persistent flag from cache buffers (#916)

Browse Source
This commit is contained in:
Sebastian Raschka
2025-11-24 20:10:02 -06:00
committed by GitHub
parent c19533851f
commit a11965fbd9
1 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
ch04/03_kv-cache/README.md
View File

@@ -55,11 +55,11 @@ To summarize the implementation details, here's a short walkthrough.
### 1. Registering the cache buffers
Inside the `MultiHeadAttention` constructor we add two non-persistent buffers, `cache_k` and `cache_v`, which will hold concatenated keys and values across steps:
Inside the `MultiHeadAttention` constructor we add two buffers, `cache_k` and `cache_v`, which will hold concatenated keys and values across steps:
```python
self.register_buffer("cache_k", None, persistent=False)
self.register_buffer("cache_v", None, persistent=False)
self.register_buffer("cache_k", None)
self.register_buffer("cache_v", None)
```