Skip to content

Commit a11965f

Browse files
authored
Remove persistent flag from cache buffers (#916)
1 parent c195338 commit a11965f

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

ch04/03_kv-cache/README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,11 +55,11 @@ To summarize the implementation details, here's a short walkthrough.
5555

5656
### 1. Registering the cache buffers
5757

58-
Inside the `MultiHeadAttention` constructor we add two non-persistent buffers, `cache_k` and `cache_v`, which will hold concatenated keys and values across steps:
58+
Inside the `MultiHeadAttention` constructor we add two buffers, `cache_k` and `cache_v`, which will hold concatenated keys and values across steps:
5959

6060
```python
61-
self.register_buffer("cache_k", None, persistent=False)
62-
self.register_buffer("cache_v", None, persistent=False)
61+
self.register_buffer("cache_k", None)
62+
self.register_buffer("cache_v", None)
6363
```
6464

6565
 

0 commit comments

Comments
 (0)