books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

Remove reundant dropout in MLP module (#105)

Browse Source
This commit is contained in:
Sebastian Raschka
2024-04-03 20:19:08 -05:00
committed by GitHub
parent dd115c1374
commit 3829ccdb34
11 changed files with 202 additions and 266 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
ch04/01_main-chapter-code/gpt.py
View File

@@ -144,7 +144,6 @@ class FeedForward(nn.Module):
nn.Linear(cfg["emb_dim"], 4 * cfg["emb_dim"]),
GELU(),
nn.Linear(4 * cfg["emb_dim"], cfg["emb_dim"]),
nn.Dropout(cfg["drop_rate"])
)
def forward(self, x):