books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

use attn_scores from sec 3.4 instead of 3.3

Browse Source
This commit is contained in:
rasbt
2024-02-14 20:23:59 -06:00
parent 231a854ae7
commit 250e6306e2
4 changed files with 45 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
ch04/01_main-chapter-code/ch04.ipynb
View File

@@ -836,7 +836,7 @@
" x = self.drop_resid(x)\n",
" x = x + shortcut # Add the original input back\n",
"\n",
" # Shortcut connection for feed-forward block\n",
" # Shortcut connection for feed forward block\n",
" shortcut = x\n",
" x = self.norm2(x)\n",
" x = self.ff(x)\n",