books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

update mha dim

Browse Source
This commit is contained in:
rasbt
2024-05-27 07:46:29 -05:00
parent 68123a5433
commit c0f564ee87
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb
View File

@@ -239,7 +239,7 @@
" self.head_dim = d_out // num_heads\n",
"\n",
" self.qkv = nn.Linear(d_in, 3 * d_out, bias=qkv_bias)\n",
" self.proj = nn.Linear(d_in, d_out)\n",
" self.proj = nn.Linear(d_out, d_out)\n",
" self.dropout = nn.Dropout(dropout)\n",
"\n",
" self.register_buffer(\n",