books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

align formulas in notes with code (#605)

Browse Source
This commit is contained in:
Sebastian Raschka
2025-04-06 16:46:53 -05:00
committed by GitHub
parent 67e0680210
commit 3654571184
1 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
ch03/01_main-chapter-code/ch03.ipynb
View File

@@ -797,9 +797,9 @@
"- Implementing the self-attention mechanism step by step, we will start by introducing the three training weight matrices $W_q$, $W_k$, and $W_v$\n", "- Implementing the self-attention mechanism step by step, we will start by introducing the three training weight matrices $W_q$, $W_k$, and $W_v$\n",
"- These three matrices are used to project the embedded input tokens, $x^{(i)}$, into query, key, and value vectors via matrix multiplication:\n", "- These three matrices are used to project the embedded input tokens, $x^{(i)}$, into query, key, and value vectors via matrix multiplication:\n",
"\n", "\n",
" - Query vector: $q^{(i)} = W_q \\,x^{(i)}$\n", " - Query vector: $q^{(i)} = x^{(i)}\\,W_q $\n",
" - Key vector: $k^{(i)} = W_k \\,x^{(i)}$\n", " - Key vector: $k^{(i)} = x^{(i)}\\,W_k $\n",
" - Value vector: $v^{(i)} = W_v \\,x^{(i)}$\n" " - Value vector: $v^{(i)} = x^{(i)}\\,W_v $\n"
] ]
}, },
{ {