books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #27 from Intelligence-Manifesto/patch-4

Browse Source 
12 -> 21
This commit is contained in:
Sebastian Raschka
2024-02-11 07:31:06 -06:00
committed by GitHub
parent baa8617921 1278615c25
commit e0b6fdbc53
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
ch03/01_main-chapter-code/ch03.ipynb
View File

@@ -132,7 +132,7 @@
"\n",
"- **Step 1:** compute unnormalized attention scores $\\omega$.\n",
"- Suppose we use the second input token as the query, that is, $q^{(2)} = x^{(2)}$, we compute the unnormalized attention scores via dot products:\n",
" - $\\omega_{12} = x^{(1)} q^{(2)\\top}$\n",
" - $\\omega_{21} = x^{(1)} q^{(2)\\top}$\n",
" - $\\omega_{22} = x^{(2)} q^{(2)\\top}$\n",
" - $\\omega_{23} = x^{(3)} q^{(2)\\top}$\n",
" - ...\n",