books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix MHAEinsum weight dimension bug when d_in != d_out (#857) (#893)

Browse Source 
* Fix MHAEinsum weight dimension bug when d_in != d_out (#857)

Previously MHAEinsum initialized weight matrices with shape (d_out, d_in) and used inappropriate einsum notation, causing failures for non-square input-output dimensions. This commit corrects weight initialization to shape (d_in, d_out), updates einsum notation to 'bnd,do->bno', and adds three unit tests to verify parity across different d_in and d_out settings. All tests pass successfully.

* use pytest

* Update .gitignore

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
This commit is contained in:
Aviral Garg
2025-11-01 08:15:31 +05:30
committed by GitHub
parent b1db33b384
commit 27d52d6378
4 changed files with 80 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@@ -328,3 +328,5 @@ cython_debug/
# pixi environments
.pixi
*.egg-info