books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix MHAEinsum weight dimension bug when d_in != d_out (#857) (#893)

Browse Source 
* Fix MHAEinsum weight dimension bug when d_in != d_out (#857)

Previously MHAEinsum initialized weight matrices with shape (d_out, d_in) and used inappropriate einsum notation, causing failures for non-square input-output dimensions. This commit corrects weight initialization to shape (d_in, d_out), updates einsum notation to 'bnd,do->bno', and adds three unit tests to verify parity across different d_in and d_out settings. All tests pass successfully.

* use pytest

* Update .gitignore

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
This commit is contained in:
Aviral Garg
2025-11-01 08:15:31 +05:30
committed by GitHub
parent b1db33b384
commit 27d52d6378
4 changed files with 80 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.github/workflows/basic-tests-linux-uv.yml vendored
View File

@@ -48,6 +48,7 @@ jobs:
run: |
source .venv/bin/activate
pytest setup/02_installing-python-libraries/tests.py
pytest ch03/02_bonus_efficient-multihead-attention/tests/test_mha_implementations.py
pytest ch04/01_main-chapter-code/tests.py
pytest ch04/03_kv-cache/tests.py
pytest ch05/01_main-chapter-code/tests.py