books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity
954 Commits 12 Branches 0 Tags
42c130623b28ec260b012f5accc61264e72dbb89
Commit Graph

954 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
casinca
42c130623b Qwen3Tokenizer fix for Qwen3 Base models and generation mismatch with HF (#828) 
* prevent `self.apply_chat_template` being applied for base Qwen models

* - added no chat template comparison in `test_chat_wrap_and_equivalence`
- removed duplicate comparison

* Revert "- added no chat template comparison in `test_chat_wrap_and_equivalence`"

This reverts commit 3a5ee8cfa1.

* Revert "prevent `self.apply_chat_template` being applied for base Qwen models"

This reverts commit df504397a8.

* copied `download_file` in `utils` from https://github.com/rasbt/reasoning-from-scratch/blob/main/reasoning_from_scratch/utils.py

* added copy of test `def test_tokenizer_equivalence()` from `reasoning-from-scratch` in `test_qwen3.py`

* removed duplicate code fragment in`test_chat_wrap_and_equivalence`

* use apply_chat_template

* add toggle for instruct model

* Update tokenizer usage

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
 2025-09-17 08:14:11 -05:00
Synix
bfc6389fab fix code comment (#834) 2025-09-17 01:36:02 +00:00
Sebastian Raschka
b6cd0a312f More efficient angles computation in RoPE (#830) 2025-09-16 03:23:33 +00:00
Sebastian Raschka
147dc49ab5 rename eval method (#832) 2025-09-16 02:47:20 +00:00
Sebastian Raschka
8add26cbe9 Improve weight tying handling (#826) 
* Improve weight tying handling

* fix
 2025-09-14 15:46:48 -05:00
rasbt
1412b139f2 main push to sync github ruleset 2025-09-14 11:59:52 -05:00
Sebastian Raschka
8f3e5b024d Add LoRA scaling (#823) 2025-09-14 11:57:55 -05:00
Sebastian Raschka
fc101b710e Added Apple Silicon GPU device update (#820) 
* Added Apple Silicon GPU device

* Added Apple Silicon GPU device

* delete: remove unused model.pth file from understanding-buffers

* update

* update

---------

Co-authored-by: missflash <missflash@gmail.com>
 2025-09-13 12:48:06 -05:00
Andreas Yin
8e170312fe fix: correct role of the beta hyperparameter on the DPO loss (#818) 
Increasing beta leads to less divergence between the new model and the reference model.
 2025-09-12 20:21:38 -05:00
Sebastian Raschka
32965e0edd remove redundant next_cache (#817) 2025-09-11 15:16:08 -05:00
Sebastian Raschka
c7a4362ca4 Add defensive context trimming for multiturn (#815) 
* Add defensive context trimming for multiturn

* add all mods
 2025-09-09 20:19:00 -05:00
Sebastian Raschka
215abdbcdd Improve multiturn stopping condition (#814) 
* Improve multiturn stopping condition

* improve
 2025-09-09 19:37:15 -05:00
Sebastian Raschka
4b0021416a Clarify Qwen3 notebook purpose (#812) 
* Clarify Qwen3 notebook purpose

* Update README.md

* Update README.md
 2025-09-06 15:31:35 -05:00
Sebastian Raschka
6d175a22df Fix IMDb spelling (#811) 
* Add SSL instructions

* Fix IMDb spelling
 2025-09-06 12:04:47 -05:00
Sebastian Raschka
18c6b970ab Add additional notes on debugging SSL issues (#810) 
* Add SSL instructions

* update old pytorch tests

* update

* update

* update

* update

* update

* update

* update

* update
 2025-09-06 11:46:50 -05:00
Sebastian Raschka
290fa10d55 Update code dependencies (#809) 2025-09-05 16:40:00 -05:00
Sebastian Raschka
5ae41c402e Fix code comment 2025-09-05 14:02:24 -05:00
Sebastian Raschka
623dc65d5d Update requirements for Intel Macs (#807) 2025-09-04 15:07:46 -05:00
Sebastian Raschka
efad18bd0b Fix accidental indentation 2025-09-04 14:41:08 -05:00
Sebastian Raschka
9bfa92fb3e Update README.md 2025-09-03 12:41:32 -05:00
Sebastian Raschka
590d8489d0 Update requirements for Intel macOS (#805) 2025-09-03 12:15:25 -05:00
rasbt
65e67a9681 fix typo 2025-09-02 10:17:51 -05:00
Hayato Hongo
2d8d6224ed added brief explanations about 2 different ways of RoPE implementations (#802) 
* added brief explanations about  2 different ways of RoPE implementations

* improve comment

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
 2025-09-02 10:14:36 -05:00
rasbt
9ea2c57c5f simplify 2025-09-01 22:15:47 -05:00
rasbt
643f800a94 remove local config files 2025-09-01 20:52:40 -05:00
Sebastian Raschka
9eee9296d9 Interactive qwen3 chat interface (#801) 
* Interactive qwen3 chat interface

* update

* update

* update url
 2025-09-01 20:50:25 -05:00
Sebastian Raschka
70edd53809 Improve RoPE (#799) 2025-08-31 11:46:36 -05:00
Sebastian Raschka
d87d91b23c Add KVCache variant of Qwen3 notebook (#800) 
* Add KVCache variant of Qwen3 notebook

* cleanup
 2025-08-31 11:11:12 -05:00
Sebastian Raschka
0e9cefcdc8 Update pixi powershell command section (#798) 2025-08-30 08:42:06 -05:00
Sebastian Raschka
a51ff65488 reasoning-from-scratch (#793) 2025-08-28 18:36:41 -05:00
Jestine Paul
a3a62c509a Improve MHA einsum (#781) 
Efficiency update for einsum as mentioned in #772
 2025-08-22 15:12:26 -05:00
casinca
670f7a4dd0 - added (missing) Gemma3 bullet point in parent folder's readme.md (#788) 
- typo in nbs
 2025-08-22 15:03:47 -05:00
Sebastian Raschka
4a84cfccf9 Minor cosmetic fixes in Gemma 3 nbs (#780) 2025-08-19 21:08:29 -05:00
Sebastian Raschka
f571b5e493 Add Gemma3 KV cache variant (#776) 
* Add Gemma3 KV cache variant

* update
 2025-08-19 12:37:49 -05:00
Sebastian Raschka
8c1f9ccf54 Improve MHA einsum (#775) 2025-08-19 10:38:15 -05:00
Sebastian Raschka
80d4732456 add HF equivalency tests for standalone nbs (#774) 
* add HF equivalency tests for standalone nbs

* update

* update

* update

* update
 2025-08-18 18:58:46 -05:00
Sebastian Raschka
a6b883c9f9 Gemma 3 270M From Scratch (#771) 
* Gemma 3 270M From Scratch

* fix path

* update readme
 2025-08-17 08:23:05 -05:00
Sebastian Raschka
e9c1c1da38 Fix qk_norm comment (#769) 2025-08-15 08:38:48 -05:00
Sebastian Raschka
b14325e56d Qwen3 and Llama3 equivalency teests with HF transformers (#768) 
* Qwen3 and Llama3 equivalency teests with HF transformers

* update
 2025-08-14 18:36:07 -05:00
Sebastian Raschka
5febcf8a1b MoE Nb readability improvements (#761) 2025-08-01 19:58:18 -05:00
Sebastian Raschka
f92b40e4ab Qwen3 Coder Flash & MoE from Scratch (#760) 
* Qwen3 Coder Flash & MoE from Scratch

* update

* refinements

* updates

* update

* update

* update
 2025-08-01 19:13:17 -05:00
casinca
145322ded8 [Minor] Qwen3 typo & optim (#758) 
* typo

* remove weight dict after loading
 2025-07-28 17:29:44 -05:00
Sebastian Raschka
b12dbf6c68 Interleaved Q and K for RoPE in Llama 2 (#750) 2025-07-23 08:02:02 -05:00
Sebastian Raschka
13f049f6a4 Minor typo: pply -> Apply (#749) 2025-07-22 08:19:25 -05:00
Sebastian Raschka
c6472f1af1 Update Python dependency in pyproject.toml (#748) 2025-07-22 08:03:57 -05:00
Sebastian Raschka
3233ddc475 get rid of redundant memory profiler import (#744) 2025-07-16 07:36:51 -05:00
Sebastian Raschka
7e9ce325de Add link to official video course (#741) 2025-07-13 10:35:12 -05:00
Matthew Hernandez
6f12edb0cc Fix issue: 731 by resolving semantic error (#738) 
* fix issue 731

* update test path

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
 2025-07-10 16:30:15 -05:00
Sebastian Raschka
a354555049 Batched KV Cache Inference for Qwen3 (#735) 2025-07-10 08:09:35 -05:00
Sebastian Raschka
b8c8237251 Qwen3 tokenizer sanity checks (#730) 2025-07-09 13:52:35 -05:00