books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,038 Commits 12 Branches 0 Tags
9320a5e25216e8ea4b1d6528081e2edbf1bd50f2
Commit Graph

1038 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
casinca
9320a5e252 fix: added KVcache in generate_text_basic_stream (#981) 2026-03-20 19:47:52 -05:00
rasbt
130cc1f63c harded the link checker 2026-03-07 17:05:47 -06:00
Sebastian Raschka
9ab6e894ac Minor typo fix (#974) 2026-03-07 16:31:40 -06:00
Sebastian Raschka
052c2dea4f Bpe whitespace fixes (#975) 2026-03-07 13:56:25 -06:00
Sebastian Raschka
3a7b98a36a Add more analysis to qwen3.5 image 2026-03-04 08:47:06 -06:00
Sebastian Raschka
ae8eebf0d7 Use full HF url 2026-03-03 16:38:05 -06:00
Sebastian Raschka
7892ec9435 Qwen3.5 from scratch (#969) 
* Qwen3.5 from scratch

* update

* update
 2026-03-03 16:31:16 -06:00
rasbt
4612d20fa8 User argpars utils to show default args on command line 2026-03-01 20:15:21 -06:00
Sebastian Raschka
c079904491 Jupyter scrolling glitch tips (#965) 2026-02-27 17:33:39 -06:00
rasbt
ec78de32dc image size 2026-02-19 16:42:19 -06:00
rasbt
10bffd62b7 image size 2026-02-19 16:41:43 -06:00
rasbt
c745ded43d formatting fix 2026-02-19 16:40:28 -06:00
Sebastian Raschka
62f0356e0d Add Tiny Aya from scratch (#962) 2026-02-19 16:33:22 -06:00
Sebastian Raschka
1ed48c2450 remove redundant assignment (#961) 2026-02-18 22:03:49 -06:00
Sebastian Raschka
2d600ccb5b Use correct input in layernorm example (#960) 
* Update CI

* Use correct example in layernorm section

* update
 2026-02-18 21:35:57 -06:00
Sebastian Raschka
be5e2a3331 Readability and code quality improvements (#959) 
* Consistent dataset naming

* consistent section headers
 2026-02-17 18:44:56 -06:00
Sebastian Raschka
7b1f740f74 Fix flex attention in PyTorch 2.10 (#957) 2026-02-09 14:12:40 -06:00
Dawid Woźniak
82010e2c77 Fix docstring parameter names in compute_dpo_loss function (#953) 2026-01-29 16:51:17 -06:00
Sebastian Raschka
e155d1b02c Update unit tests for CI (#952) 
* Update CI

* Revert submodule pointer update

* Update

* update

* update
 2026-01-27 17:44:55 -06:00
Pádraic Slattery
59d9262047 chore: Update outdated GitHub Actions versions (#951) 2026-01-19 12:22:29 -06:00
Sebastian Raschka
47cfc61800 link GRPO notebook (#950) 2026-01-18 11:42:03 -06:00
casinca
9c4be478f8 Optional weight tying for Qwen3 and Llama3.2 pretraining (#949) 
* optional weight tying for Qwen3 and Llama3.2

* typo
 2026-01-14 09:07:04 -06:00
Maxwell De Jong
e0dbec3331 Fix encoding of multiple preceding spaces in BPE tokenizer. (#945) 
* Fix encoding of multiple preceding spaces in BPE tokenizer.

* Add test

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
 2026-01-10 10:27:23 -06:00
Sebastian Raschka
90e0f3cc15 Chapter 5 with alternative LLMs (Qwen3, Llama 3) (#943) 
* ch05-alt-llms

* fix double import

* style fix

* Fix urls

* Fix urls

* Fix urls
 2026-01-09 14:58:20 -06:00
Henry
9df9e69cd2 Correct batch_idx in appendix A logging (#942) 
* Correct batch_idx in appendix A logging

* update batch idx

---------

Co-authored-by: Sebastian Raschka <sebastian@Sebastians-MacBook-Air.local>
 2026-01-07 19:14:17 -06:00
Gerardo Moreno
491fd58463 Fix Olmo3 YaRN RoPE implementation bug (#940) 
* Olmo3 fix RoPE YaRN implementation

* Update cell outputs

* update olmo layer debugger

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
 2026-01-03 18:59:57 -06:00
Maheshkumar P
b26fa01381 Correct 'pix' to 'pixi' in README.md (#935) 
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
 2026-01-02 15:17:56 -06:00
Sebastian Raschka
e10af0a1b9 Clean up native-uv.md documentation (#938) 
Removed unnecessary text and whitespace in the documentation.
 2026-01-02 15:17:43 -06:00
Sebastian Raschka
14c7afaa58 Fix GitHub CI timeout issue for link checker (#937) 
* Fix GitHub CI timeout issue for link checker

* update problematic links
 2026-01-02 14:34:31 -06:00
rasbt
5f3268c2a6 yearly update 2026-01-01 18:19:44 -06:00
Sebastian Raschka
d85ba93799 Cover Python 3.12 (#933) 2025-12-27 16:30:13 -06:00
rasbt
1cea30f9b1 upload saved nb 2025-12-20 18:41:34 -06:00
rasbt
2b9a67c00d Update memory efficient loading nb 2025-12-20 18:35:13 -06:00
rasbt
695ecb61ce update submodule 2025-12-20 11:38:06 -06:00
Sebastian Raschka
1c9f49c812 Add some appendix E runtimes (#927) 
* Add some appendix E runtimes

* Use more lightweigth CPU torch

* Use more lightweigth CPU torch

* Use more lightweigth CPU torch
 2025-12-19 14:13:43 -06:00
Sebastian Raschka
57430d2a13 Gated DeltaNet updates (#926) 2025-12-18 20:28:53 -06:00
talentJay-ux
d7f178d28b Sliding window KV Cache bug fix (#925) 
1. Fix bug because of KV cache and GPT's ptr pointer doesn't get reset when window_size > context_length
2. Fix bug because of KV cache and GPT's ptr pointer doesn't get reset
3. Fix KV Cache import issue for gpt_with_kv_cache_optimized
 2025-12-15 18:47:01 -06:00
Sebastian Raschka
a11965fbd9 Remove persistent flag from cache buffers (#916) 2025-11-24 20:10:02 -06:00
Sebastian Raschka
c19533851f Add Olmo 3 README (#915) 
* Add Olmo 3 README

* update links

* update links

* update links
 2025-11-23 10:53:48 -06:00
Sebastian Raschka
bc6f335526 Olmo 3 from scratch (#914) 
* Olmo 3 from scratch

* update

* update

* update
 2025-11-22 22:42:18 -06:00
Sebastian Raschka
398b079efa RoPE decay plot (#910) 
* RoPE decay plot

* RoPE decay plot
 2025-11-17 17:29:49 -06:00
Sebastian Raschka
28a8408d4d Update README wrt multi-query attention 
Clarified the implications of using multi-query attention on modeling performance and memory usage.
 2025-11-17 16:39:32 -06:00
Sebastian Raschka
a4094470c7 Write-up on how to get the most out of this book (#909) 2025-11-12 20:20:48 -06:00
casinca
7d92267170 fix(GatedDeltaNet): Init param A from log of a uniform distrib (#906) 2025-11-09 14:22:52 -06:00
rasbt
35354fac80 Use consistent title case 2025-11-06 15:22:24 -06:00
Sebastian Raschka
58f45ae5a7 Fix empty device issue (#904) 2025-11-05 20:04:44 -06:00
Sebastian Raschka
bcc73f731d n_heads × d_head -> d_head × d_head in DeltaNet (#903) 
Clarified the explanation of the memory size calculation for `KV_cache_DeltaNet` and updated the quadratic term from `n_heads × d_head` to `d_head × d_head`.
 2025-11-05 18:28:37 -06:00
Sebastian Raschka
488bef7e3f Image resizing 2025-11-02 21:05:38 -06:00
Sebastian Raschka
c6b8332a59 Gated DeltaNet write-up (#901) 
* Gated DeltaNet write-up

* Add copyright and source information to script

Added copyright notice and source information.

* Remove unused import of Path in plot_memory_estimates

* Fix url
 2025-11-02 21:03:42 -06:00
Sebastian Raschka
d6c3990c57 Training on MPS in PyTorch 2.9 (#900) 
* Training on MPS in PyTorch 2.9

* update
 2025-11-01 16:55:09 -05:00