LLMs-from-scratch

books/LLMs-from-scratch

Fork 0

mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00

Commit Graph

Select branches

Hide Pull Requests

cleaning

debug-exclude-newer

dockerfile

fix-qwen3-tokeniizer-eos

fix_masking_kv_cache

gemma-3

head-dim

main

qwen-tokenizer-fix

requirements-update

uv-improvements

weezymatt-bugfix/semantic-error

#1

#10

#100

#1000

#1003

#1004

#1004

#101

#102

#104

#105

#106

#107

#109

#110

#111

#113

#115

#116

#117

#118

#119

#12

#120

#121

#122

#124

#125

#127

#128

#13

#131

#132

#133

#134

#135

#136

#137

#138

#139

#14

#141

#142

#143

#144

#149

#151

#152

#153

#154

#156

#157

#158

#159

#160

#161

#162

#163

#164

#165

#166

#169

#17

#170

#171

#173

#174

#176

#177

#178

#179

#18

#180

#182

#183

#184

#187

#189

#19

#193

#195

#196

#197

#198

#199

#2

#20

#200

#201

#203

#204

#205

#206

#207

#208

#211

#212

#213

#216

#218

#219

#22

#222

#223

#224

#227

#228

#229

#233

#234

#235

#236

#237

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#250

#252

#253

#254

#258

#259

#26

#260

#261

#263

#264

#265

#269

#27

#270

#274

#278

#28

#281

#285

#288

#289

#29

#290

#291

#294

#295

#297

#298

#3

#300

#301

#303

#304

#305

#307

#31

#313

#314

#318

#319

#32

#320

#321

#323

#324

#325

#329

#33

#333

#335

#336

#337

#339

#344

#345

#346

#349

#352

#353

#356

#36

#360

#361

#366

#367

#368

#369

#37

#372

#373

#375

#376

#378

#38

#380

#381

#383

#384

#386

#388

#389

#39

#391

#393

#395

#396

#397

#398

#4

#400

#401

#403

#404

#405

#406

#407

#408

#412

#413

#414

#419

#420

#421

#423

#427

#432

#436

#437

#438

#439

#441

#451

#458

#461

#465

#465

#466

#468

#469

#470

#471

#473

#479

#481

#483

#484

#487

#492

#495

#496

#498

#5

#50

#500

#504

#506

#507

#509

#511

#512

#516

#518

#519

#52

#520

#521

#522

#523

#524

#525

#526

#529

#530

#531

#533

#534

#535

#536

#537

#539

#54

#540

#541

#542

#543

#544

#545

#546

#547

#549

#55

#550

#551

#553

#555

#557

#56

#560

#561

#562

#563

#565

#571

#572

#573

#574

#575

#576

#577

#578

#581

#582

#583

#584

#585

#588

#589

#590

#591

#592

#593

#596

#597

#598

#6

#600

#602

#604

#605

#606

#611

#612

#613

#614

#615

#616

#617

#620

#621

#627

#628

#629

#63

#630

#638

#64

#640

#642

#643

#646

#649

#65

#653

#658

#66

#660

#661

#663

#665

#667

#668

#671

#672

#673

#674

#676

#677

#678

#68

#680

#682

#685

#687

#688

#69

#695

#696

#697

#698

#699

#7

#70

#700

#702

#703

#704

#705

#706

#708

#709

#71

#710

#711

#713

#714

#715

#716

#719

#722

#723

#725

#726

#727

#728

#729

#73

#730

#732

#733

#735

#737

#738

#74

#741

#744

#747

#748

#749

#75

#750

#752

#758

#76

#760

#761

#764

#767

#768

#769

#77

#770

#771

#773

#774

#775

#776

#78

#780

#781

#786

#788

#79

#793

#798

#799

#800

#801

#802

#805

#807

#809

#81

#810

#811

#812

#814

#815

#817

#818

#819

#82

#820

#823

#826

#828

#829

#83

#830

#831

#832

#833

#834

#839

#84

#841

#842

#843

#845

#847

#849

#850

#851

#858

#859

#861

#862

#863

#865

#866

#867

#869

#870

#874

#875

#876

#878

#879

#880

#881

#883

#887

#888

#89

#890

#891

#892

#893

#897

#898

#9

#90

#900

#901

#903

#904

#906

#909

#910

#914

#915

#916

#918

#920

#925

#926

#927

#933

#935

#937

#938

#940

#942

#943

#945

#946

#949

#950

#951

#952

#953

#954

#957

#958

#959

#96

#960

#961

#962

#965

#969

#97

#974

#975

#976

#981

#987

#989

#99

#991

#993

#994

#998

#999

186e83c579 Revert "prevent self.apply_chat_template being applied for base Qwen models" casinca 2025-09-16 09:43:01 +02:00
02a1cb1159 Revert "- added no chat template comparison in test_chat_wrap_and_equivalence" casinca 2025-09-16 09:42:47 +02:00
701b5ad54d Merge branch 'main' into qwen-tokenizer-fix casinca 2025-09-16 09:38:45 +02:00
b6cd0a312f More efficient angles computation in RoPE (#830) Sebastian Raschka 2025-09-15 22:23:33 -05:00
147dc49ab5 rename eval method (#832) Sebastian Raschka 2025-09-15 21:47:20 -05:00
3a5ee8cfa1 - added no chat template comparison in test_chat_wrap_and_equivalence - removed duplicate comparison casinca 2025-09-15 19:30:31 +02:00
df504397a8 prevent self.apply_chat_template being applied for base Qwen models casinca 2025-09-15 16:26:17 +02:00
8add26cbe9 Improve weight tying handling (#826) Sebastian Raschka 2025-09-14 15:46:48 -05:00
1412b139f2 main push to sync github ruleset rasbt 2025-09-14 11:59:52 -05:00
8f3e5b024d Add LoRA scaling (#823) Sebastian Raschka 2025-09-14 11:57:55 -05:00
fc101b710e Added Apple Silicon GPU device update (#820) Sebastian Raschka 2025-09-13 12:48:06 -05:00
8e170312fe fix: correct role of the beta hyperparameter on the DPO loss (#818) Andreas Yin 2025-09-13 03:21:38 +02:00
32965e0edd remove redundant next_cache (#817) Sebastian Raschka 2025-09-11 15:16:08 -05:00
c7a4362ca4 Add defensive context trimming for multiturn (#815) Sebastian Raschka 2025-09-09 20:19:00 -05:00
215abdbcdd Improve multiturn stopping condition (#814) Sebastian Raschka 2025-09-09 19:37:15 -05:00
4b0021416a Clarify Qwen3 notebook purpose (#812) Sebastian Raschka 2025-09-06 15:31:35 -05:00
6d175a22df Fix IMDb spelling (#811) Sebastian Raschka 2025-09-06 12:04:47 -05:00
18c6b970ab Add additional notes on debugging SSL issues (#810) Sebastian Raschka 2025-09-06 11:46:50 -05:00
290fa10d55 Update code dependencies (#809) Sebastian Raschka 2025-09-05 16:40:00 -05:00
5ae41c402e Fix code comment Sebastian Raschka 2025-09-05 14:02:24 -05:00
623dc65d5d Update requirements for Intel Macs (#807) Sebastian Raschka 2025-09-04 15:07:46 -05:00
efad18bd0b Fix accidental indentation Sebastian Raschka 2025-09-04 14:41:08 -05:00
9bfa92fb3e Update README.md Sebastian Raschka 2025-09-03 12:41:32 -05:00
590d8489d0 Update requirements for Intel macOS (#805) Sebastian Raschka 2025-09-03 12:15:25 -05:00
65e67a9681 fix typo rasbt 2025-09-02 10:17:51 -05:00
2d8d6224ed added brief explanations about 2 different ways of RoPE implementations (#802) Hayato Hongo 2025-09-03 00:14:36 +09:00
9ea2c57c5f simplify rasbt 2025-09-01 22:15:47 -05:00
643f800a94 remove local config files rasbt 2025-09-01 20:52:40 -05:00
9eee9296d9 Interactive qwen3 chat interface (#801) Sebastian Raschka 2025-09-01 20:50:25 -05:00
70edd53809 Improve RoPE (#799) Sebastian Raschka 2025-08-31 11:46:36 -05:00
d87d91b23c Add KVCache variant of Qwen3 notebook (#800) Sebastian Raschka 2025-08-31 11:11:12 -05:00
0e9cefcdc8 Update pixi powershell command section (#798) Sebastian Raschka 2025-08-30 08:42:06 -05:00
a51ff65488 reasoning-from-scratch (#793) Sebastian Raschka 2025-08-28 18:36:41 -05:00
a3a62c509a Improve MHA einsum (#781) Jestine Paul 2025-08-23 04:12:26 +08:00
670f7a4dd0 - added (missing) Gemma3 bullet point in parent folder's readme.md (#788) casinca 2025-08-22 22:03:47 +02:00
4a84cfccf9 Minor cosmetic fixes in Gemma 3 nbs (#780) Sebastian Raschka 2025-08-19 21:08:29 -05:00
f571b5e493 Add Gemma3 KV cache variant (#776) Sebastian Raschka 2025-08-19 12:37:49 -05:00
8c1f9ccf54 Improve MHA einsum (#775) Sebastian Raschka 2025-08-19 10:38:15 -05:00
80d4732456 add HF equivalency tests for standalone nbs (#774) Sebastian Raschka 2025-08-18 18:58:46 -05:00
a6b883c9f9 Gemma 3 270M From Scratch (#771) Sebastian Raschka 2025-08-17 08:23:05 -05:00
8fd29ed079 Gemma 3 270M from scratch gemma-3 rasbt 2025-08-16 19:49:38 -05:00
e9c1c1da38 Fix qk_norm comment (#769) Sebastian Raschka 2025-08-15 08:38:48 -05:00
27fa95d24b Fix qk_norm comment (#769) Sebastian Raschka 2025-08-15 08:38:48 -05:00
b14325e56d Qwen3 and Llama3 equivalency teests with HF transformers (#768) Sebastian Raschka 2025-08-14 18:36:07 -05:00
07c3122b5c Qwen3 and Llama3 equivalency teests with HF transformers (#768) Sebastian Raschka 2025-08-14 18:36:07 -05:00
06aa6d470a Fix eos token usage in Qwen3 tokenizer fix-qwen3-tokeniizer-eos rasbt 2025-08-05 13:42:18 -05:00
2e3205f747 MoE Nb readability improvements (#761) Sebastian Raschka 2025-08-01 19:58:18 -05:00
5febcf8a1b MoE Nb readability improvements (#761) Sebastian Raschka 2025-08-01 19:58:18 -05:00
71ef67be46 Qwen3 Coder Flash & MoE from Scratch (#760) Sebastian Raschka 2025-08-01 19:13:17 -05:00
f92b40e4ab Qwen3 Coder Flash & MoE from Scratch (#760) Sebastian Raschka 2025-08-01 19:13:17 -05:00
d6213a398a [Minor] Qwen3 typo & optim (#758) casinca 2025-07-29 00:29:44 +02:00
145322ded8 [Minor] Qwen3 typo & optim (#758) casinca 2025-07-29 00:29:44 +02:00
4aa398c79d Comment typo: head_dim -> head_dim // 2 head-dim rasbt 2025-07-23 08:16:30 -05:00
19c065b342 Interleaved Q and K for RoPE in Llama 2 (#750) Sebastian Raschka 2025-07-23 08:02:02 -05:00
b12dbf6c68 Interleaved Q and K for RoPE in Llama 2 (#750) Sebastian Raschka 2025-07-23 08:02:02 -05:00
b74ab9611e Minor typo: pply -> Apply (#749) Sebastian Raschka 2025-07-22 08:19:25 -05:00
13f049f6a4 Minor typo: pply -> Apply (#749) Sebastian Raschka 2025-07-22 08:19:25 -05:00
068751c053 Update Python dependency in pyproject.toml (#748) Sebastian Raschka 2025-07-22 08:03:57 -05:00
c6472f1af1 Update Python dependency in pyproject.toml (#748) Sebastian Raschka 2025-07-22 08:03:57 -05:00
38591b0049 get rid of redundant memory profiler import (#744) Sebastian Raschka 2025-07-16 07:36:51 -05:00
3233ddc475 get rid of redundant memory profiler import (#744) Sebastian Raschka 2025-07-16 07:36:51 -05:00
db3659241d Add link to official video course (#741) Sebastian Raschka 2025-07-13 10:35:12 -05:00
7e9ce325de Add link to official video course (#741) Sebastian Raschka 2025-07-13 10:35:12 -05:00
3d43a27ca9 Fix issue: 731 by resolving semantic error (#738) Matthew Hernandez 2025-07-10 14:30:15 -07:00
6f12edb0cc Fix issue: 731 by resolving semantic error (#738) Matthew Hernandez 2025-07-10 14:30:15 -07:00
1552023bd4 Fix issue 724: unused args (#726) weezymatt-bugfix/semantic-error Matthew Hernandez 2025-07-08 04:37:39 -07:00
5188a41267 Batched KV Cache Inference for Qwen3 (#735) Sebastian Raschka 2025-07-10 08:09:35 -05:00
6515889431 Qwen3 tokenizer sanity checks (#730) Sebastian Raschka 2025-07-09 13:52:35 -05:00
14fa50dfc8 Add more sophisticated Qwen3 tokenizer (#729) Sebastian Raschka 2025-07-09 13:16:26 -05:00
f596aab0cb Simplify KV cache usage (#728) Sebastian Raschka 2025-07-08 12:56:55 -05:00
4c2745509f Update Qwen3 tokenizer test (#727) Sebastian Raschka 2025-07-08 06:59:46 -05:00
420081ae06 fix merge conflict rasbt 2025-07-10 12:52:10 -05:00
a200698698 Batched KV Cache Inference for Qwen3 (#735) Sebastian Raschka 2025-07-10 08:09:35 -05:00
a354555049 Batched KV Cache Inference for Qwen3 (#735) Sebastian Raschka 2025-07-10 08:09:35 -05:00
ede5cac208 Fix issue: 731 by resolving the semantic error 85853890+weezymatt@users.noreply.github.com 2025-07-09 21:24:27 -06:00
a96370df4a Fix issue: 731 by resolving the semantic error 85853890+weezymatt@users.noreply.github.com 2025-07-09 21:09:35 -06:00
7dc1dcbe27 Qwen3 tokenizer sanity checks (#730) Sebastian Raschka 2025-07-09 13:52:35 -05:00
b8c8237251 Qwen3 tokenizer sanity checks (#730) Sebastian Raschka 2025-07-09 13:52:35 -05:00
d23b1f07b8 Add more sophisticated Qwen3 tokenizer (#729) Sebastian Raschka 2025-07-09 13:16:26 -05:00
21c41721cc Add more sophisticated Qwen3 tokenizer (#729) Sebastian Raschka 2025-07-09 13:16:26 -05:00
90c824506c Simplify KV cache usage (#728) Sebastian Raschka 2025-07-08 12:56:55 -05:00
3c9dc4807b Simplify KV cache usage (#728) Sebastian Raschka 2025-07-08 12:56:55 -05:00
b5bd8d2de2 Update Qwen3 tokenizer test (#727) Sebastian Raschka 2025-07-08 06:59:46 -05:00
9cf64170ed Update Qwen3 tokenizer test (#727) Sebastian Raschka 2025-07-08 06:59:46 -05:00
80c1bb2cf4 Fix issue 724: unused args (#726) Matthew Hernandez 2025-07-08 04:37:39 -07:00
83c76891fc Fix issue 724: unused args (#726) Matthew Hernandez 2025-07-08 04:37:39 -07:00
ce24cd428a Update README.md Sebastian Raschka 2025-07-06 17:58:33 -05:00
c8c6e7814a Update README.md Sebastian Raschka 2025-07-06 17:58:33 -05:00
ab57562066 Add prerequisite section (#723) Sebastian Raschka 2025-07-06 12:45:42 -05:00
6103acbedb Add prerequisite section (#723) Sebastian Raschka 2025-07-06 12:45:42 -05:00
95a77a747a Fix pip uv typo in installation instructions (#722) Sebastian Raschka 2025-07-03 15:18:48 -05:00
c62fc7cad1 Fix pip uv typo in installation instructions (#722) Sebastian Raschka 2025-07-03 15:18:48 -05:00
30645a6d64 Handle other Qwen3 tokenizer settings (#716) Sebastian Raschka 2025-06-30 17:49:51 -05:00
0405b0c8e7 Handle other Qwen3 tokenizer settings (#716) Sebastian Raschka 2025-06-30 17:49:51 -05:00
6a4d0a1022 Fix d_out code comment in bonus materials (#715) Sebastian Raschka 2025-06-28 10:07:16 -05:00
4e61dc4224 Fix d_out code comment in bonus materials (#715) Sebastian Raschka 2025-06-28 10:07:16 -05:00
dc2f8e95d4 Support different Qwen3 sizes in pkg (#714) Sebastian Raschka 2025-06-28 08:00:23 -05:00
c4ec55edac Support different Qwen3 sizes in pkg (#714) Sebastian Raschka 2025-06-28 08:00:23 -05:00
8c8ff24118 Use test mode arg in ch07 (#713) Sebastian Raschka 2025-06-27 19:28:56 -05:00
ddbaf0d83e Use test mode arg in ch07 (#713) Sebastian Raschka 2025-06-27 19:28:56 -05:00

1 2 3 4 5 ...