LLMs-from-scratch

books/LLMs-from-scratch

Fork 0

mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00

Commit Graph

Select branches

Hide Pull Requests

cleaning

debug-exclude-newer

dockerfile

fix-qwen3-tokeniizer-eos

fix_masking_kv_cache

gemma-3

head-dim

main

qwen-tokenizer-fix

requirements-update

uv-improvements

weezymatt-bugfix/semantic-error

#1

#10

#100

#1000

#1003

#1004

#1004

#101

#102

#104

#105

#106

#107

#109

#110

#111

#113

#115

#116

#117

#118

#119

#12

#120

#121

#122

#124

#125

#127

#128

#13

#131

#132

#133

#134

#135

#136

#137

#138

#139

#14

#141

#142

#143

#144

#149

#151

#152

#153

#154

#156

#157

#158

#159

#160

#161

#162

#163

#164

#165

#166

#169

#17

#170

#171

#173

#174

#176

#177

#178

#179

#18

#180

#182

#183

#184

#187

#189

#19

#193

#195

#196

#197

#198

#199

#2

#20

#200

#201

#203

#204

#205

#206

#207

#208

#211

#212

#213

#216

#218

#219

#22

#222

#223

#224

#227

#228

#229

#233

#234

#235

#236

#237

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#250

#252

#253

#254

#258

#259

#26

#260

#261

#263

#264

#265

#269

#27

#270

#274

#278

#28

#281

#285

#288

#289

#29

#290

#291

#294

#295

#297

#298

#3

#300

#301

#303

#304

#305

#307

#31

#313

#314

#318

#319

#32

#320

#321

#323

#324

#325

#329

#33

#333

#335

#336

#337

#339

#344

#345

#346

#349

#352

#353

#356

#36

#360

#361

#366

#367

#368

#369

#37

#372

#373

#375

#376

#378

#38

#380

#381

#383

#384

#386

#388

#389

#39

#391

#393

#395

#396

#397

#398

#4

#400

#401

#403

#404

#405

#406

#407

#408

#412

#413

#414

#419

#420

#421

#423

#427

#432

#436

#437

#438

#439

#441

#451

#458

#461

#465

#465

#466

#468

#469

#470

#471

#473

#479

#481

#483

#484

#487

#492

#495

#496

#498

#5

#50

#500

#504

#506

#507

#509

#511

#512

#516

#518

#519

#52

#520

#521

#522

#523

#524

#525

#526

#529

#530

#531

#533

#534

#535

#536

#537

#539

#54

#540

#541

#542

#543

#544

#545

#546

#547

#549

#55

#550

#551

#553

#555

#557

#56

#560

#561

#562

#563

#565

#571

#572

#573

#574

#575

#576

#577

#578

#581

#582

#583

#584

#585

#588

#589

#590

#591

#592

#593

#596

#597

#598

#6

#600

#602

#604

#605

#606

#611

#612

#613

#614

#615

#616

#617

#620

#621

#627

#628

#629

#63

#630

#638

#64

#640

#642

#643

#646

#649

#65

#653

#658

#66

#660

#661

#663

#665

#667

#668

#671

#672

#673

#674

#676

#677

#678

#68

#680

#682

#685

#687

#688

#69

#695

#696

#697

#698

#699

#7

#70

#700

#702

#703

#704

#705

#706

#708

#709

#71

#710

#711

#713

#714

#715

#716

#719

#722

#723

#725

#726

#727

#728

#729

#73

#730

#732

#733

#735

#737

#738

#74

#741

#744

#747

#748

#749

#75

#750

#752

#758

#76

#760

#761

#764

#767

#768

#769

#77

#770

#771

#773

#774

#775

#776

#78

#780

#781

#786

#788

#79

#793

#798

#799

#800

#801

#802

#805

#807

#809

#81

#810

#811

#812

#814

#815

#817

#818

#819

#82

#820

#823

#826

#828

#829

#83

#830

#831

#832

#833

#834

#839

#84

#841

#842

#843

#845

#847

#849

#850

#851

#858

#859

#861

#862

#863

#865

#866

#867

#869

#870

#874

#875

#876

#878

#879

#880

#881

#883

#887

#888

#89

#890

#891

#892

#893

#897

#898

#9

#90

#900

#901

#903

#904

#906

#909

#910

#914

#915

#916

#918

#920

#925

#926

#927

#933

#935

#937

#938

#940

#942

#943

#945

#946

#949

#950

#951

#952

#953

#954

#957

#958

#959

#96

#960

#961

#962

#965

#969

#97

#974

#975

#976

#981

#987

#989

#99

#991

#993

#994

#998

#999

47030fd8c1 update badges rasbt 2025-02-17 12:00:46 -06:00
24f78865df update badges rasbt 2025-02-17 12:00:46 -06:00
1d1ac0b9a2 update readme badges rasbt 2025-02-17 11:49:41 -06:00
2f67cbca0b update readme badges rasbt 2025-02-17 11:49:41 -06:00
bd0484c1be feat: Add pixi environment (#534) Matthew Feickert 2025-02-17 10:33:53 -07:00
a8b8eb4731 feat: Add pixi environment (#534) Matthew Feickert 2025-02-17 10:33:53 -07:00
15592c1204 note about google colab (#535) Sebastian Raschka 2025-02-16 21:12:36 -06:00
16738b61fd note about google colab (#535) Sebastian Raschka 2025-02-16 21:12:36 -06:00
4d5f37abd4 Update README.md Sebastian Raschka 2025-02-16 13:37:32 -06:00
bacb7aa90c Update README.md Sebastian Raschka 2025-02-16 13:37:32 -06:00
5c8d42ec8d Fixing small typo in package description of the pyproject.toml file (#533) ridhachahed 2025-02-16 20:19:26 +01:00
03d72e6eb5 Fixing small typo in package description of the pyproject.toml file (#533) ridhachahed 2025-02-16 20:19:26 +01:00
5016499d1d Uv workflow improvements (#531) Sebastian Raschka 2025-02-16 13:16:51 -06:00
a08d7aaa84 Uv workflow improvements (#531) Sebastian Raschka 2025-02-16 13:16:51 -06:00
2f0afedaa7 reduce redundancies rasbt 2025-02-15 21:31:10 -06:00
29353c74d8 reduce redundancies rasbt 2025-02-15 21:31:10 -06:00
0d1f8c2bad Add performance comparison Sebastian Raschka 2025-02-15 21:16:36 -06:00
fc11940ace Add performance comparison Sebastian Raschka 2025-02-15 21:16:36 -06:00
2968af0559 formatting rasbt 2025-02-15 21:05:58 -06:00
7e335f8af0 formatting rasbt 2025-02-15 21:05:58 -06:00
723d08cb9d cosmetics rasbt 2025-02-15 20:53:26 -06:00
61ca01c7c5 cosmetics rasbt 2025-02-15 20:53:26 -06:00
aa60bb3cd5 Native uv docs (#530) Sebastian Raschka 2025-02-15 20:35:23 -06:00
3e3dc3c5dc Native uv docs (#530) Sebastian Raschka 2025-02-15 20:35:23 -06:00
fac61004b0 Update README.md Sebastian Raschka 2025-02-15 13:17:43 -06:00
e9c4dac3ad Update README.md Sebastian Raschka 2025-02-15 13:17:43 -06:00
4ab8719e9d Switch from pip to uv (#529) Sebastian Raschka 2025-02-15 13:13:13 -06:00
88fd849b88 Switch from pip to uv (#529) Sebastian Raschka 2025-02-15 13:13:13 -06:00
e818be42e1 Update link to vocab size increase (#526) Sebastian Raschka 2025-02-14 08:03:01 -06:00
074a6efb33 Update link to vocab size increase (#526) Sebastian Raschka 2025-02-14 08:03:01 -06:00
6370898ce6 PyTorch tips for better training performance (#525) Sebastian Raschka 2025-02-12 16:10:34 -06:00
908dd2f71e PyTorch tips for better training performance (#525) Sebastian Raschka 2025-02-12 16:10:34 -06:00
d16863c7db Add torchrun bonus code (#524) Sebastian Raschka 2025-02-11 17:01:09 -06:00
3c29b67cd0 Add torchrun bonus code (#524) Sebastian Raschka 2025-02-11 17:01:09 -06:00
83b47adf0d Comment that DDP-script.py does not work with GPUs > 2 (#523) Victor Skvortsov 2025-02-12 00:23:49 +05:00
f90bec7dfb Comment that DDP-script.py does not work with GPUs > 2 (#523) Victor Skvortsov 2025-02-12 00:23:49 +05:00
9dce43ec31 Upgrade to NumPy 2.0 (#520) Sebastian Raschka 2025-02-09 06:21:58 -06:00
a6cc574605 Upgrade to NumPy 2.0 (#520) Sebastian Raschka 2025-02-09 06:21:58 -06:00
5efa731c0f Mention small discrepancy due to Dropout non-reproducibility in PyTorch (#519) Sebastian Raschka 2025-02-06 14:59:52 -06:00
68e2efe1c9 Mention small discrepancy due to Dropout non-reproducibility in PyTorch (#519) Sebastian Raschka 2025-02-06 14:59:52 -06:00
b3e0458fa6 fixed indention and enumeration for nvct (#518) Daniel Kleine 2025-02-06 15:17:12 +01:00
bd8f7522cb fixed indention and enumeration for nvct (#518) Daniel Kleine 2025-02-06 15:17:12 +01:00
d684ff418a Fix typo in Ch02 comments (#516) Sebastian Raschka 2025-02-04 20:16:07 -06:00
2dc46bedc6 Fix typo in Ch02 comments (#516) Sebastian Raschka 2025-02-04 20:16:07 -06:00
7e2092dd01 More pythonic way to find the longest sequence (#512) Sebastian Raschka 2025-02-01 10:22:47 -06:00
8cfa52bf1d More pythonic way to find the longest sequence (#512) Sebastian Raschka 2025-02-01 10:22:47 -06:00
c1c10c89c8 Test PyTorch nightly releases (#509) Sebastian Raschka 2025-01-30 12:45:48 -06:00
0e14c76dee Test PyTorch nightly releases (#509) Sebastian Raschka 2025-01-30 12:45:48 -06:00
fd24a3679a Alternative weight loading via .safetensors (#507) Sebastian Raschka 2025-01-29 08:15:29 -06:00
25ea71e713 Alternative weight loading via .safetensors (#507) Sebastian Raschka 2025-01-29 08:15:29 -06:00
ebfc50f076 Fix default argument in ex 7.2 (#506) Sebastian Raschka 2025-01-25 10:46:48 -06:00
9daa7e7511 Fix default argument in ex 7.2 (#506) Sebastian Raschka 2025-01-25 10:46:48 -06:00
d387696f93 A few cosmetic updates (#504) Sebastian Raschka 2025-01-23 09:38:55 -06:00
fd8d77a79d A few cosmetic updates (#504) Sebastian Raschka 2025-01-23 09:38:55 -06:00
bb13ff519e Test for PyTorch 2.6 release candidate (#500) Sebastian Raschka 2025-01-22 18:37:48 -06:00
0911e71497 Test for PyTorch 2.6 release candidate (#500) Sebastian Raschka 2025-01-22 18:37:48 -06:00
dcaac28b92 Bonus material: extending tokenizers (#496) Sebastian Raschka 2025-01-22 09:26:54 -06:00
a22d612be6 Bonus material: extending tokenizers (#496) Sebastian Raschka 2025-01-22 09:26:54 -06:00
9175590ea4 add GPT2TokenizerFast to BPE comparison (#498) Daniel Kleine 2025-01-22 16:26:44 +01:00
dce46038da add GPT2TokenizerFast to BPE comparison (#498) Daniel Kleine 2025-01-22 16:26:44 +01:00
654734053a fix: preserve newline tokens in BPE encoder (#495) Austin Welch 2025-01-21 13:47:15 -05:00
0f35e370ed fix: preserve newline tokens in BPE encoder (#495) Austin Welch 2025-01-21 13:47:15 -05:00
3f9facbc55 BPE: fixed typo (#492) Daniel Kleine 2025-01-21 03:49:53 +01:00
60acb94894 BPE: fixed typo (#492) Daniel Kleine 2025-01-21 03:49:53 +01:00
b17d097742 Implementingthe BPE Tokenizer from Scratch (#487) Sebastian Raschka 2025-01-17 12:22:00 -06:00
0d4967eda6 Implementingthe BPE Tokenizer from Scratch (#487) Sebastian Raschka 2025-01-17 12:22:00 -06:00
f936ad4b4e 04_optional-aws-sagemaker-notebook (#451) rvaneijk 2025-01-17 17:07:10 +01:00
2fef2116a6 04_optional-aws-sagemaker-notebook (#451) rvaneijk 2025-01-17 17:07:10 +01:00
37aed8fc2c Include mathematical breakdown for exercise solution 4.1 (#483) Sebastian Raschka 2025-01-14 19:23:00 -06:00
126adb7663 Include mathematical breakdown for exercise solution 4.1 (#483) Sebastian Raschka 2025-01-14 19:23:00 -06:00
15af754304 Print out embeddings for more illustrative learning (#481) Henry Shi 2025-01-13 12:44:06 -08:00
b3150eebd8 Print out embeddings for more illustrative learning (#481) Henry Shi 2025-01-13 12:44:06 -08:00
bed5f89378 fix reward margins plot label in dpo nb rasbt 2025-01-12 14:04:05 -06:00
b524afe3da fix reward margins plot label in dpo nb rasbt 2025-01-12 14:04:05 -06:00
992f3068d1 Auto download DPO dataset if not already available in path (#479) Sebastian Raschka 2025-01-12 12:27:28 -06:00
4bfbcd069d Auto download DPO dataset if not already available in path (#479) Sebastian Raschka 2025-01-12 12:27:28 -06:00
05f2a398b8 adds no-grad context for reference model to DPO (#473) Sebastian Raschka 2025-01-07 20:49:01 -06:00
a48f9c7fe2 adds no-grad context for reference model to DPO (#473) Sebastian Raschka 2025-01-07 20:49:01 -06:00
b3dc7073ee fix ch07 unit test (#470) Sebastian Raschka 2025-01-05 17:40:57 -06:00
2d7ca7ee4b fix ch07 unit test (#470) Sebastian Raschka 2025-01-05 17:40:57 -06:00
7659af7cdd Add backup URL for gpt2 weights (#469) Sebastian Raschka 2025-01-05 11:28:09 -06:00
701090815e Add backup URL for gpt2 weights (#469) Sebastian Raschka 2025-01-05 11:28:09 -06:00
976c92010c typo fixed (#468) QS 2025-01-05 16:17:13 +01:00
9b95557ba2 typo fixed (#468) QS 2025-01-05 16:17:13 +01:00
65ee619d3b Minor readability improvement in dataloader.ipynb (#461) Tao Qian 2025-01-04 09:26:10 -08:00
cec445f146 Minor readability improvement in dataloader.ipynb (#461) Tao Qian 2025-01-04 09:26:10 -08:00
05a816e270 fix misplaced parenthesis and update license (#466) Sebastian Raschka 2025-01-04 11:14:08 -06:00
1b635f760e fix misplaced parenthesis and update license (#466) Sebastian Raschka 2025-01-04 11:14:08 -06:00
57fdd94358 [minor] typo & comments (#441) casinca 2024-11-18 11:52:42 +01:00
bb31de8999 [minor] typo & comments (#441) casinca 2024-11-18 11:52:42 +01:00
dcef9b7d6f Fixed command for row 16 additional experiment (#439) Daniel Kleine 2024-11-16 22:50:00 +01:00
e95c898545 Fixed command for row 16 additional experiment (#439) Daniel Kleine 2024-11-16 22:50:00 +01:00
841b64f2b9 Add flexible padding bonus experiment (#438) Sebastian Raschka 2024-11-15 08:51:01 +09:00
ccade77bf4 Add flexible padding bonus experiment (#438) Sebastian Raschka 2024-11-15 08:51:01 +09:00
541b237eff Add utility to prevent double execution of certain cells (#437) Sebastian Raschka 2024-11-14 19:56:49 +09:00
f6281ab91b Add utility to prevent double execution of certain cells (#437) Sebastian Raschka 2024-11-14 19:56:49 +09:00
129d0d740f Add missing device transfer in gpt_generate.py (#436) Sebastian Raschka 2024-11-14 19:12:53 +09:00
f61c008c5d Add missing device transfer in gpt_generate.py (#436) Sebastian Raschka 2024-11-14 19:12:53 +09:00
6d1bb2cfb4 Add chapter names Sebastian Raschka 2024-11-08 08:39:34 -06:00
27a6a7e64a Add chapter names Sebastian Raschka 2024-11-08 08:39:34 -06:00

... 3 4 5 6 7 ...