LLMs-from-scratch

books/LLMs-from-scratch

Fork 0

mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00

Commit Graph

Select branches

Hide Pull Requests

cleaning

debug-exclude-newer

dockerfile

fix-qwen3-tokeniizer-eos

fix_masking_kv_cache

gemma-3

head-dim

main

qwen-tokenizer-fix

requirements-update

uv-improvements

weezymatt-bugfix/semantic-error

#1

#10

#100

#1000

#1003

#1004

#1004

#101

#102

#104

#105

#106

#107

#109

#110

#111

#113

#115

#116

#117

#118

#119

#12

#120

#121

#122

#124

#125

#127

#128

#13

#131

#132

#133

#134

#135

#136

#137

#138

#139

#14

#141

#142

#143

#144

#149

#151

#152

#153

#154

#156

#157

#158

#159

#160

#161

#162

#163

#164

#165

#166

#169

#17

#170

#171

#173

#174

#176

#177

#178

#179

#18

#180

#182

#183

#184

#187

#189

#19

#193

#195

#196

#197

#198

#199

#2

#20

#200

#201

#203

#204

#205

#206

#207

#208

#211

#212

#213

#216

#218

#219

#22

#222

#223

#224

#227

#228

#229

#233

#234

#235

#236

#237

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#250

#252

#253

#254

#258

#259

#26

#260

#261

#263

#264

#265

#269

#27

#270

#274

#278

#28

#281

#285

#288

#289

#29

#290

#291

#294

#295

#297

#298

#3

#300

#301

#303

#304

#305

#307

#31

#313

#314

#318

#319

#32

#320

#321

#323

#324

#325

#329

#33

#333

#335

#336

#337

#339

#344

#345

#346

#349

#352

#353

#356

#36

#360

#361

#366

#367

#368

#369

#37

#372

#373

#375

#376

#378

#38

#380

#381

#383

#384

#386

#388

#389

#39

#391

#393

#395

#396

#397

#398

#4

#400

#401

#403

#404

#405

#406

#407

#408

#412

#413

#414

#419

#420

#421

#423

#427

#432

#436

#437

#438

#439

#441

#451

#458

#461

#465

#465

#466

#468

#469

#470

#471

#473

#479

#481

#483

#484

#487

#492

#495

#496

#498

#5

#50

#500

#504

#506

#507

#509

#511

#512

#516

#518

#519

#52

#520

#521

#522

#523

#524

#525

#526

#529

#530

#531

#533

#534

#535

#536

#537

#539

#54

#540

#541

#542

#543

#544

#545

#546

#547

#549

#55

#550

#551

#553

#555

#557

#56

#560

#561

#562

#563

#565

#571

#572

#573

#574

#575

#576

#577

#578

#581

#582

#583

#584

#585

#588

#589

#590

#591

#592

#593

#596

#597

#598

#6

#600

#602

#604

#605

#606

#611

#612

#613

#614

#615

#616

#617

#620

#621

#627

#628

#629

#63

#630

#638

#64

#640

#642

#643

#646

#649

#65

#653

#658

#66

#660

#661

#663

#665

#667

#668

#671

#672

#673

#674

#676

#677

#678

#68

#680

#682

#685

#687

#688

#69

#695

#696

#697

#698

#699

#7

#70

#700

#702

#703

#704

#705

#706

#708

#709

#71

#710

#711

#713

#714

#715

#716

#719

#722

#723

#725

#726

#727

#728

#729

#73

#730

#732

#733

#735

#737

#738

#74

#741

#744

#747

#748

#749

#75

#750

#752

#758

#76

#760

#761

#764

#767

#768

#769

#77

#770

#771

#773

#774

#775

#776

#78

#780

#781

#786

#788

#79

#793

#798

#799

#800

#801

#802

#805

#807

#809

#81

#810

#811

#812

#814

#815

#817

#818

#819

#82

#820

#823

#826

#828

#829

#83

#830

#831

#832

#833

#834

#839

#84

#841

#842

#843

#845

#847

#849

#850

#851

#858

#859

#861

#862

#863

#865

#866

#867

#869

#870

#874

#875

#876

#878

#879

#880

#881

#883

#887

#888

#89

#890

#891

#892

#893

#897

#898

#9

#90

#900

#901

#903

#904

#906

#909

#910

#914

#915

#916

#918

#920

#925

#926

#927

#933

#935

#937

#938

#940

#942

#943

#945

#946

#949

#950

#951

#952

#953

#954

#957

#958

#959

#96

#960

#961

#962

#965

#969

#97

#974

#975

#976

#981

#987

#989

#99

#991

#993

#994

#998

#999

ab17357474 Correct BERT experiments (#600) Sebastian Raschka 2025-04-05 10:05:15 -05:00
371ab9e8ff Correct BERT experiments (#600) Sebastian Raschka 2025-04-05 10:05:15 -05:00
14f976e024 Add ModernBERT (#598) Sebastian Raschka 2025-04-05 09:13:30 -05:00
4a9654173c Add ModernBERT (#598) Sebastian Raschka 2025-04-05 09:13:30 -05:00
396e96ab07 Fix Llama language typo in bonus materials (#597) Sebastian Raschka 2025-04-02 21:41:36 -05:00
d4c8d8f2c9 Fix Llama language typo in bonus materials (#597) Sebastian Raschka 2025-04-02 21:41:36 -05:00
f61baf86f2 Fix link (#596) Sebastian Raschka 2025-04-02 09:47:07 -05:00
49330d0990 Fix link (#596) Sebastian Raschka 2025-04-02 09:47:07 -05:00
2dc2df593a Llama3Fast (#593) Sebastian Raschka 2025-04-01 12:56:11 -05:00
43e25a5165 Llama3Fast (#593) Sebastian Raschka 2025-04-01 12:56:11 -05:00
4128a91c1d Add Llama 3.2 to pkg (#591) Sebastian Raschka 2025-03-31 18:59:47 -05:00
aedad7efc3 Add Llama 3.2 to pkg (#591) Sebastian Raschka 2025-03-31 18:59:47 -05:00
d7c316533a removing unused RoPE parameters (#590) casinca 2025-04-01 00:10:39 +02:00
152a087a37 removing unused RoPE parameters (#590) casinca 2025-04-01 00:10:39 +02:00
4715dc3be5 remove redundant context_length in GQA cleaning rasbt 2025-03-31 16:49:10 -05:00
06ebac3c34 Merge branch 'main' into cleaning Sebastian Raschka 2025-03-31 16:26:05 -05:00
d75f74bd0c Fix data download if UCI is temporarily down (#592) Sebastian Raschka 2025-03-31 16:25:53 -05:00
222803737d Fix data download if UCI is temporarily down (#592) Sebastian Raschka 2025-03-31 16:25:53 -05:00
a10d0ea661 removing unused RoPE parameters casinca 2025-03-31 19:19:27 +02:00
0bdcce4e40 Clarify dataset length in chapter 2 (#589) Sebastian Raschka 2025-03-30 16:01:37 -05:00
6ea4dd3ae7 Clarify dataset length in chapter 2 (#589) Sebastian Raschka 2025-03-30 16:01:37 -05:00
4e3b752e5e Memory optimized Llama (#588) Sebastian Raschka 2025-03-30 15:18:12 -05:00
0f6894f41e Memory optimized Llama (#588) Sebastian Raschka 2025-03-30 15:18:12 -05:00
e55e3e88e1 Alt weight loading code via PyTorch (#585) Sebastian Raschka 2025-03-27 20:10:23 -05:00
3f93d73d6d Alt weight loading code via PyTorch (#585) Sebastian Raschka 2025-03-27 20:10:23 -05:00
e07a7abdd5 Add GPTModelFast (#584) Sebastian Raschka 2025-03-27 14:00:25 -05:00
ffd4035144 Add GPTModelFast (#584) Sebastian Raschka 2025-03-27 14:00:25 -05:00
c9271ac427 Adjust comment to save compiled model (#583) Sebastian Raschka 2025-03-27 10:43:45 -05:00
2e143f17b8 Adjust comment to save compiled model (#583) Sebastian Raschka 2025-03-27 10:43:45 -05:00
d4d420361c updated .gitignore (#581) Daniel Kleine 2025-03-26 19:21:14 +01:00
f01e163aad updated .gitignore (#581) Daniel Kleine 2025-03-26 19:21:14 +01:00
857acfcc12 Vocab padding clarification (#582) Sebastian Raschka 2025-03-26 13:19:55 -05:00
92f1313e00 Vocab padding clarification (#582) Sebastian Raschka 2025-03-26 13:19:55 -05:00
fee7d4bb05 More explicit torchrun usage doc (#578) Sebastian Raschka 2025-03-24 12:01:03 -05:00
b7893457da More explicit torchrun usage doc (#578) Sebastian Raschka 2025-03-24 12:01:03 -05:00
cf6fb73553 Add readme (#577) Sebastian Raschka 2025-03-23 19:35:12 -05:00
feb1e9a83d Add readme (#577) Sebastian Raschka 2025-03-23 19:35:12 -05:00
7114ccd10d Add PyPI package (#576) Sebastian Raschka 2025-03-23 19:28:49 -05:00
c21bfe4a23 Add PyPI package (#576) Sebastian Raschka 2025-03-23 19:28:49 -05:00
85f2bc0a58 Speed comparison figure (#575) Sebastian Raschka 2025-03-21 11:29:49 -05:00
7757c3d308 Speed comparison figure (#575) Sebastian Raschka 2025-03-21 11:29:49 -05:00
84591abd5f add ch04 code along video (#573) Sebastian Raschka 2025-03-17 11:20:55 -05:00
73f4342664 add ch04 code along video (#573) Sebastian Raschka 2025-03-17 11:20:55 -05:00
d21d821412 Add chapter 3 coding along video link (#572) Sebastian Raschka 2025-03-16 16:07:14 -05:00
4db0e826b7 Add chapter 3 coding along video link (#572) Sebastian Raschka 2025-03-16 16:07:14 -05:00
384b9ce959 Better instruction eva prompt (#571) Sebastian Raschka 2025-03-15 17:13:15 -05:00
54474fb452 Better instruction eva prompt (#571) Sebastian Raschka 2025-03-15 17:13:15 -05:00
1ec5631c70 Fix minor printing issue and note inconsistency across platforms (#563) Greg Gandenberger 2025-03-14 15:12:09 -05:00
c1611d4ea8 Fix minor printing issue and note inconsistency across platforms (#563) Greg Gandenberger 2025-03-14 15:12:09 -05:00
67244aba8a Clarification about launching jupyter lab (#565) Sebastian Raschka 2025-03-14 14:37:22 -05:00
8d01d3b0c1 Clarification about launching jupyter lab (#565) Sebastian Raschka 2025-03-14 14:37:22 -05:00
634a531223 Cosmetic improvements to the BPE code (#562) Sebastian Raschka 2025-03-09 10:49:40 -05:00
2f41429cf4 Cosmetic improvements to the BPE code (#562) Sebastian Raschka 2025-03-09 10:49:40 -05:00
6aec412421 Fix BPE bonus materials (#561) Sebastian Raschka 2025-03-08 17:21:30 -06:00
f63f04d8d5 Fix BPE bonus materials (#561) Sebastian Raschka 2025-03-08 17:21:30 -06:00
faadd13871 Update mha plot (#560) Sebastian Raschka 2025-03-06 20:29:04 -06:00
96ca2fcb2f Update mha plot (#560) Sebastian Raschka 2025-03-06 20:29:04 -06:00
4fb0ea9d1f Specify UTF-8 encoding in the json load command explicitely (#557) Sebastian Raschka 2025-03-05 11:46:21 -06:00
86b714a5e0 Specify UTF-8 encoding in the json load command explicitely (#557) Sebastian Raschka 2025-03-05 11:46:21 -06:00
de60da9a6b Add a note about "zsh: illegal hardware instruction python" error (#555) Sebastian Raschka 2025-03-02 15:18:24 -06:00
5fc78ff9fd Add a note about "zsh: illegal hardware instruction python" error (#555) Sebastian Raschka 2025-03-02 15:18:24 -06:00
5be0e3cbbd add link to supplementary ch02 video (#553) Sebastian Raschka 2025-03-02 13:17:42 -06:00
e9ad6cf86d add link to supplementary ch02 video (#553) Sebastian Raschka 2025-03-02 13:17:42 -06:00
839a7e9bfc Use correct ch02 title (#551) Sebastian Raschka 2025-02-28 10:16:21 -06:00
e7740b3312 Use correct ch02 title (#551) Sebastian Raschka 2025-02-28 10:16:21 -06:00
db58925d7f Add BPE from scratch link (#550) Sebastian Raschka 2025-02-28 09:57:41 -06:00
b1773897d3 Add BPE from scratch link (#550) Sebastian Raschka 2025-02-28 09:57:41 -06:00
80c95987c1 Add note about context_length (#549) Greg Gandenberger 2025-02-27 08:36:41 -06:00
b92c0dff89 Add note about context_length (#549) Greg Gandenberger 2025-02-27 08:36:41 -06:00
b6f9c8956f Add setup video tutorial (#547) Sebastian Raschka 2025-02-26 13:53:22 -06:00
8939fdc846 Add setup video tutorial (#547) Sebastian Raschka 2025-02-26 13:53:22 -06:00
f5a296d8e7 Consitent spacing (#546) Sebastian Raschka 2025-02-23 17:52:00 -06:00
cd5cf8112b Consitent spacing (#546) Sebastian Raschka 2025-02-23 17:52:00 -06:00
fa5760a8de GitHub markdown updates (#545) Sebastian Raschka 2025-02-23 12:25:44 -06:00
f12b899d96 GitHub markdown updates (#545) Sebastian Raschka 2025-02-23 12:25:44 -06:00
11801be0e9 Simplify the setup in CI (#542) Sebastian Raschka 2025-02-20 11:56:52 -06:00
bca82a9443 Simplify the setup in CI (#542) Sebastian Raschka 2025-02-20 11:56:52 -06:00
4fad4695f6 Fix timeout issue related to spam data backup url (#544) Sebastian Raschka 2025-02-20 09:26:23 -06:00
d1e99f6092 Fix timeout issue related to spam data backup url (#544) Sebastian Raschka 2025-02-20 09:26:23 -06:00
eb6787397c Add backup url for Spam Dataset (#543) Sebastian Raschka 2025-02-20 08:08:28 -06:00
c39aa32ef5 Add backup url for Spam Dataset (#543) Sebastian Raschka 2025-02-20 08:08:28 -06:00
ca0eee4cf9 simplify and use pythorch 3.12 uv-improvements rasbt 2025-02-19 21:01:15 -06:00
361a75d9e2 powershell -> bash rasbt 2025-02-19 19:35:51 -06:00
0e1d1efc10 powershell -> bash rasbt 2025-02-19 19:33:20 -06:00
6b3b8d545b use env rasbt 2025-02-19 19:27:23 -06:00
b13abe3187 improve setup rasbt 2025-02-19 19:22:29 -06:00
06885790d7 Merge branch 'main' into uv-improvements Sebastian Raschka 2025-02-19 19:20:43 -06:00
8cb412eb18 improve setup rasbt 2025-02-19 19:19:18 -06:00
2254102270 Don't explicitly install Python. Use uv sync instead of uv add. (#540) Tim Hopper 2025-02-19 11:42:13 -05:00
1039bf9b80 Don't explicitly install Python. Use uv sync instead of uv add. (#540) Tim Hopper 2025-02-19 11:42:13 -05:00
af4b73ca7b Improve BPE vocabulary saving and pair frequency handling (#539) Kasen 2025-02-19 23:51:04 +08:00
7bd36dccb4 Improve BPE vocabulary saving and pair frequency handling (#539) Kasen 2025-02-19 23:51:04 +08:00
58aabe7dd8 added uv to Dockerfile (#537) Daniel Kleine 2025-02-18 21:53:19 +01:00
3098b4f889 added uv to Dockerfile (#537) Daniel Kleine 2025-02-18 21:53:19 +01:00
0a5214b804 Fix incorrect indentation (#536) Kasen 2025-02-19 04:47:31 +08:00
b47884ced0 Fix incorrect indentation (#536) Kasen 2025-02-19 04:47:31 +08:00
18dd8ab633 Badge url updates Sebastian Raschka 2025-02-17 12:07:47 -06:00
67c226bf67 Badge url updates Sebastian Raschka 2025-02-17 12:07:47 -06:00
acb1018917 update badges rasbt 2025-02-17 12:02:06 -06:00
9ccecd13ae update badges rasbt 2025-02-17 12:02:06 -06:00

... 2 3 4 5 6 ...