LLMs-from-scratch

books/LLMs-from-scratch

Fork 0

mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00

Commit Graph

Select branches

Hide Pull Requests

cleaning

debug-exclude-newer

dockerfile

fix-qwen3-tokeniizer-eos

fix_masking_kv_cache

gemma-3

head-dim

main

qwen-tokenizer-fix

requirements-update

uv-improvements

weezymatt-bugfix/semantic-error

#1

#10

#100

#1000

#1003

#1004

#1004

#101

#102

#104

#105

#106

#107

#109

#110

#111

#113

#115

#116

#117

#118

#119

#12

#120

#121

#122

#124

#125

#127

#128

#13

#131

#132

#133

#134

#135

#136

#137

#138

#139

#14

#141

#142

#143

#144

#149

#151

#152

#153

#154

#156

#157

#158

#159

#160

#161

#162

#163

#164

#165

#166

#169

#17

#170

#171

#173

#174

#176

#177

#178

#179

#18

#180

#182

#183

#184

#187

#189

#19

#193

#195

#196

#197

#198

#199

#2

#20

#200

#201

#203

#204

#205

#206

#207

#208

#211

#212

#213

#216

#218

#219

#22

#222

#223

#224

#227

#228

#229

#233

#234

#235

#236

#237

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#250

#252

#253

#254

#258

#259

#26

#260

#261

#263

#264

#265

#269

#27

#270

#274

#278

#28

#281

#285

#288

#289

#29

#290

#291

#294

#295

#297

#298

#3

#300

#301

#303

#304

#305

#307

#31

#313

#314

#318

#319

#32

#320

#321

#323

#324

#325

#329

#33

#333

#335

#336

#337

#339

#344

#345

#346

#349

#352

#353

#356

#36

#360

#361

#366

#367

#368

#369

#37

#372

#373

#375

#376

#378

#38

#380

#381

#383

#384

#386

#388

#389

#39

#391

#393

#395

#396

#397

#398

#4

#400

#401

#403

#404

#405

#406

#407

#408

#412

#413

#414

#419

#420

#421

#423

#427

#432

#436

#437

#438

#439

#441

#451

#458

#461

#465

#465

#466

#468

#469

#470

#471

#473

#479

#481

#483

#484

#487

#492

#495

#496

#498

#5

#50

#500

#504

#506

#507

#509

#511

#512

#516

#518

#519

#52

#520

#521

#522

#523

#524

#525

#526

#529

#530

#531

#533

#534

#535

#536

#537

#539

#54

#540

#541

#542

#543

#544

#545

#546

#547

#549

#55

#550

#551

#553

#555

#557

#56

#560

#561

#562

#563

#565

#571

#572

#573

#574

#575

#576

#577

#578

#581

#582

#583

#584

#585

#588

#589

#590

#591

#592

#593

#596

#597

#598

#6

#600

#602

#604

#605

#606

#611

#612

#613

#614

#615

#616

#617

#620

#621

#627

#628

#629

#63

#630

#638

#64

#640

#642

#643

#646

#649

#65

#653

#658

#66

#660

#661

#663

#665

#667

#668

#671

#672

#673

#674

#676

#677

#678

#68

#680

#682

#685

#687

#688

#69

#695

#696

#697

#698

#699

#7

#70

#700

#702

#703

#704

#705

#706

#708

#709

#71

#710

#711

#713

#714

#715

#716

#719

#722

#723

#725

#726

#727

#728

#729

#73

#730

#732

#733

#735

#737

#738

#74

#741

#744

#747

#748

#749

#75

#750

#752

#758

#76

#760

#761

#764

#767

#768

#769

#77

#770

#771

#773

#774

#775

#776

#78

#780

#781

#786

#788

#79

#793

#798

#799

#800

#801

#802

#805

#807

#809

#81

#810

#811

#812

#814

#815

#817

#818

#819

#82

#820

#823

#826

#828

#829

#83

#830

#831

#832

#833

#834

#839

#84

#841

#842

#843

#845

#847

#849

#850

#851

#858

#859

#861

#862

#863

#865

#866

#867

#869

#870

#874

#875

#876

#878

#879

#880

#881

#883

#887

#888

#89

#890

#891

#892

#893

#897

#898

#9

#90

#900

#901

#903

#904

#906

#909

#910

#914

#915

#916

#918

#920

#925

#926

#927

#933

#935

#937

#938

#940

#942

#943

#945

#946

#949

#950

#951

#952

#953

#954

#957

#958

#959

#96

#960

#961

#962

#965

#969

#97

#974

#975

#976

#981

#987

#989

#99

#991

#993

#994

#998

#999

74e04a9169 Add "What's next" section (#432) Sebastian Raschka 2024-11-07 20:12:59 -06:00
f4ed263847 Add "What's next" section (#432) Sebastian Raschka 2024-11-07 20:12:59 -06:00
5348565e0f add dropout scaling note rasbt 2024-11-06 05:52:47 -06:00
1183fd7837 add dropout scaling note rasbt 2024-11-06 05:52:47 -06:00
2fd07e2cfd potential little fixes appendix-D4 .ipynb (#427) casinca 2024-11-03 19:12:58 +01:00
9ce0be333b potential little fixes appendix-D4 .ipynb (#427) casinca 2024-11-03 19:12:58 +01:00
95f8a4084f Update CITATION.cff Sebastian Raschka 2024-11-01 21:32:17 -05:00
ba3137fa2c Update CITATION.cff Sebastian Raschka 2024-11-01 21:32:17 -05:00
9d2fd4c22e Update CITATION.cff Sebastian Raschka 2024-11-01 21:29:22 -05:00
734f36aac1 Update CITATION.cff Sebastian Raschka 2024-11-01 21:29:22 -05:00
81e78d0ad8 Add citation file rasbt 2024-11-01 21:21:25 -05:00
7553e87af0 Add citation file rasbt 2024-11-01 21:21:25 -05:00
50500e94b5 Note about warm-up steps rasbt 2024-11-01 16:47:12 -05:00
f03f545a17 Note about warm-up steps rasbt 2024-11-01 16:47:12 -05:00
7e6f8ce020 updated RoPE statement (#423) Daniel Kleine 2024-10-30 14:00:08 +01:00
81eed9afe2 updated RoPE statement (#423) Daniel Kleine 2024-10-30 14:00:08 +01:00
62e44b2415 Update README.md Sebastian Raschka 2024-10-29 20:20:48 -05:00
b5f2aa3500 Update README.md Sebastian Raschka 2024-10-29 20:20:48 -05:00
e85d154522 Fix argument name in LlamaTokenizer constructor (#421) ROHAN WINSOR 2024-10-30 04:31:36 +05:30
cd24a27161 Fix argument name in LlamaTokenizer constructor (#421) ROHAN WINSOR 2024-10-30 04:31:36 +05:30
2b24a7ef30 minor fixes: Llama 3.2 standalone (#420) Daniel Kleine 2024-10-26 04:08:06 +02:00
e8c2f962e9 minor fixes: Llama 3.2 standalone (#420) Daniel Kleine 2024-10-26 04:08:06 +02:00
75ede3e340 RoPE theta rescaling (#419) Sebastian Raschka 2024-10-25 15:27:23 -05:00
1516de54a5 RoPE theta rescaling (#419) Sebastian Raschka 2024-10-25 15:27:23 -05:00
0ed1e0d099 fixed typos (#414) Daniel Kleine 2024-10-25 01:23:53 +02:00
5ff72c2850 fixed typos (#414) Daniel Kleine 2024-10-25 01:23:53 +02:00
8b60460319 Updated Llama 2 to 3 paths (#413) Daniel Kleine 2024-10-24 14:40:08 +02:00
d38083c401 Updated Llama 2 to 3 paths (#413) Daniel Kleine 2024-10-24 14:40:08 +02:00
632d7772b2 Update test-requirements-extra.txt Sebastian Raschka 2024-10-23 19:19:58 -05:00
e1dfd2cb7a Update test-requirements-extra.txt Sebastian Raschka 2024-10-23 19:19:58 -05:00
f8bdfe12e1 RoPE updates (#412) Sebastian Raschka 2024-10-23 18:07:49 -05:00
7cd6a670ed RoPE updates (#412) Sebastian Raschka 2024-10-23 18:07:49 -05:00
6dd3fbd79d Update tests.py Sebastian Raschka 2024-10-23 07:48:33 -05:00
4f9c9fb703 Update tests.py Sebastian Raschka 2024-10-23 07:48:33 -05:00
cba4f89514 updates for PyTorch 2.5 (#408) Daniel Kleine 2024-10-23 03:23:31 +02:00
ef4018181e updates for PyTorch 2.5 (#408) Daniel Kleine 2024-10-23 03:23:31 +02:00
9726ca6546 RoPE increase (#407) Sebastian Raschka 2024-10-21 19:58:38 -05:00
534a704364 RoPE increase (#407) Sebastian Raschka 2024-10-21 19:58:38 -05:00
208d8030c9 Set sampler in DDP example Sebastian Raschka 2024-10-21 09:26:01 -05:00
75133605c5 Set sampler in DDP example Sebastian Raschka 2024-10-21 09:26:01 -05:00
3c3dae0967 Add mean pooling experiment to classifier bonus experiments (#406) Sebastian Raschka 2024-10-20 11:04:18 -05:00
38969864e6 Add mean pooling experiment to classifier bonus experiments (#406) Sebastian Raschka 2024-10-20 11:04:18 -05:00
c4bac22bff Test PyTorch 2.5 (#405) Sebastian Raschka 2024-10-20 10:23:31 -05:00
467197bbf5 Test PyTorch 2.5 (#405) Sebastian Raschka 2024-10-20 10:23:31 -05:00
42b703fc0b Note about SSL certificates (#404) Sebastian Raschka 2024-10-19 16:27:19 -05:00
1f61aeb7c4 Note about SSL certificates (#404) Sebastian Raschka 2024-10-19 16:27:19 -05:00
3567fb656d update mmap section rasbt 2024-10-14 14:27:19 -05:00
cd2753a36d update mmap section rasbt 2024-10-14 14:27:19 -05:00
31fb74133a add mmap=True comparison rasbt 2024-10-14 11:09:55 -05:00
08362fd290 add mmap=True comparison rasbt 2024-10-14 11:09:55 -05:00
3d54af20f5 Memory efficient weight loading (#401) Sebastian Raschka 2024-10-14 10:30:25 -05:00
05b04f2a5a Memory efficient weight loading (#401) Sebastian Raschka 2024-10-14 10:30:25 -05:00
59a5c83726 remove redundant code line rasbt 2024-10-13 15:58:11 -05:00
a20ce1b817 remove redundant code line rasbt 2024-10-13 15:58:11 -05:00
6a9bedc2ec Update bonus section formatting (#400) Sebastian Raschka 2024-10-12 10:26:08 -05:00
b6c4b2f9f1 Update bonus section formatting (#400) Sebastian Raschka 2024-10-12 10:26:08 -05:00
35ecca0feb Update check-links.yml Sebastian Raschka 2024-10-11 12:20:49 -05:00
233a3b0c8b Update check-links.yml Sebastian Raschka 2024-10-11 12:20:49 -05:00
76d0807eab update card rasbt 2024-10-11 12:15:01 -05:00
93d9dae95f update card rasbt 2024-10-11 12:15:01 -05:00
c36f623472 update reference numbers rasbt 2024-10-11 12:12:05 -05:00
1f4fca9f8e update reference numbers rasbt 2024-10-11 12:12:05 -05:00
b66d846cf6 Add MFU formula as reference material (#395) Sebastian Raschka 2024-10-10 19:42:53 -05:00
6d0f59a49c Add MFU formula as reference material (#395) Sebastian Raschka 2024-10-10 19:42:53 -05:00
1715aaacbc Update check-links.yml Sebastian Raschka 2024-10-08 08:38:48 -05:00
1a8d2929dd Update check-links.yml Sebastian Raschka 2024-10-08 08:38:48 -05:00
37db3f0913 Add Llama 3.2 RoPE to CI (#391) Sebastian Raschka 2024-10-08 08:28:34 -05:00
ec18b6a8a3 Add Llama 3.2 RoPE to CI (#391) Sebastian Raschka 2024-10-08 08:28:34 -05:00
06604f4b84 Introduce buffers to improve Llama 3.2 efficiency (#389) Sebastian Raschka 2024-10-06 12:49:04 -05:00
1eb0b3810a Introduce buffers to improve Llama 3.2 efficiency (#389) Sebastian Raschka 2024-10-06 12:49:04 -05:00
4f9775d91c fixed Llama 2 to 3.2 NBs (#388) Daniel Kleine 2024-10-06 16:56:55 +02:00
a0c0c765a8 fixed Llama 2 to 3.2 NBs (#388) Daniel Kleine 2024-10-06 16:56:55 +02:00
81053ccadd Add a note about weight tying in Llama 3.2 (#386) Sebastian Raschka 2024-10-05 09:20:54 -05:00
0972ded530 Add a note about weight tying in Llama 3.2 (#386) Sebastian Raschka 2024-10-05 09:20:54 -05:00
58c3bb3d9d Llama 3 (#384) Sebastian Raschka 2024-10-05 07:52:15 -05:00
8a448a4410 Llama 3 (#384) Sebastian Raschka 2024-10-05 07:52:15 -05:00
8d6b25785d Llama 3.2 requirements file Sebastian Raschka 2024-10-05 07:32:43 -05:00
8553644440 Llama 3.2 requirements file Sebastian Raschka 2024-10-05 07:32:43 -05:00
6f86c78763 Implement Llama 3.2 (#383) Sebastian Raschka 2024-10-05 07:30:47 -05:00
b44096acef Implement Llama 3.2 (#383) Sebastian Raschka 2024-10-05 07:30:47 -05:00
d313f61c86 Cos-sin fix in Llama 2 bonus notebook (#381) Sebastian Raschka 2024-10-03 20:45:40 -05:00
a5405c255d Cos-sin fix in Llama 2 bonus notebook (#381) Sebastian Raschka 2024-10-03 20:45:40 -05:00
feb0647c79 Improve rope settings for llama3 (#380) Sebastian Raschka 2024-10-03 08:29:54 -05:00
b993c2b25b Improve rope settings for llama3 (#380) Sebastian Raschka 2024-10-03 08:29:54 -05:00
2ae4ad15ba add section numbers rasbt 2024-09-30 08:42:22 -05:00
278a50a348 add section numbers rasbt 2024-09-30 08:42:22 -05:00
505e9a5fa5 Improve DDP on Windows (#376) Sebastian Raschka 2024-09-29 16:53:48 -05:00
4caafddb93 Improve DDP on Windows (#376) Sebastian Raschka 2024-09-29 16:53:48 -05:00
58d0ce83a4 llama note rasbt 2024-09-26 07:41:11 -05:00
bfa4215774 llama note rasbt 2024-09-26 07:41:11 -05:00
68505fab64 Fix truncation issue in classify_review function (#373) Sebastian Raschka 2024-09-25 19:54:36 -05:00
7ef5129e18 Fix truncation issue in classify_review function (#373) Sebastian Raschka 2024-09-25 19:54:36 -05:00
b8497c1bf5 Add llama2 unit tests (#372) Sebastian Raschka 2024-09-25 19:40:36 -05:00
b56d0b2942 Add llama2 unit tests (#372) Sebastian Raschka 2024-09-25 19:40:36 -05:00
a23fca84d5 improve formatting rasbt 2024-09-24 18:49:17 -05:00
a6d8e93da3 improve formatting rasbt 2024-09-24 18:49:17 -05:00
4541177063 ch05/07 gpt_to_llama text improvements (#369) Daniel Kleine 2024-09-25 01:45:49 +02:00
ff31b345b0 ch05/07 gpt_to_llama text improvements (#369) Daniel Kleine 2024-09-25 01:45:49 +02:00
941629d2c7 add json import rasbt 2024-09-23 09:12:31 -05:00
d144bd5b7a add json import rasbt 2024-09-23 09:12:31 -05:00

... 4 5 6 7 8 ...