LLMs-from-scratch

books/LLMs-from-scratch

Fork 0

mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00

Commit Graph

Select branches

Hide Pull Requests

cleaning

debug-exclude-newer

dockerfile

fix-qwen3-tokeniizer-eos

fix_masking_kv_cache

gemma-3

head-dim

main

qwen-tokenizer-fix

requirements-update

uv-improvements

weezymatt-bugfix/semantic-error

#1

#10

#100

#1000

#1003

#1004

#1004

#101

#102

#104

#105

#106

#107

#109

#110

#111

#113

#115

#116

#117

#118

#119

#12

#120

#121

#122

#124

#125

#127

#128

#13

#131

#132

#133

#134

#135

#136

#137

#138

#139

#14

#141

#142

#143

#144

#149

#151

#152

#153

#154

#156

#157

#158

#159

#160

#161

#162

#163

#164

#165

#166

#169

#17

#170

#171

#173

#174

#176

#177

#178

#179

#18

#180

#182

#183

#184

#187

#189

#19

#193

#195

#196

#197

#198

#199

#2

#20

#200

#201

#203

#204

#205

#206

#207

#208

#211

#212

#213

#216

#218

#219

#22

#222

#223

#224

#227

#228

#229

#233

#234

#235

#236

#237

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#250

#252

#253

#254

#258

#259

#26

#260

#261

#263

#264

#265

#269

#27

#270

#274

#278

#28

#281

#285

#288

#289

#29

#290

#291

#294

#295

#297

#298

#3

#300

#301

#303

#304

#305

#307

#31

#313

#314

#318

#319

#32

#320

#321

#323

#324

#325

#329

#33

#333

#335

#336

#337

#339

#344

#345

#346

#349

#352

#353

#356

#36

#360

#361

#366

#367

#368

#369

#37

#372

#373

#375

#376

#378

#38

#380

#381

#383

#384

#386

#388

#389

#39

#391

#393

#395

#396

#397

#398

#4

#400

#401

#403

#404

#405

#406

#407

#408

#412

#413

#414

#419

#420

#421

#423

#427

#432

#436

#437

#438

#439

#441

#451

#458

#461

#465

#465

#466

#468

#469

#470

#471

#473

#479

#481

#483

#484

#487

#492

#495

#496

#498

#5

#50

#500

#504

#506

#507

#509

#511

#512

#516

#518

#519

#52

#520

#521

#522

#523

#524

#525

#526

#529

#530

#531

#533

#534

#535

#536

#537

#539

#54

#540

#541

#542

#543

#544

#545

#546

#547

#549

#55

#550

#551

#553

#555

#557

#56

#560

#561

#562

#563

#565

#571

#572

#573

#574

#575

#576

#577

#578

#581

#582

#583

#584

#585

#588

#589

#590

#591

#592

#593

#596

#597

#598

#6

#600

#602

#604

#605

#606

#611

#612

#613

#614

#615

#616

#617

#620

#621

#627

#628

#629

#63

#630

#638

#64

#640

#642

#643

#646

#649

#65

#653

#658

#66

#660

#661

#663

#665

#667

#668

#671

#672

#673

#674

#676

#677

#678

#68

#680

#682

#685

#687

#688

#69

#695

#696

#697

#698

#699

#7

#70

#700

#702

#703

#704

#705

#706

#708

#709

#71

#710

#711

#713

#714

#715

#716

#719

#722

#723

#725

#726

#727

#728

#729

#73

#730

#732

#733

#735

#737

#738

#74

#741

#744

#747

#748

#749

#75

#750

#752

#758

#76

#760

#761

#764

#767

#768

#769

#77

#770

#771

#773

#774

#775

#776

#78

#780

#781

#786

#788

#79

#793

#798

#799

#800

#801

#802

#805

#807

#809

#81

#810

#811

#812

#814

#815

#817

#818

#819

#82

#820

#823

#826

#828

#829

#83

#830

#831

#832

#833

#834

#839

#84

#841

#842

#843

#845

#847

#849

#850

#851

#858

#859

#861

#862

#863

#865

#866

#867

#869

#870

#874

#875

#876

#878

#879

#880

#881

#883

#887

#888

#89

#890

#891

#892

#893

#897

#898

#9

#90

#900

#901

#903

#904

#906

#909

#910

#914

#915

#916

#918

#920

#925

#926

#927

#933

#935

#937

#938

#940

#942

#943

#945

#946

#949

#950

#951

#952

#953

#954

#957

#958

#959

#96

#960

#961

#962

#965

#969

#97

#974

#975

#976

#981

#987

#989

#99

#991

#993

#994

#998

#999

0b998dff97 track tokens seen in chapter5, track examples seen in chapter6 (#319) TITC 2024-08-13 20:09:05 +08:00
38390b2a8d track tokens seen in chapter5, track examples seen in chapter6 (#319) TITC 2024-08-13 20:09:05 +08:00
6533ce63c1 fix code cell ordering rasbt 2024-08-12 19:04:05 -05:00
5f0c55ddee fix code cell ordering rasbt 2024-08-12 19:04:05 -05:00
75abb61f13 Small typo fix (#313) Jeroen Van Goey 2024-08-12 14:54:12 +02:00
76e6910a1a Small typo fix (#313) Jeroen Van Goey 2024-08-12 14:54:12 +02:00
b3d550bfd5 Adds .vscode folder to .gitignore (#314) Eric Thomson 2024-08-12 08:49:11 -04:00
da5236ee72 Adds .vscode folder to .gitignore (#314) Eric Thomson 2024-08-12 08:49:11 -04:00
f58c2b969b update attention benchmarks (#307) Sebastian Raschka 2024-08-10 09:44:11 -05:00
3f6652d87e update attention benchmarks (#307) Sebastian Raschka 2024-08-10 09:44:11 -05:00
8d79fb13b0 Update README.md Sebastian Raschka 2024-08-10 07:54:51 -05:00
7feb8cad86 Update README.md Sebastian Raschka 2024-08-10 07:54:51 -05:00
c91999b9f4 fixed bash command (#305) Daniel Kleine 2024-08-10 04:29:04 +02:00
13dbc548f8 fixed bash command (#305) Daniel Kleine 2024-08-10 04:29:04 +02:00
3067ed83dc remove all non-English texts and notice (#304) TITC 2024-08-10 06:09:14 +08:00
09a3a73f2d remove all non-English texts and notice (#304) TITC 2024-08-10 06:09:14 +08:00
de277ebfe6 Update README.md Sebastian Raschka 2024-08-08 07:50:45 -05:00
f1c3d451fe Update README.md Sebastian Raschka 2024-08-08 07:50:45 -05:00
4601219c32 Update README.md Sebastian Raschka 2024-08-08 07:47:31 -05:00
81e9cea3d3 Update README.md Sebastian Raschka 2024-08-08 07:47:31 -05:00
895512ebee Revert accidental edit Sebastian Raschka 2024-08-06 19:54:34 -05:00
c5eaae11b1 Revert accidental edit Sebastian Raschka 2024-08-06 19:54:34 -05:00
7a5771932b note about logistic sigmoid rasbt 2024-08-06 19:48:30 -05:00
06151a809e note about logistic sigmoid rasbt 2024-08-06 19:48:30 -05:00
9ef132bb64 note about logistic sigmoid rasbt 2024-08-06 19:48:06 -05:00
26df0c474c note about logistic sigmoid rasbt 2024-08-06 19:48:06 -05:00
2245f8d9c1 extend equation description rasbt 2024-08-06 19:46:50 -05:00
e810f9f004 extend equation description rasbt 2024-08-06 19:46:50 -05:00
a65e06ff99 add more explanations rasbt 2024-08-06 19:45:11 -05:00
c8090f30ef add more explanations rasbt 2024-08-06 19:45:11 -05:00
b1a420c8ca Update README.md Sebastian Raschka 2024-08-06 08:02:01 -05:00
98d24a1607 Update README.md Sebastian Raschka 2024-08-06 08:02:01 -05:00
7374d617b4 total training iters may equal to warmup_iters (#301) TITC 2024-08-06 20:10:05 +08:00
d16527ddf2 total training iters may equal to warmup_iters (#301) TITC 2024-08-06 20:10:05 +08:00
192bdc3501 improve gradient accumulation (#300) Sebastian Raschka 2024-08-05 18:27:20 -05:00
70e5714556 improve gradient accumulation (#300) Sebastian Raschka 2024-08-05 18:27:20 -05:00
089901db26 small figure update rasbt 2024-08-05 17:57:16 -05:00
36fbc7aa74 small figure update rasbt 2024-08-05 17:57:16 -05:00
b39234fc25 Update README.md Sebastian Raschka 2024-08-05 17:47:06 -05:00
50332cf75b Update README.md Sebastian Raschka 2024-08-05 17:47:06 -05:00
dcdf04e3bd minor DPO fixes (#298) Daniel Kleine 2024-08-05 15:40:46 +02:00
8318d1f002 minor DPO fixes (#298) Daniel Kleine 2024-08-05 15:40:46 +02:00
6030071e3f update model path rasbt 2024-08-05 07:36:08 -05:00
36b9d5e0eb update model path rasbt 2024-08-05 07:36:08 -05:00
22681878a8 Update ch05.ipynb (#297) SSebo 2024-08-05 20:12:27 +08:00
7643c6c0c4 Update ch05.ipynb (#297) SSebo 2024-08-05 20:12:27 +08:00
e4ed2c91cc Update README.md Sebastian Raschka 2024-08-04 16:06:38 -05:00
16e83434b5 Update README.md Sebastian Raschka 2024-08-04 16:06:38 -05:00
f302f5e8d5 improve latex rendering in dpo notebook rasbt 2024-08-04 09:19:54 -05:00
60aada801b improve latex rendering in dpo notebook rasbt 2024-08-04 09:19:54 -05:00
2c6cdb497f Update matplotlib tests on Windows (#295) Sebastian Raschka 2024-08-04 09:18:19 -05:00
e130ca293c Update matplotlib tests on Windows (#295) Sebastian Raschka 2024-08-04 09:18:19 -05:00
09dc080cf3 Direct Preference Optimization from scratch (#294) Sebastian Raschka 2024-08-04 08:57:36 -05:00
52435804eb Direct Preference Optimization from scratch (#294) Sebastian Raschka 2024-08-04 08:57:36 -05:00
3ea0798d44 Update README.md Sebastian Raschka 2024-08-01 18:17:42 -05:00
ff7a6db212 Update README.md Sebastian Raschka 2024-08-01 18:17:42 -05:00
eabeab04d1 restructure into local and cloud setup rasbt 2024-07-31 06:59:04 -05:00
b5fc1a6061 restructure into local and cloud setup rasbt 2024-07-31 06:59:04 -05:00
b4b2ac668c Add video tutorial Sebastian Raschka 2024-07-30 06:57:46 -05:00
1b100179c0 Add video tutorial Sebastian Raschka 2024-07-30 06:57:46 -05:00
6c2d34b03d Update README.md Sebastian Raschka 2024-07-30 06:55:41 -05:00
f5a003744e Update README.md Sebastian Raschka 2024-07-30 06:55:41 -05:00
358717870b add state_dict example rasbt 2024-07-28 14:15:32 -05:00
0dad0a3c04 add state_dict example rasbt 2024-07-28 14:15:32 -05:00
ce33e706ba Fix 8-billion-parameter spelling rasbt 2024-07-28 10:48:56 -05:00
a7869ad2bf Fix 8-billion-parameter spelling rasbt 2024-07-28 10:48:56 -05:00
60752e3b3a fixed typos and formatting (#291) Daniel Kleine 2024-07-28 17:04:33 +02:00
9a3b04f92f fixed typos and formatting (#291) Daniel Kleine 2024-07-28 17:04:33 +02:00
75a24c4897 Update README.md Sebastian Raschka 2024-07-28 09:28:11 -05:00
9bf5d67d61 Update README.md Sebastian Raschka 2024-07-28 09:28:11 -05:00
011ee83ea7 Update README.md Sebastian Raschka 2024-07-28 08:21:38 -05:00
4f7f5bd443 Update README.md Sebastian Raschka 2024-07-28 08:21:38 -05:00
1b86a61a2e buffer tutorial Sebastian Raschka 2024-07-27 17:06:16 -05:00
f4fc0ededd buffer tutorial Sebastian Raschka 2024-07-27 17:06:16 -05:00
f66c089f0b Test with PyTorch 2.0 and 2.4 (#290) Sebastian Raschka 2024-07-27 15:09:02 -05:00
263eee8921 Test with PyTorch 2.0 and 2.4 (#290) Sebastian Raschka 2024-07-27 15:09:02 -05:00
7bf0d13ee8 Update README.md Sebastian Raschka 2024-07-27 13:39:23 -05:00
9a2a269530 Update README.md Sebastian Raschka 2024-07-27 13:39:23 -05:00
7eb3f941a4 update figure caption rasbt 2024-07-27 13:30:18 -05:00
9eb2e2a5f6 update figure caption rasbt 2024-07-27 13:30:18 -05:00
4a0a8cd9e4 formatting rasbt 2024-07-27 09:51:24 -05:00
c87e4364b7 formatting rasbt 2024-07-27 09:51:24 -05:00
2b7bd46a93 Generate preference dataset with Llama 3.1 70B (#289) Sebastian Raschka 2024-07-27 09:44:04 -05:00
99af403b9f Generate preference dataset with Llama 3.1 70B (#289) Sebastian Raschka 2024-07-27 09:44:04 -05:00
3da0f18da7 update rasbt 2024-07-27 07:12:42 -05:00
7f1e071fff update rasbt 2024-07-27 07:12:42 -05:00
1e873d4cbc Understanding PyTorch Buffers (#288) Sebastian Raschka 2024-07-26 08:45:36 -05:00
deea13e5c2 Understanding PyTorch Buffers (#288) Sebastian Raschka 2024-07-26 08:45:36 -05:00
6dd8666d9c Test code in pytorch 2.4 (#285) Sebastian Raschka 2024-07-24 21:53:41 -05:00
08040f024c Test code in pytorch 2.4 (#285) Sebastian Raschka 2024-07-24 21:53:41 -05:00
bce3a708f9 47,678-->48,725 (#281) TITC 2024-07-23 10:24:57 +08:00
6cbe6520a2 47,678-->48,725 (#281) TITC 2024-07-23 10:24:57 +08:00
5d8a0c22ab use utf-8 encoding in json rasbt 2024-07-22 07:01:54 -05:00
46fcde15d8 use utf-8 encoding in json rasbt 2024-07-22 07:01:54 -05:00
3f6f2af3a3 Simplify embedding vs linear layer code (#278) Sebastian Raschka 2024-07-21 10:21:10 -07:00
fa56c80402 Simplify embedding vs linear layer code (#278) Sebastian Raschka 2024-07-21 10:21:10 -07:00
7356020b44 Update README.md Sebastian Raschka 2024-07-21 10:42:28 -05:00
bbe09e9799 Update README.md Sebastian Raschka 2024-07-21 10:42:28 -05:00
b80b14cf09 Update README.md Sebastian Raschka 2024-07-21 10:41:53 -05:00
0d28079a8a Update README.md Sebastian Raschka 2024-07-21 10:41:53 -05:00

... 6 7 8 9 10 ...