LLMs-from-scratch

books/LLMs-from-scratch

Fork 0

mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00

Commit Graph

Select branches

Hide Pull Requests

cleaning

debug-exclude-newer

dockerfile

fix-qwen3-tokeniizer-eos

fix_masking_kv_cache

gemma-3

head-dim

main

qwen-tokenizer-fix

requirements-update

uv-improvements

weezymatt-bugfix/semantic-error

#1

#10

#100

#1000

#1003

#1004

#1004

#101

#102

#104

#105

#106

#107

#109

#110

#111

#113

#115

#116

#117

#118

#119

#12

#120

#121

#122

#124

#125

#127

#128

#13

#131

#132

#133

#134

#135

#136

#137

#138

#139

#14

#141

#142

#143

#144

#149

#151

#152

#153

#154

#156

#157

#158

#159

#160

#161

#162

#163

#164

#165

#166

#169

#17

#170

#171

#173

#174

#176

#177

#178

#179

#18

#180

#182

#183

#184

#187

#189

#19

#193

#195

#196

#197

#198

#199

#2

#20

#200

#201

#203

#204

#205

#206

#207

#208

#211

#212

#213

#216

#218

#219

#22

#222

#223

#224

#227

#228

#229

#233

#234

#235

#236

#237

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#250

#252

#253

#254

#258

#259

#26

#260

#261

#263

#264

#265

#269

#27

#270

#274

#278

#28

#281

#285

#288

#289

#29

#290

#291

#294

#295

#297

#298

#3

#300

#301

#303

#304

#305

#307

#31

#313

#314

#318

#319

#32

#320

#321

#323

#324

#325

#329

#33

#333

#335

#336

#337

#339

#344

#345

#346

#349

#352

#353

#356

#36

#360

#361

#366

#367

#368

#369

#37

#372

#373

#375

#376

#378

#38

#380

#381

#383

#384

#386

#388

#389

#39

#391

#393

#395

#396

#397

#398

#4

#400

#401

#403

#404

#405

#406

#407

#408

#412

#413

#414

#419

#420

#421

#423

#427

#432

#436

#437

#438

#439

#441

#451

#458

#461

#465

#465

#466

#468

#469

#470

#471

#473

#479

#481

#483

#484

#487

#492

#495

#496

#498

#5

#50

#500

#504

#506

#507

#509

#511

#512

#516

#518

#519

#52

#520

#521

#522

#523

#524

#525

#526

#529

#530

#531

#533

#534

#535

#536

#537

#539

#54

#540

#541

#542

#543

#544

#545

#546

#547

#549

#55

#550

#551

#553

#555

#557

#56

#560

#561

#562

#563

#565

#571

#572

#573

#574

#575

#576

#577

#578

#581

#582

#583

#584

#585

#588

#589

#590

#591

#592

#593

#596

#597

#598

#6

#600

#602

#604

#605

#606

#611

#612

#613

#614

#615

#616

#617

#620

#621

#627

#628

#629

#63

#630

#638

#64

#640

#642

#643

#646

#649

#65

#653

#658

#66

#660

#661

#663

#665

#667

#668

#671

#672

#673

#674

#676

#677

#678

#68

#680

#682

#685

#687

#688

#69

#695

#696

#697

#698

#699

#7

#70

#700

#702

#703

#704

#705

#706

#708

#709

#71

#710

#711

#713

#714

#715

#716

#719

#722

#723

#725

#726

#727

#728

#729

#73

#730

#732

#733

#735

#737

#738

#74

#741

#744

#747

#748

#749

#75

#750

#752

#758

#76

#760

#761

#764

#767

#768

#769

#77

#770

#771

#773

#774

#775

#776

#78

#780

#781

#786

#788

#79

#793

#798

#799

#800

#801

#802

#805

#807

#809

#81

#810

#811

#812

#814

#815

#817

#818

#819

#82

#820

#823

#826

#828

#829

#83

#830

#831

#832

#833

#834

#839

#84

#841

#842

#843

#845

#847

#849

#850

#851

#858

#859

#861

#862

#863

#865

#866

#867

#869

#870

#874

#875

#876

#878

#879

#880

#881

#883

#887

#888

#89

#890

#891

#892

#893

#897

#898

#9

#90

#900

#901

#903

#904

#906

#909

#910

#914

#915

#916

#918

#920

#925

#926

#927

#933

#935

#937

#938

#940

#942

#943

#945

#946

#949

#950

#951

#952

#953

#954

#957

#958

#959

#96

#960

#961

#962

#965

#969

#97

#974

#975

#976

#981

#987

#989

#99

#991

#993

#994

#998

#999

fd6d0d13e4 Update requirements.txt Sebastian Raschka 2024-05-26 07:02:13 -05:00
602bc07fce Update requirements.txt Sebastian Raschka 2024-05-26 07:02:13 -05:00
88737211ba Add recent bonus materials Sebastian Raschka 2024-05-26 06:40:57 -05:00
6f8b125a6e Add recent bonus materials Sebastian Raschka 2024-05-26 06:40:57 -05:00
a634c829d2 Merge pull request #179 from rasbt/create-json-entries Sebastian Raschka 2024-05-25 19:17:00 -04:00
df44b254dc Merge pull request #179 from rasbt/create-json-entries Sebastian Raschka 2024-05-25 19:17:00 -04:00
5d48181434 skip platform links rasbt 2024-05-25 18:09:19 -05:00
80c8584c9d skip platform links rasbt 2024-05-25 18:09:19 -05:00
d4efdfe4d9 OpenAI API example to create instruction examples rasbt 2024-05-25 18:03:37 -05:00
0c253ad843 OpenAI API example to create instruction examples rasbt 2024-05-25 18:03:37 -05:00
443c1dd167 pep8 rasbt 2024-05-25 11:42:59 -05:00
9c933008ed pep8 rasbt 2024-05-25 11:42:59 -05:00
7a600a6271 style and requirements rasbt 2024-05-25 11:38:55 -05:00
b5dd2259b8 style and requirements rasbt 2024-05-25 11:38:55 -05:00
477dea73ec dataset utils rasbt 2024-05-25 11:22:51 -05:00
abdb2fc61f dataset utils rasbt 2024-05-25 11:22:51 -05:00
605b4b03d7 Update README.md Sebastian Raschka 2024-05-25 10:45:29 -05:00
49b8e2e767 Update README.md Sebastian Raschka 2024-05-25 10:45:29 -05:00
a0b5603423 Make header more clear Sebastian Raschka 2024-05-25 10:44:12 -05:00
39a831a4d8 Make header more clear Sebastian Raschka 2024-05-25 10:44:12 -05:00
fe8bb9291e update formatting rasbt 2024-05-24 07:20:37 -05:00
98d453b666 update formatting rasbt 2024-05-24 07:20:37 -05:00
a9703f4603 update header rasbt 2024-05-24 07:15:41 -05:00
80113872c7 update header rasbt 2024-05-24 07:15:41 -05:00
ee6afe260a clarify Sebastian Raschka 2024-05-24 06:58:12 -05:00
96e0a2be08 clarify Sebastian Raschka 2024-05-24 06:58:12 -05:00
ecb1788a9a Update cover Sebastian Raschka 2024-05-24 06:56:20 -05:00
3eec1546f5 Update cover Sebastian Raschka 2024-05-24 06:56:20 -05:00
7589df7b25 Update cover and add amazon link Sebastian Raschka 2024-05-24 06:55:19 -05:00
0a85d34875 Update cover and add amazon link Sebastian Raschka 2024-05-24 06:55:19 -05:00
313979a4ce Merge pull request #177 from rasbt/flops-analysis Sebastian Raschka 2024-05-23 21:45:16 -04:00
5477970fe7 Merge pull request #177 from rasbt/flops-analysis Sebastian Raschka 2024-05-23 21:45:16 -04:00
d93fbbd4b9 flops analysis rasbt 2024-05-23 20:35:41 -05:00
e5e6aaf9f1 flops analysis rasbt 2024-05-23 20:35:41 -05:00
aa084656e0 update how to retrieve learning rate rasbt 2024-05-23 17:19:01 -05:00
b40c260859 update how to retrieve learning rate rasbt 2024-05-23 17:19:01 -05:00
30ba6a3f4b trainable token -> trainable token position rasbt 2024-05-23 11:43:20 -05:00
b43a7a8820 trainable token -> trainable token position rasbt 2024-05-23 11:43:20 -05:00
209a103d66 Merge pull request #176 from rasbt/dataset-length-warning Sebastian Raschka 2024-05-23 07:58:47 -04:00
95cea64824 Merge pull request #176 from rasbt/dataset-length-warning Sebastian Raschka 2024-05-23 07:58:47 -04:00
c35cf65dbf add assertion about data set length rasbt 2024-05-23 06:50:43 -05:00
18e729643d add assertion about data set length rasbt 2024-05-23 06:50:43 -05:00
66f02723ca Merge pull request #174 from rasbt/update-regex Sebastian Raschka 2024-05-22 21:37:50 -04:00
ec70194d19 Merge pull request #174 from rasbt/update-regex Sebastian Raschka 2024-05-22 21:37:50 -04:00
1407085f07 reset cell count for better nbdiff rasbt 2024-05-22 20:27:09 -05:00
5b1dcf0b33 reset cell count for better nbdiff rasbt 2024-05-22 20:27:09 -05:00
85c3210105 update regex rasbt 2024-05-22 20:15:31 -05:00
7686d4569f update regex rasbt 2024-05-22 20:15:31 -05:00
1bf24669f8 Merge pull request #173 from rasbt/device-setting Sebastian Raschka 2024-05-22 18:59:58 -04:00
9587b58cf7 Merge pull request #173 from rasbt/device-setting Sebastian Raschka 2024-05-22 18:59:58 -04:00
c4cd48475c Fix device setting rasbt 2024-05-22 17:51:51 -05:00
86f6c2df43 Fix device setting rasbt 2024-05-22 17:51:51 -05:00
6aa4572ff7 Merge pull request #171 from d-kleine/main Sebastian Raschka 2024-05-22 18:42:29 -04:00
7a2a157844 Merge pull request #171 from d-kleine/main Sebastian Raschka 2024-05-22 18:42:29 -04:00
4b0fdab1de removed empty line Daniel Kleine 2024-05-22 16:15:13 +00:00
982cbe5e40 removed empty line Daniel Kleine 2024-05-22 16:15:13 +00:00
a81ba9bd8b fixed last_two_blocks Daniel Kleine 2024-05-22 02:02:43 +00:00
195849cf8d fixed last_two_blocks Daniel Kleine 2024-05-22 02:02:43 +00:00
ef17f9bda0 Merge pull request #170 from rasbt/last-two-blocks Sebastian Raschka 2024-05-21 20:57:55 -04:00
9f8377a154 Merge pull request #170 from rasbt/last-two-blocks Sebastian Raschka 2024-05-21 20:57:55 -04:00
80d857c605 fix table alignment rasbt 2024-05-21 19:51:22 -05:00
caf3725001 fix table alignment rasbt 2024-05-21 19:51:22 -05:00
8a27baf7c2 experiment with last two blocks rasbt 2024-05-21 19:49:34 -05:00
725bed56f7 experiment with last two blocks rasbt 2024-05-21 19:49:34 -05:00
d2f9f908dc Merge pull request #169 from d-kleine/main Sebastian Raschka 2024-05-21 20:27:33 -04:00
af73fa6055 Merge pull request #169 from d-kleine/main Sebastian Raschka 2024-05-21 20:27:33 -04:00
130a69ce27 improved readability of Additional Experiments table Daniel Kleine 2024-05-21 19:26:25 +00:00
f39087e573 improved readability of Additional Experiments table Daniel Kleine 2024-05-21 19:26:25 +00:00
69da9ed447 removed unnecessary .gitignore Daniel Kleine 2024-05-21 19:25:16 +00:00
aa67e6e1ac removed unnecessary .gitignore Daniel Kleine 2024-05-21 19:25:16 +00:00
3d0b94b504 Merge pull request #166 from rasbt/update-lora-init Sebastian Raschka 2024-05-19 21:33:28 -04:00
451a62994d Merge pull request #166 from rasbt/update-lora-init Sebastian Raschka 2024-05-19 21:33:28 -04:00
5e1ba8343d 100x -> 50x rasbt 2024-05-19 20:26:53 -05:00
f3a2e93160 100x -> 50x rasbt 2024-05-19 20:26:53 -05:00
9512957b07 use macbook version rasbt 2024-05-19 20:19:02 -05:00
0d48725b5c use macbook version rasbt 2024-05-19 20:19:02 -05:00
7b9b53c9f2 update lora init rasbt 2024-05-19 20:11:56 -05:00
c2028871e4 update lora init rasbt 2024-05-19 20:11:56 -05:00
3b72e55c26 remove duplicated text rasbt 2024-05-19 11:34:47 -05:00
a8a28017c0 remove duplicated text rasbt 2024-05-19 11:34:47 -05:00
15e103bf27 Merge pull request #165 from d-kleine/main Sebastian Raschka 2024-05-19 12:34:31 -04:00
1dd9c0a71e Merge pull request #165 from d-kleine/main Sebastian Raschka 2024-05-19 12:34:31 -04:00
7b397fcd46 updated .gitignore Daniel Kleine 2024-05-19 16:07:20 +00:00
e7914182c6 updated .gitignore Daniel Kleine 2024-05-19 16:07:20 +00:00
bc5cbbf1bd change defaults to 0 temp rasbt 2024-05-19 09:04:49 -05:00
a5593f9860 change defaults to 0 temp rasbt 2024-05-19 09:04:49 -05:00
59f5ed8d68 use default value for temperature rasbt 2024-05-19 08:48:10 -05:00
1463b2ae47 use default value for temperature rasbt 2024-05-19 08:48:10 -05:00
faffebae4b add ignore index experiment rasbt 2024-05-19 07:24:49 -05:00
1b340c9eb6 add ignore index experiment rasbt 2024-05-19 07:24:49 -05:00
5541f7c8fe add test mode for dataset download rasbt 2024-05-18 17:38:19 -05:00
02e6f06a11 add test mode for dataset download rasbt 2024-05-18 17:38:19 -05:00
bdea15f6c6 new experiment w/o causal mask rasbt 2024-05-18 17:03:36 -05:00
5ef4edf2b5 new experiment w/o causal mask rasbt 2024-05-18 17:03:36 -05:00
00a466f0b9 fix row number typo Sebastian Raschka 2024-05-18 15:54:13 -05:00
57634f2045 fix row number typo Sebastian Raschka 2024-05-18 15:54:13 -05:00
c05e573121 Merge pull request #164 from rasbt/eos_id-token Sebastian Raschka 2024-05-18 16:10:25 -04:00
e8212c3f7c Merge pull request #164 from rasbt/eos_id-token Sebastian Raschka 2024-05-18 16:10:25 -04:00
9d84935b69 add eos_id option for ch07 rasbt 2024-05-18 12:35:40 -05:00
4851d5a0fa add eos_id option for ch07 rasbt 2024-05-18 12:35:40 -05:00

... 10 11 12 13 14 ...