LLMs-from-scratch

books/LLMs-from-scratch

Fork 0

mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00

Commit Graph

Select branches

Hide Pull Requests

cleaning

debug-exclude-newer

dockerfile

fix-qwen3-tokeniizer-eos

fix_masking_kv_cache

gemma-3

head-dim

main

qwen-tokenizer-fix

requirements-update

uv-improvements

weezymatt-bugfix/semantic-error

#1

#10

#100

#1000

#1003

#1004

#1004

#101

#102

#104

#105

#106

#107

#109

#110

#111

#113

#115

#116

#117

#118

#119

#12

#120

#121

#122

#124

#125

#127

#128

#13

#131

#132

#133

#134

#135

#136

#137

#138

#139

#14

#141

#142

#143

#144

#149

#151

#152

#153

#154

#156

#157

#158

#159

#160

#161

#162

#163

#164

#165

#166

#169

#17

#170

#171

#173

#174

#176

#177

#178

#179

#18

#180

#182

#183

#184

#187

#189

#19

#193

#195

#196

#197

#198

#199

#2

#20

#200

#201

#203

#204

#205

#206

#207

#208

#211

#212

#213

#216

#218

#219

#22

#222

#223

#224

#227

#228

#229

#233

#234

#235

#236

#237

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#250

#252

#253

#254

#258

#259

#26

#260

#261

#263

#264

#265

#269

#27

#270

#274

#278

#28

#281

#285

#288

#289

#29

#290

#291

#294

#295

#297

#298

#3

#300

#301

#303

#304

#305

#307

#31

#313

#314

#318

#319

#32

#320

#321

#323

#324

#325

#329

#33

#333

#335

#336

#337

#339

#344

#345

#346

#349

#352

#353

#356

#36

#360

#361

#366

#367

#368

#369

#37

#372

#373

#375

#376

#378

#38

#380

#381

#383

#384

#386

#388

#389

#39

#391

#393

#395

#396

#397

#398

#4

#400

#401

#403

#404

#405

#406

#407

#408

#412

#413

#414

#419

#420

#421

#423

#427

#432

#436

#437

#438

#439

#441

#451

#458

#461

#465

#465

#466

#468

#469

#470

#471

#473

#479

#481

#483

#484

#487

#492

#495

#496

#498

#5

#50

#500

#504

#506

#507

#509

#511

#512

#516

#518

#519

#52

#520

#521

#522

#523

#524

#525

#526

#529

#530

#531

#533

#534

#535

#536

#537

#539

#54

#540

#541

#542

#543

#544

#545

#546

#547

#549

#55

#550

#551

#553

#555

#557

#56

#560

#561

#562

#563

#565

#571

#572

#573

#574

#575

#576

#577

#578

#581

#582

#583

#584

#585

#588

#589

#590

#591

#592

#593

#596

#597

#598

#6

#600

#602

#604

#605

#606

#611

#612

#613

#614

#615

#616

#617

#620

#621

#627

#628

#629

#63

#630

#638

#64

#640

#642

#643

#646

#649

#65

#653

#658

#66

#660

#661

#663

#665

#667

#668

#671

#672

#673

#674

#676

#677

#678

#68

#680

#682

#685

#687

#688

#69

#695

#696

#697

#698

#699

#7

#70

#700

#702

#703

#704

#705

#706

#708

#709

#71

#710

#711

#713

#714

#715

#716

#719

#722

#723

#725

#726

#727

#728

#729

#73

#730

#732

#733

#735

#737

#738

#74

#741

#744

#747

#748

#749

#75

#750

#752

#758

#76

#760

#761

#764

#767

#768

#769

#77

#770

#771

#773

#774

#775

#776

#78

#780

#781

#786

#788

#79

#793

#798

#799

#800

#801

#802

#805

#807

#809

#81

#810

#811

#812

#814

#815

#817

#818

#819

#82

#820

#823

#826

#828

#829

#83

#830

#831

#832

#833

#834

#839

#84

#841

#842

#843

#845

#847

#849

#850

#851

#858

#859

#861

#862

#863

#865

#866

#867

#869

#870

#874

#875

#876

#878

#879

#880

#881

#883

#887

#888

#89

#890

#891

#892

#893

#897

#898

#9

#90

#900

#901

#903

#904

#906

#909

#910

#914

#915

#916

#918

#920

#925

#926

#927

#933

#935

#937

#938

#940

#942

#943

#945

#946

#949

#950

#951

#952

#953

#954

#957

#958

#959

#96

#960

#961

#962

#965

#969

#97

#974

#975

#976

#981

#987

#989

#99

#991

#993

#994

#998

#999

354bb35726 use training set len rasbt 2024-04-29 21:50:07 -05:00
0ac19a1e50 use training set len rasbt 2024-04-29 21:50:07 -05:00
a5b353667d Rename drop_resid to drop_shortcut (#136) Sebastian Raschka 2024-04-28 14:31:27 -05:00
97ed38116a Rename drop_resid to drop_shortcut (#136) Sebastian Raschka 2024-04-28 14:31:27 -05:00
d1edfcb63f add roberta option (#135) Sebastian Raschka 2024-04-28 13:57:36 -05:00
70cd174091 add roberta option (#135) Sebastian Raschka 2024-04-28 13:57:36 -05:00
d088753fca Formatting improvements (#134) Sebastian Raschka 2024-04-28 12:05:32 -05:00
ca47c5e4b2 Formatting improvements (#134) Sebastian Raschka 2024-04-28 12:05:32 -05:00
5ae5e9df3b Try windows runners (#133) Sebastian Raschka 2024-04-28 07:39:23 -05:00
9a5d4d8ac9 Try windows runners (#133) Sebastian Raschka 2024-04-28 07:39:23 -05:00
1887b89af6 Update README.md Sebastian Raschka 2024-04-27 07:59:42 -05:00
e1d094b655 Update README.md Sebastian Raschka 2024-04-27 07:59:42 -05:00
0f03c20483 Data loader intuition with numbers (#132) Sebastian Raschka 2024-04-27 07:56:41 -05:00
fc3d70f72f Data loader intuition with numbers (#132) Sebastian Raschka 2024-04-27 07:56:41 -05:00
0528446584 Make code more consistent and add projection layer (#131) Sebastian Raschka 2024-04-26 17:13:08 -05:00
4adb96d7ee Make code more consistent and add projection layer (#131) Sebastian Raschka 2024-04-26 17:13:08 -05:00
4bbd476e7a IMDB experiments (#128) Sebastian Raschka 2024-04-25 07:20:53 -05:00
59b4fd3e25 IMDB experiments (#128) Sebastian Raschka 2024-04-25 07:20:53 -05:00
51f4980a42 style checks rasbt 2024-04-24 07:48:51 -05:00
258aff3e9a style checks rasbt 2024-04-24 07:48:51 -05:00
d311bae25a add usage rasbt 2024-04-24 07:27:04 -05:00
46d09b30d9 add usage rasbt 2024-04-24 07:27:04 -05:00
fb54b064c9 add more experiments rasbt 2024-04-24 07:23:11 -05:00
5ef438aa3b add more experiments rasbt 2024-04-24 07:23:11 -05:00
b2cf956054 update requirements rasbt 2024-04-24 06:38:02 -05:00
642f819910 update requirements rasbt 2024-04-24 06:38:02 -05:00
881075aeb0 rename folder rasbt 2024-04-23 21:02:57 -05:00
3b4484029d rename folder rasbt 2024-04-23 21:02:57 -05:00
379a8ab39c update figures in bonus notebook rasbt 2024-04-23 21:01:27 -05:00
c7cdedf981 update figures in bonus notebook rasbt 2024-04-23 21:01:27 -05:00
f656ef996d Chapter 6 ablation studies (#127) Sebastian Raschka 2024-04-23 09:51:52 -05:00
16964a6486 Chapter 6 ablation studies (#127) Sebastian Raschka 2024-04-23 09:51:52 -05:00
44a009f7e6 update stride wording Sebastian Raschka 2024-04-22 20:40:48 -05:00
0bd2608a6c update stride wording Sebastian Raschka 2024-04-22 20:40:48 -05:00
4abaa168ac fix merge conflict rasbt 2024-04-22 07:05:40 -05:00
90d239b4f7 fix merge conflict rasbt 2024-04-22 07:05:40 -05:00
df4fc602d8 update numbering rasbt 2024-04-22 07:00:20 -05:00
72be9f4e8e update numbering rasbt 2024-04-22 07:00:20 -05:00
2dd7bf9cda file header rasbt 2024-04-22 06:51:54 -05:00
868955f6a5 file header rasbt 2024-04-22 06:51:54 -05:00
79d40c25bf remove requests dependency (#125) Sebastian Raschka 2024-04-21 14:15:05 -05:00
44b3815960 remove requests dependency (#125) Sebastian Raschka 2024-04-21 14:15:05 -05:00
90fb214822 update figures rasbt 2024-04-20 11:42:03 -05:00
d202cabdee update figures rasbt 2024-04-20 11:42:03 -05:00
4557d5830e Return nan if val loader is empty (#124) Sebastian Raschka 2024-04-20 08:02:30 -05:00
c70ddff558 Return nan if val loader is empty (#124) Sebastian Raschka 2024-04-20 08:02:30 -05:00
b5878a80ff Use dim=-1 for consistency (#122) Sebastian Raschka 2024-04-18 05:56:23 -05:00
7740d556a0 Use dim=-1 for consistency (#122) Sebastian Raschka 2024-04-18 05:56:23 -05:00
49f01d06d0 Calculate warmup steps as a fraction (#121) Sebastian Raschka 2024-04-17 20:30:42 -05:00
e0ce5ca459 Calculate warmup steps as a fraction (#121) Sebastian Raschka 2024-04-17 20:30:42 -05:00
fdcb8f99fa extend setup instructions (#120) Sebastian Raschka 2024-04-15 22:05:03 -04:00
8d53e8d8cd extend setup instructions (#120) Sebastian Raschka 2024-04-15 22:05:03 -04:00
e5c567ad02 Update README.md Sebastian Raschka 2024-04-14 12:42:02 -05:00
b59eacb01f Update README.md Sebastian Raschka 2024-04-14 12:42:02 -05:00
6aa58ee2a9 shorten badge names rasbt 2024-04-14 12:41:23 -05:00
0729afa835 shorten badge names rasbt 2024-04-14 12:41:23 -05:00
dc7bfde0e7 Check hyperlink badge Sebastian Raschka 2024-04-14 12:38:55 -05:00
a9c1b94d09 Check hyperlink badge Sebastian Raschka 2024-04-14 12:38:55 -05:00
ef2de4718e use torch no grad for loss (#119) Sebastian Raschka 2024-04-14 09:13:07 -04:00
155ac03f61 use torch no grad for loss (#119) Sebastian Raschka 2024-04-14 09:13:07 -04:00
98f1e97452 Update README.md Sebastian Raschka 2024-04-13 15:04:08 -05:00
a3a5574758 Update README.md Sebastian Raschka 2024-04-13 15:04:08 -05:00
bae4b0fb08 Make datesets and loaders compatible with multiprocessing (#118) Sebastian Raschka 2024-04-13 14:57:56 -04:00
dd51d4ad83 Make datesets and loaders compatible with multiprocessing (#118) Sebastian Raschka 2024-04-13 14:57:56 -04:00
8fe63a9a0e use correct lr Sebastian Raschka 2024-04-12 19:55:07 -04:00
9f3f231ac7 use correct lr Sebastian Raschka 2024-04-12 19:55:07 -04:00
bbce1cb143 Automated link checking (#117) Sebastian Raschka 2024-04-12 19:08:34 -04:00
55ebabf95c Automated link checking (#117) Sebastian Raschka 2024-04-12 19:08:34 -04:00
d1a4157d71 improve check-links.yml Sebastian Raschka 2024-04-11 17:23:15 -04:00
33b27368a3 improve check-links.yml Sebastian Raschka 2024-04-11 17:23:15 -04:00
02f8b2ee26 unit test indicator placement Sebastian Raschka 2024-04-10 22:15:07 -04:00
5ca4384eb7 unit test indicator placement Sebastian Raschka 2024-04-10 22:15:07 -04:00
40539a2542 setup instruction note Sebastian Raschka 2024-04-10 22:13:22 -04:00
ae3020bc12 setup instruction note Sebastian Raschka 2024-04-10 22:13:22 -04:00
790d0808b2 Organized setup instructions (#115) Sebastian Raschka 2024-04-10 22:09:46 -04:00
e757091301 Organized setup instructions (#115) Sebastian Raschka 2024-04-10 22:09:46 -04:00
0b866c133f Use instance tokenizer (#116) James Holcombe 2024-04-10 21:16:19 -04:00
05718c6b94 Use instance tokenizer (#116) James Holcombe 2024-04-10 21:16:19 -04:00
94f6582cff move devcontainer (#113) Sebastian Raschka 2024-04-08 20:49:37 -04:00
028a346498 move devcontainer (#113) Sebastian Raschka 2024-04-08 20:49:37 -04:00
b01204ca3a Added PDF display support to Docker image and VS Code and updated first step for gutenberg project (#111) Daniel Kleine 2024-04-09 02:37:55 +02:00
61b6e35ddf Added PDF display support to Docker image and VS Code and updated first step for gutenberg project (#111) Daniel Kleine 2024-04-09 02:37:55 +02:00
8462a777bd address suggestions to improve clarity rasbt 2024-04-07 08:41:09 -05:00
58d5bd9e39 address suggestions to improve clarity rasbt 2024-04-07 08:41:09 -05:00
040ce578be renumber exercises rasbt 2024-04-07 06:03:41 -05:00
42eda8b70f renumber exercises rasbt 2024-04-07 06:03:41 -05:00
c76941e061 added VS Code extensions recommendations (#110) Daniel Kleine 2024-04-05 15:30:56 +02:00
e43e0760f9 added VS Code extensions recommendations (#110) Daniel Kleine 2024-04-05 15:30:56 +02:00
84b785ddd0 variable renaming for clarity rasbt 2024-04-05 07:26:42 -05:00
c5a17393fc variable renaming for clarity rasbt 2024-04-05 07:26:42 -05:00
c31e99720d rename hparams to settings rasbt 2024-04-05 07:24:46 -05:00
8c36399e7c rename hparams to settings rasbt 2024-04-05 07:24:46 -05:00
7d0b9b78b0 Updated devcontainer, .gitignore and README for gutenberg project (#107) Daniel Kleine 2024-04-05 13:53:01 +02:00
44c0494406 Updated devcontainer, .gitignore and README for gutenberg project (#107) Daniel Kleine 2024-04-05 13:53:01 +02:00
25f533efe0 Fix Loss in Gutenberg bonus section (#109) Sebastian Raschka 2024-04-04 20:54:09 -05:00
adc2964fc5 Fix Loss in Gutenberg bonus section (#109) Sebastian Raschka 2024-04-04 20:54:09 -05:00
c8cffefb6f cleanup rasbt 2024-04-04 07:58:41 -05:00
6de0417321 cleanup rasbt 2024-04-04 07:58:41 -05:00
ccd7cebbb3 Rename variable to context_length to make it easier on readers (#106) Sebastian Raschka 2024-04-04 07:27:41 -05:00
2de60d1bfb Rename variable to context_length to make it easier on readers (#106) Sebastian Raschka 2024-04-04 07:27:41 -05:00

... 13 14 15 16 17 ...