LLMs-from-scratch

books/LLMs-from-scratch

Fork 0

mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00

Commit Graph

Select branches

Hide Pull Requests

cleaning

debug-exclude-newer

dockerfile

fix-qwen3-tokeniizer-eos

fix_masking_kv_cache

gemma-3

head-dim

main

qwen-tokenizer-fix

requirements-update

uv-improvements

weezymatt-bugfix/semantic-error

#1

#10

#100

#1000

#1003

#1004

#1004

#101

#102

#104

#105

#106

#107

#109

#110

#111

#113

#115

#116

#117

#118

#119

#12

#120

#121

#122

#124

#125

#127

#128

#13

#131

#132

#133

#134

#135

#136

#137

#138

#139

#14

#141

#142

#143

#144

#149

#151

#152

#153

#154

#156

#157

#158

#159

#160

#161

#162

#163

#164

#165

#166

#169

#17

#170

#171

#173

#174

#176

#177

#178

#179

#18

#180

#182

#183

#184

#187

#189

#19

#193

#195

#196

#197

#198

#199

#2

#20

#200

#201

#203

#204

#205

#206

#207

#208

#211

#212

#213

#216

#218

#219

#22

#222

#223

#224

#227

#228

#229

#233

#234

#235

#236

#237

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#250

#252

#253

#254

#258

#259

#26

#260

#261

#263

#264

#265

#269

#27

#270

#274

#278

#28

#281

#285

#288

#289

#29

#290

#291

#294

#295

#297

#298

#3

#300

#301

#303

#304

#305

#307

#31

#313

#314

#318

#319

#32

#320

#321

#323

#324

#325

#329

#33

#333

#335

#336

#337

#339

#344

#345

#346

#349

#352

#353

#356

#36

#360

#361

#366

#367

#368

#369

#37

#372

#373

#375

#376

#378

#38

#380

#381

#383

#384

#386

#388

#389

#39

#391

#393

#395

#396

#397

#398

#4

#400

#401

#403

#404

#405

#406

#407

#408

#412

#413

#414

#419

#420

#421

#423

#427

#432

#436

#437

#438

#439

#441

#451

#458

#461

#465

#465

#466

#468

#469

#470

#471

#473

#479

#481

#483

#484

#487

#492

#495

#496

#498

#5

#50

#500

#504

#506

#507

#509

#511

#512

#516

#518

#519

#52

#520

#521

#522

#523

#524

#525

#526

#529

#530

#531

#533

#534

#535

#536

#537

#539

#54

#540

#541

#542

#543

#544

#545

#546

#547

#549

#55

#550

#551

#553

#555

#557

#56

#560

#561

#562

#563

#565

#571

#572

#573

#574

#575

#576

#577

#578

#581

#582

#583

#584

#585

#588

#589

#590

#591

#592

#593

#596

#597

#598

#6

#600

#602

#604

#605

#606

#611

#612

#613

#614

#615

#616

#617

#620

#621

#627

#628

#629

#63

#630

#638

#64

#640

#642

#643

#646

#649

#65

#653

#658

#66

#660

#661

#663

#665

#667

#668

#671

#672

#673

#674

#676

#677

#678

#68

#680

#682

#685

#687

#688

#69

#695

#696

#697

#698

#699

#7

#70

#700

#702

#703

#704

#705

#706

#708

#709

#71

#710

#711

#713

#714

#715

#716

#719

#722

#723

#725

#726

#727

#728

#729

#73

#730

#732

#733

#735

#737

#738

#74

#741

#744

#747

#748

#749

#75

#750

#752

#758

#76

#760

#761

#764

#767

#768

#769

#77

#770

#771

#773

#774

#775

#776

#78

#780

#781

#786

#788

#79

#793

#798

#799

#800

#801

#802

#805

#807

#809

#81

#810

#811

#812

#814

#815

#817

#818

#819

#82

#820

#823

#826

#828

#829

#83

#830

#831

#832

#833

#834

#839

#84

#841

#842

#843

#845

#847

#849

#850

#851

#858

#859

#861

#862

#863

#865

#866

#867

#869

#870

#874

#875

#876

#878

#879

#880

#881

#883

#887

#888

#89

#890

#891

#892

#893

#897

#898

#9

#90

#900

#901

#903

#904

#906

#909

#910

#914

#915

#916

#918

#920

#925

#926

#927

#933

#935

#937

#938

#940

#942

#943

#945

#946

#949

#950

#951

#952

#953

#954

#957

#958

#959

#96

#960

#961

#962

#965

#969

#97

#974

#975

#976

#981

#987

#989

#99

#991

#993

#994

#998

#999

684562733a Add rsync to dockerfile Sebastian Raschka 2024-04-03 20:28:02 -05:00
a940373a14 Add rsync to dockerfile Sebastian Raschka 2024-04-03 20:28:02 -05:00
5beff4e25a Remove reundant dropout in MLP module (#105) Sebastian Raschka 2024-04-03 20:19:08 -05:00
3829ccdb34 Remove reundant dropout in MLP module (#105) Sebastian Raschka 2024-04-03 20:19:08 -05:00
edcae09884 improve importlib experience for windows users rasbt 2024-04-03 06:31:15 -05:00
dd115c1374 improve importlib experience for windows users rasbt 2024-04-03 06:31:15 -05:00
cd12b4a937 rename batch to text rasbt 2024-04-02 20:46:53 -05:00
e14585e954 rename batch to text rasbt 2024-04-02 20:46:53 -05:00
21140b98d4 update notes rasbt 2024-04-02 18:27:13 -05:00
7d1eadd0be update notes rasbt 2024-04-02 18:27:13 -05:00
0b47dfc381 "Typographical error (#104) Intelligence-Manifesto 2024-04-03 07:07:21 +08:00
96b1fde3f1 "Typographical error (#104) Intelligence-Manifesto 2024-04-03 07:07:21 +08:00
ec4fe5377d fixing the README for python setup under appendix-A (#102) Suman Debnath 2024-04-02 16:51:11 -04:00
7b7d23a4e1 fixing the README for python setup under appendix-A (#102) Suman Debnath 2024-04-02 16:51:11 -04:00
d081928e90 code -> markdown (#101) Intelligence-Manifesto 2024-04-03 03:37:45 +08:00
5a3f779405 code -> markdown (#101) Intelligence-Manifesto 2024-04-03 03:37:45 +08:00
809c944d30 Use max size properly Sebastian Raschka 2024-04-02 13:29:23 -05:00
2fab89d47e Use max size properly Sebastian Raschka 2024-04-02 13:29:23 -05:00
5af3834760 Gutenberg for Windows users (#99) Sebastian Raschka 2024-04-02 08:54:24 -05:00
4a617b8343 Gutenberg for Windows users (#99) Sebastian Raschka 2024-04-02 08:54:24 -05:00
f30dd2dd2b improve instructions rasbt 2024-04-02 07:12:22 -05:00
776a517d18 figure scaling rasbt 2024-04-01 08:05:01 -05:00
005835bfce make figures for appendix d rasbt 2024-03-31 21:24:41 -05:00
ac2bdb02bd make figures for appendix d rasbt 2024-03-31 21:22:49 -05:00
ee096986ea upload exercise solutions of ch05 rasbt 2024-03-31 20:28:51 -05:00
a6bd197897 updated github actions versions (#96) Daniel Kleine 2024-03-31 17:49:12 +02:00
83adc4a2ac add weight sizes rasbt 2024-03-31 08:45:14 -05:00
1c173e4f44 update figures rasbt 2024-03-30 09:43:51 -05:00
ca96b7aee5 minor updates rasbt 2024-03-29 20:42:32 -05:00
797cfb20de fix test rasbt 2024-03-29 09:03:36 -05:00
5b222e2d6f Fix small typos in ch02.ipynb (#89) Jeff Hammerbacher 2024-03-29 09:25:52 -04:00
71b6d1b7d4 Merge branch 'main' of https://github.com/rasbt/LLMs-from-scratch rasbt 2024-03-29 08:16:29 -05:00
ab1e56a323 reorg files and make standalone download file rasbt 2024-03-29 08:16:22 -05:00
4537dbf001 Update README.md Sebastian Raschka 2024-03-28 09:14:52 -05:00
3ad442ee90 skip version cell rasbt 2024-03-28 08:23:33 -05:00
3c5b288ca0 minor typo fixes rasbt 2024-03-28 08:02:05 -05:00
c10f5c9bf2 suggest galore rasbt 2024-03-27 19:58:32 -05:00
f24da86abe title case rasbt 2024-03-27 07:30:09 -05:00
713b3ee188 add readme rasbt 2024-03-27 07:29:16 -05:00
88b2dd780a make batch loss calculatution more efficient rasbt 2024-03-27 07:11:56 -05:00
3cb5a52a1b simplify calc_loss_loader rasbt 2024-03-26 20:34:50 -05:00
c88e8edf72 use probas in argmax rasbt 2024-03-26 08:38:27 -05:00
9cc9c4244e simplify rasbt 2024-03-26 07:52:36 -05:00
12fff1ddcb add endoftext token rasbt 2024-03-26 06:47:05 -05:00
de576296de simplify .view code rasbt 2024-03-25 08:09:31 -05:00
d4989e01c5 Update README.md Sebastian Raschka 2024-03-25 06:39:43 -05:00
45e7826954 Merge branch 'main' of https://github.com/rasbt/LLMs-from-scratch rasbt 2024-03-24 07:09:18 -05:00
c1d939c64e update chapter reference rasbt 2024-03-24 07:09:08 -05:00
0f0fdef576 small typo fixes rasbt 2024-03-23 11:28:20 -05:00
cf39abac04 Add and link bonus material (#84) Sebastian Raschka 2024-03-23 07:27:43 -05:00
35c6e12730 ignore ch05 tmp files rasbt 2024-03-23 06:52:08 -05:00
001507481e add colon and semicolon to tokenizer rasbt 2024-03-23 06:50:34 -05:00
5d02559993 small cosmetic updates (#83) Sebastian Raschka 2024-03-22 09:15:40 -05:00
075a9580ea reader proj and citation rasbt 2024-03-21 17:55:32 -05:00
4582995ced Add alternative weight loading strategy as backup (#82) Sebastian Raschka 2024-03-20 08:43:18 -05:00
820d5e3ed1 remove duplicate import rasbt 2024-03-19 20:41:35 -05:00
4bab1b6f33 remove redundant dir rasbt 2024-03-19 09:27:27 -05:00
a2cd8436cb Ch05 supplementary code (#81) Sebastian Raschka 2024-03-19 09:26:26 -05:00
861a2788f3 add check for small validation sets rasbt 2024-03-19 06:34:52 -05:00
ca96abac8a Set up basic test gh worklows (#79) Sebastian Raschka 2024-03-18 11:58:37 -05:00
9d6da22ebb Update pep8 (#78) Sebastian Raschka 2024-03-18 08:16:17 -05:00
e316cafd9f Update pep8-linter.yml Sebastian Raschka 2024-03-18 08:16:08 -05:00
329d046b5d simplify requirements file (#76) Sebastian Raschka 2024-03-18 08:00:49 -05:00
3e122fa656 Update pep8-linter.yml Sebastian Raschka 2024-03-18 07:57:17 -05:00
805e352737 Update pep8-linter.yml Sebastian Raschka 2024-03-18 07:47:30 -05:00
9acb589650 Update pep8-linter.yml Sebastian Raschka 2024-03-18 07:41:23 -05:00
e213a0cede Create pep8-linter.yml Sebastian Raschka 2024-03-18 07:00:28 -05:00
48253c4f88 Ch05 (#75) Sebastian Raschka 2024-03-17 21:07:19 -05:00
3e25216240 Merge pull request #74 from Intelligence-Manifesto/patch-7 Sebastian Raschka 2024-03-17 16:03:36 -05:00
c49aa22738 three -> four Intelligence-Manifesto 2024-03-17 23:40:44 +08:00
4fc6de7afa add notes rasbt 2024-03-17 09:29:06 -05:00
b58f66b684 Merge pull request #73 from rasbt/notes-ext-figures Sebastian Raschka 2024-03-17 09:09:08 -05:00
d60da19fd0 add more notes and embed figures externally to save space rasbt 2024-03-17 09:08:38 -05:00
b655e628a2 revert back to Apache 2.0 rasbt 2024-03-17 08:07:31 -05:00
861c296312 add imports and version on top rasbt 2024-03-16 09:50:00 -05:00
ff8657ac92 fix ipywidgets formatting issue rasbt 2024-03-16 08:35:43 -05:00
a155879d71 update formatting rasbt 2024-03-16 08:10:58 -05:00
44b0febe68 Merge pull request #71 from Intelligence-Manifesto/patch-6 Sebastian Raschka 2024-03-15 16:07:22 -05:00
d4b4e3d0f0 the above -> the following Intelligence-Manifesto 2024-03-15 05:00:28 +08:00
ee8efcbcf6 fix plotting rasbt 2024-03-14 07:41:40 -05:00
f25760c394 Merge pull request #70 from d-kleine/main Sebastian Raschka 2024-03-14 06:50:26 -05:00
809ea9d196 Update README.md Daniel Kleine 2024-03-13 18:51:20 +01:00
1870b4bacd update stride param rasbt 2024-03-13 08:39:59 -05:00
0b66c55950 Merge pull request #69 from rasbt/pretraining-on-proj-gutenberg Sebastian Raschka 2024-03-13 08:38:33 -05:00
0d517e98b9 update rasbt 2024-03-13 08:37:54 -05:00
f2c8eeb6b8 pretraining on project gutenberg rasbt 2024-03-13 08:34:39 -05:00
569f6bc7f0 benchmark numbers rasbt 2024-03-13 07:12:10 -05:00
319e919062 Merge pull request #68 from taihaozesong/fix_ch03_impl_wrapper Sebastian Raschka 2024-03-13 07:02:13 -05:00
f1fa9df15c Fix mha wrapper implementations in ch03 bonus taihaozesong 2024-03-13 18:02:26 +08:00
00b121a5af Merge pull request #66 from rasbt/appendix-d Sebastian Raschka 2024-03-11 07:08:57 -05:00
6a585e08bc Add appendix D rasbt 2024-03-11 07:07:36 -05:00
8c1871f16e Merge pull request #65 from d-kleine/main Sebastian Raschka 2024-03-11 06:39:33 -05:00
e524ddb6f4 Merge pull request #64 from shenxiangzhuang/fix/chap2_notebook_links Sebastian Raschka 2024-03-11 06:38:07 -05:00
3787227c41 Updated Dockerfile with following changes: * changed CUDA files to pytorch 2.0.1 (for reproducibility) * fixed RUN command (for updating Ubuntu and installing Git) Daniel Kleine 2024-03-11 08:06:48 +00:00
fa2864ddbf fix: inner links Xiangzhuang Shen 2024-03-11 10:52:56 +08:00
321f3d33f9 add cuda warmup rasbt 2024-03-10 10:31:49 -05:00
4d67a8be61 Merge pull request #63 from joel-foo/main Sebastian Raschka 2024-03-10 09:48:52 -05:00
dbb5e65a29 Remove duplicate cells joel-foo 2024-03-10 21:40:57 +08:00
244137e8a1 amend rasbt 2024-03-10 08:05:22 -05:00
76205521d7 different dropout behavior on macos and linux rasbt 2024-03-10 07:58:10 -05:00

... 14 15 16 17 18 ...