LLMs-from-scratch

books/LLMs-from-scratch

Fork 0

mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00

Commit Graph

Select branches

Hide Pull Requests

cleaning

debug-exclude-newer

dockerfile

fix-qwen3-tokeniizer-eos

fix_masking_kv_cache

gemma-3

head-dim

main

qwen-tokenizer-fix

requirements-update

uv-improvements

weezymatt-bugfix/semantic-error

#1

#10

#100

#1000

#1003

#1004

#1004

#101

#102

#104

#105

#106

#107

#109

#110

#111

#113

#115

#116

#117

#118

#119

#12

#120

#121

#122

#124

#125

#127

#128

#13

#131

#132

#133

#134

#135

#136

#137

#138

#139

#14

#141

#142

#143

#144

#149

#151

#152

#153

#154

#156

#157

#158

#159

#160

#161

#162

#163

#164

#165

#166

#169

#17

#170

#171

#173

#174

#176

#177

#178

#179

#18

#180

#182

#183

#184

#187

#189

#19

#193

#195

#196

#197

#198

#199

#2

#20

#200

#201

#203

#204

#205

#206

#207

#208

#211

#212

#213

#216

#218

#219

#22

#222

#223

#224

#227

#228

#229

#233

#234

#235

#236

#237

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#250

#252

#253

#254

#258

#259

#26

#260

#261

#263

#264

#265

#269

#27

#270

#274

#278

#28

#281

#285

#288

#289

#29

#290

#291

#294

#295

#297

#298

#3

#300

#301

#303

#304

#305

#307

#31

#313

#314

#318

#319

#32

#320

#321

#323

#324

#325

#329

#33

#333

#335

#336

#337

#339

#344

#345

#346

#349

#352

#353

#356

#36

#360

#361

#366

#367

#368

#369

#37

#372

#373

#375

#376

#378

#38

#380

#381

#383

#384

#386

#388

#389

#39

#391

#393

#395

#396

#397

#398

#4

#400

#401

#403

#404

#405

#406

#407

#408

#412

#413

#414

#419

#420

#421

#423

#427

#432

#436

#437

#438

#439

#441

#451

#458

#461

#465

#465

#466

#468

#469

#470

#471

#473

#479

#481

#483

#484

#487

#492

#495

#496

#498

#5

#50

#500

#504

#506

#507

#509

#511

#512

#516

#518

#519

#52

#520

#521

#522

#523

#524

#525

#526

#529

#530

#531

#533

#534

#535

#536

#537

#539

#54

#540

#541

#542

#543

#544

#545

#546

#547

#549

#55

#550

#551

#553

#555

#557

#56

#560

#561

#562

#563

#565

#571

#572

#573

#574

#575

#576

#577

#578

#581

#582

#583

#584

#585

#588

#589

#590

#591

#592

#593

#596

#597

#598

#6

#600

#602

#604

#605

#606

#611

#612

#613

#614

#615

#616

#617

#620

#621

#627

#628

#629

#63

#630

#638

#64

#640

#642

#643

#646

#649

#65

#653

#658

#66

#660

#661

#663

#665

#667

#668

#671

#672

#673

#674

#676

#677

#678

#68

#680

#682

#685

#687

#688

#69

#695

#696

#697

#698

#699

#7

#70

#700

#702

#703

#704

#705

#706

#708

#709

#71

#710

#711

#713

#714

#715

#716

#719

#722

#723

#725

#726

#727

#728

#729

#73

#730

#732

#733

#735

#737

#738

#74

#741

#744

#747

#748

#749

#75

#750

#752

#758

#76

#760

#761

#764

#767

#768

#769

#77

#770

#771

#773

#774

#775

#776

#78

#780

#781

#786

#788

#79

#793

#798

#799

#800

#801

#802

#805

#807

#809

#81

#810

#811

#812

#814

#815

#817

#818

#819

#82

#820

#823

#826

#828

#829

#83

#830

#831

#832

#833

#834

#839

#84

#841

#842

#843

#845

#847

#849

#850

#851

#858

#859

#861

#862

#863

#865

#866

#867

#869

#870

#874

#875

#876

#878

#879

#880

#881

#883

#887

#888

#89

#890

#891

#892

#893

#897

#898

#9

#90

#900

#901

#903

#904

#906

#909

#910

#914

#915

#916

#918

#920

#925

#926

#927

#933

#935

#937

#938

#940

#942

#943

#945

#946

#949

#950

#951

#952

#953

#954

#957

#958

#959

#96

#960

#961

#962

#965

#969

#97

#974

#975

#976

#981

#987

#989

#99

#991

#993

#994

#998

#999

73822b8bfa move ex 3.3 solution outside main chapter rasbt 2024-03-10 07:18:24 -05:00
da33ce8054 remove redundant unsqueeze in mask rasbt 2024-03-09 17:42:25 -06:00
6ba97adaee add PyTorch version rasbt 2024-03-09 17:35:44 -06:00
1d819c3d9c Merge pull request #56 from eltociear/patch-2 Sebastian Raschka 2024-03-09 10:27:11 -06:00
5ca60321c4 add a100 numbers rasbt 2024-03-09 10:20:08 -06:00
47519f4d14 Update compare-bpe-tiktoken.ipynb Ikko Eltociear Ashimine 2024-03-10 01:11:35 +09:00
29ca41799a use need_weights=False rasbt 2024-03-09 10:09:17 -06:00
5643c88db9 add pytorch mha rasbt 2024-03-08 09:30:55 -06:00
3beaea46ce add lowres figs for better navigation rasbt 2024-03-08 07:18:06 -06:00
404f48aa74 automatically run on gpu or cpu rasbt 2024-03-07 20:14:03 -06:00
c5b17c3d67 simplify rasbt 2024-03-07 07:52:24 -06:00
f454944d5d add setup recommendations rasbt 2024-03-07 07:32:30 -06:00
083d11fbd0 Merge pull request #55 from rayedbw/patch-4 Sebastian Raschka 2024-03-07 06:31:01 -06:00
99a5e28def rename q,k,v for consistency with chapter 3 rasbt 2024-03-07 06:30:40 -06:00
496079c61e Update mha-implementations.ipynb Rayed Bin Wahed 2024-03-06 23:03:57 +08:00
b6fe1a37b3 also add simple wrapper rasbt 2024-03-06 08:38:53 -06:00
571377a2d6 update title rasbt 2024-03-06 08:34:04 -06:00
d2835931b7 Merge pull request #54 from rasbt/mha-variants Sebastian Raschka 2024-03-06 08:32:53 -06:00
87fcfd9245 mha variants rasbt 2024-03-06 08:30:32 -06:00
d4754f1bdd change dim=1 to dim=-1 rasbt 2024-03-04 18:54:43 -06:00
b50c42ffbb Merge pull request #52 from rasbt/use-embedding-dropout Sebastian Raschka 2024-03-04 07:07:46 -06:00
e0df4df433 add dropout for embedding layers rasbt 2024-03-04 07:05:06 -06:00
3198363c4f add wording from three to four rasbt 2024-03-04 06:42:58 -06:00
29672da3b0 stride consistency rasbt 2024-03-03 19:37:06 -06:00
742f0a6d29 add missing output in bonus rasbt 2024-03-03 17:29:46 -06:00
f526a8d7fb add requirements file for bonus notebook rasbt 2024-03-02 16:54:24 -06:00
cc2383c4de remove duplicated exercise code rasbt 2024-03-02 16:44:36 -06:00
c071ea73f9 Update DDP-script.py Sebastian Raschka 2024-03-01 18:31:05 -06:00
c9dccb0c40 Merge pull request #33 from rayedbw/patch-1 Sebastian Raschka 2024-02-29 20:00:09 -06:00
267e33cfaf remove redundant import rasbt 2024-02-29 19:59:05 -06:00
d419c02792 Merge pull request #39 from rayedbw/patch-3 Sebastian Raschka 2024-02-29 12:30:50 -06:00
32087331ae Update Dockerfile Rayed Bin Wahed 2024-03-01 02:10:01 +08:00
a94d53a752 Merge pull request #38 from rayedbw/patch-2 Sebastian Raschka 2024-02-29 12:06:05 -06:00
c47e434162 Update README.md Rayed Bin Wahed 2024-03-01 01:56:58 +08:00
7d732a5db0 add readme for devcontainer rasbt 2024-02-29 09:00:06 -06:00
ee24acd481 Merge branch 'main' of https://github.com/rasbt/LLMs-from-scratch rasbt 2024-02-29 08:31:20 -06:00
b827bf4eea remove redundant double-unsequeeze rasbt 2024-02-29 08:31:07 -06:00
3278243dd5 Merge pull request #31 from rayedbw/main Sebastian Raschka 2024-02-29 08:24:29 -06:00
fb770ef97c update docker files and docs rasbt 2024-02-29 08:22:53 -06:00
2fb035435e Update ch04.ipynb Rayed Bin Wahed 2024-02-27 23:05:36 +08:00
d89aaf319d update folder name rasbt 2024-02-27 08:53:04 -06:00
a060f923d3 Merge pull request #32 from rasbt/hparam Sebastian Raschka 2024-02-27 08:52:01 -06:00
87a743076d hparam tuning script rasbt 2024-02-27 08:51:03 -06:00
f6266c3756 improve code comments rasbt 2024-02-27 06:40:35 -06:00
45a10dd823 Add devcontainer starter doc Rayed Bin Wahed 2024-02-27 13:04:06 +08:00
fa7e659eb3 Add devcontainer Rayed Bin Wahed 2024-02-26 20:24:15 +08:00
78ed2e35bc Add requirements.txt to main repo Sebastian Raschka 2024-02-25 13:32:30 -06:00
3debb2f0df Update README.md Sebastian Raschka 2024-02-25 13:31:32 -06:00
3f186ab072 use .shape instead of .size() for consistency rasbt 2024-02-25 08:47:25 -06:00
cdcd73ba7f drop_last=True rasbt 2024-02-25 07:23:38 -06:00
6243726ab3 rename to dataloader v1 rasbt 2024-02-24 07:48:18 -06:00
4e68649f16 comment update rasbt 2024-02-24 06:52:17 -06:00
f057156181 use smaller number of tokens to emphasize next token prediction goal rasbt 2024-02-15 20:09:20 -06:00
557ddfc684 make a new example for shortcut connections rasbt 2024-02-15 19:34:12 -06:00
250e6306e2 use attn_scores from sec 3.4 instead of 3.3 rasbt 2024-02-14 20:23:59 -06:00
231a854ae7 use less ambiguous var name rasbt 2024-02-13 07:05:37 -06:00
320f63829f Merge pull request #29 from Intelligence-Manifesto/patch-5 Sebastian Raschka 2024-02-12 07:34:37 -06:00
6a09e7b03a **step 2** Intelligence-Manifesto 2024-02-12 18:32:28 +08:00
1d6f2c9084 rearrange exercise order rasbt 2024-02-11 14:46:05 -06:00
79d90d8147 Merge pull request #28 from rasbt/ch4-exercise-solutions Sebastian Raschka 2024-02-11 11:52:18 -06:00
fe332006de ch4 exercise solutions rasbt 2024-02-11 11:51:39 -06:00
103f7826ad use same iter to make figs consistent rasbt 2024-02-11 09:12:52 -06:00
352b83d225 make softmax explicit rasbt 2024-02-11 08:42:21 -06:00
7d86023fc4 make softmax explicit rasbt 2024-02-11 08:41:45 -06:00
5840b4b5f8 update name of last section rasbt 2024-02-11 07:35:07 -06:00
e0b6fdbc53 Merge pull request #27 from Intelligence-Manifesto/patch-4 Sebastian Raschka 2024-02-11 07:31:06 -06:00
1278615c25 12 -> 21 Intelligence-Manifesto 2024-02-11 14:17:55 +08:00
baa8617921 variable name fix rasbt 2024-02-10 17:53:54 -06:00
496b52f842 format the other GPT architecture sizes rasbt 2024-02-10 17:47:56 -06:00
40477c55b3 add missing ex sol to table rasbt 2024-02-10 10:13:21 -06:00
10aa2d099d add print statements for illustration purposes rasbt 2024-02-10 10:10:14 -06:00
cc459b6b5a Merge branch 'main' of https://github.com/rasbt/LLMs-from-scratch rasbt 2024-02-08 20:17:01 -06:00
5d1d8ce511 add shape information for clarity rasbt 2024-02-08 20:16:54 -06:00
24d71784e2 Merge pull request #26 from Intelligence-Manifesto/patch-3 Sebastian Raschka 2024-02-08 17:19:29 -06:00
a61583e31e if -> in Intelligence-Manifesto 2024-02-09 04:18:07 +08:00
55ced1d95e Update README.md Sebastian Raschka 2024-02-06 07:38:52 -06:00
3a5fc79b38 add and update readme files rasbt 2024-02-05 06:51:58 -06:00
2b38b63a7a move overview up rasbt 2024-02-04 15:57:03 -06:00
bb50de7210 adjust figure width rasbt 2024-02-04 10:12:11 -06:00
1653f6953a adjust figure width rasbt 2024-02-04 10:09:36 -06:00
f2fe1d0d85 Merge pull request #24 from rasbt/ch04 Sebastian Raschka 2024-02-04 10:06:56 -06:00
b86cc682f0 update links in readme rasbt 2024-02-04 10:05:55 -06:00
ec312e581b add chapter 4 code rasbt 2024-02-04 10:02:05 -06:00
16b30ccd5c set stride equal to max len rasbt 2024-02-03 08:50:56 -06:00
b9974da94d Merge pull request #22 from Intelligence-Manifesto/patch-2 Sebastian Raschka 2024-02-03 08:46:40 -06:00
78ab68b4d3 missing single quotes Intelligence-Manifesto 2024-02-03 03:41:31 +08:00
d261abce4c add forward pass rasbt 2024-01-31 08:00:19 -06:00
fcb13fd636 add code backbone ch04 rasbt 2024-01-29 08:13:52 -06:00
3002777dd2 Update README.md Sebastian Raschka 2024-01-28 18:41:02 -06:00
c4803bb91a add libmamba rasbt 2024-01-28 12:47:29 -06:00
22442b7012 use torch.softmax instead of F.softmax rasbt 2024-01-28 12:25:49 -06:00
eb142491b2 Merge pull request #20 from eltociear/patch-1 Sebastian Raschka 2024-01-28 09:07:49 -06:00
3a3a4ac1f1 Update bpe_openai_gpt2.py Ikko Eltociear Ashimine 2024-01-28 23:26:23 +09:00
f6e80a376d Fix link Sebastian Raschka 2024-01-26 20:30:20 -06:00
04c113c54f Merge pull request #19 from Intelligence-Manifesto/patch-1 Sebastian Raschka 2024-01-25 17:41:49 -06:00
004d6a2af9 a repetition of the word "by" Intelligence-Manifesto 2024-01-26 00:43:57 +08:00
0eab244f00 add full toc rasbt 2024-01-24 07:09:35 -06:00
f27c9e6135 rename create_dataloader to create_dataloader_v1 rasbt 2024-01-24 07:02:05 -06:00
f6896d17ef image resizing rasbt 2024-01-21 20:25:21 -06:00
4c45c37c51 Update README.md Sebastian Raschka 2024-01-21 20:17:32 -06:00

... 15 16 17 18 19 ...