# Large Language Models from Scratch

Details will follow ...

## Table of Contents

| Chapter                                   | Main code                                                    | Code + supplementary         |
| ----------------------------------------- | ------------------------------------------------------------ | ---------------------------- |
| Ch 1: Understanding Large Language Models | No code                                                      | No code                      |
| Ch 2: Working with Text Data              | [ch02.ipynb](ch02/01_main-chapter-code/ch02.ipynb)<br />[dataloader.ipynb](ch02/01_main-chapter-code/dataloader.ipynb) | [./ch02](./ch02)             |
| Ch 3: Understanding Attention Mechanisms  | [ch03.ipynb](ch03/01_main-chapter-code/ch03.ipynb)<br />[multihead-attention.ipynb](ch03/01_main-chapter-code/multihead-attention.ipynb) | [./ch03](./ch03)             |
| ...                                       | ...                                                          | ...                          |
| Appendix A: Introduction to PyTorch       | [code-part1.ipynb](03_main-chapter-code/01_main-chapter-code/code-part1.ipynb)<br />[code-part2.ipynb](03_main-chapter-code/01_main-chapter-code/code-part2.ipynb)<br />[DDP-script.py](03_main-chapter-code/01_main-chapter-code/DDP-script.py) | [./appendix-A](./appendix-A) |