mirror of
https://github.com/rasbt/LLMs-from-scratch.git
synced 2026-04-10 12:33:42 +00:00
* Switch from urllib to requests to improve reliability * Keep ruff linter-specific * update * update * update
Byte Pair Encoding (BPE) Tokenizer From Scratch
- bpe-from-scratch.ipynb contains optional (bonus) code that explains and shows how the BPE tokenizer works under the hood.