books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
7bd263144eeb7b220feb013559073055ec6b0139
LLMs-from-scratch/ch02/05_bpe-from-scratch
History
Sebastian Raschka 7bd263144e Switch from urllib to requests to improve reliability (#867) 
* Switch from urllib to requests to improve reliability

* Keep ruff linter-specific

* update

* update

* update
2025-10-07 15:22:59 -05:00
..
bpe-from-scratch.ipynb
Switch from urllib to requests to improve reliability (#867)
2025-10-07 15:22:59 -05:00
README.md
Bonus material: extending tokenizers (#496)
2025-01-22 09:26:54 -06:00
tests.py
main push to sync github ruleset
2025-09-14 11:59:52 -05:00

README.md

Byte Pair Encoding (BPE) Tokenizer From Scratch

  • bpe-from-scratch.ipynb contains optional (bonus) code that explains and shows how the BPE tokenizer works under the hood.
Reference in New Issue View Git Blame Copy Permalink