mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00

Files

Sebastian Raschka f63f04d8d5 Fix BPE bonus materials (#561 )

* Fix BPE bonus materials

* fix bpe implementation

* update

* Add 'Hello, world. Is this-- a test?' test case

* update link to test file

* update path handling

* update path handling

* fix pytest paths

2025-03-08 17:21:30 -06:00

tests

Fix BPE bonus materials (#561 )

2025-03-08 17:21:30 -06:00

bpe-from-scratch.ipynb

Fix BPE bonus materials (#561 )

2025-03-08 17:21:30 -06:00

README.md

Bonus material: extending tokenizers (#496 )

2025-01-22 09:26:54 -06:00

README.md

Byte Pair Encoding (BPE) Tokenizer From Scratch

bpe-from-scratch.ipynb contains optional (bonus) code that explains and shows how the BPE tokenizer works under the hood.