Commit Graph

4 Commits

Author SHA1 Message Date
Maxwell De Jong
e0dbec3331 Fix encoding of multiple preceding spaces in BPE tokenizer. (#945)
* Fix encoding of multiple preceding spaces in BPE tokenizer.

* Add test

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
2026-01-10 10:27:23 -06:00
Sebastian Raschka
fecfdd16ff Add simpler BPE, and make previous BPE better (#870)
* Add simpler BPE, and make previous BPE better

* update

* Update README.md
2025-10-08 22:22:34 -05:00
rasbt
1412b139f2 main push to sync github ruleset 2025-09-14 11:59:52 -05:00
Matthew Hernandez
6f12edb0cc Fix issue: 731 by resolving semantic error (#738)
* fix issue 731

* update test path

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-07-10 16:30:15 -05:00