books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update Qwen3 tokenizer test (#727)

Browse Source 
* Update Qwen3 tokenizer test

* add tokenizers to dev dependencies

* add tokenizers to dev dependencies
This commit is contained in:
Sebastian Raschka
2025-07-08 06:59:46 -05:00
committed by GitHub
parent 83c76891fc
commit 9cf64170ed
3 changed files with 20 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
pyproject.toml
View File

@@ -29,6 +29,7 @@ dev = [
"build>=1.2.2.post1",
"llms-from-scratch",
"twine>=6.1.0",
"tokenizers>=0.21.1",
]
[tool.ruff]