books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

Improve rope settings for llama3 (#380)

Browse Source
This commit is contained in:
Sebastian Raschka
2024-10-03 08:29:54 -05:00
committed by GitHub
parent 278a50a348
commit b993c2b25b
3 changed files with 57 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.gitignore vendored
View File

@@ -35,8 +35,9 @@ ch05/01_main-chapter-code/model.pth
ch05/01_main-chapter-code/model_and_optimizer.pth
ch05/03_bonus_pretraining_on_gutenberg/model_checkpoints
ch05/06_user_interface/gpt2
ch05/07_gpt_to_llama/models--meta-llama--Llama-2-7b
ch05/07_gpt_to_llama/models--meta-llama--Llama-2-7b-chat
ch05/07_gpt_to_llama/Llama-2-7b
ch05/07_gpt_to_llama/Llama-2-7b-chat
ch05/07_gpt_to_llama/.cache
ch06/01_main-chapter-code/gpt2
ch06/02_bonus_additional-experiments/gpt2