books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add and link bonus material (#84)

Browse Source
This commit is contained in:
Sebastian Raschka
2024-03-23 07:27:43 -05:00
committed by GitHub
parent 35c6e12730
commit cf39abac04
12 changed files with 110 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
ch05/04_learning_rate_schedulers/README.md Normal file
View File

@@ -0,0 +1,5 @@
# Adding Bells and Whistles to the Training Loop
The main chapter used a relatively simple training function to keep the code readable and fit Chapter 5 within the page limits. Optionally, we can add a linear warm-up, a cosine decay schedule, and gradient clipping to improve the training stability and convergence.
You can find the code for this more sophisticated training function in [Appendix D: Adding Bells and Whistles to the Training Loop](../../appendix-D/01_main-chapter-code/appendix-D.ipynb).