books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

Ch05 supplementary code (#81)

Browse Source
This commit is contained in:
Sebastian Raschka
2024-03-19 09:26:26 -05:00
committed by GitHub
parent 861a2788f3
commit a2cd8436cb
34 changed files with 796 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
ch05/03_bonus_pretraining_on_gutenberg/prepare_dataset.py
View File

@@ -1,4 +1,8 @@
# -*- coding: utf-8 -*-
# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
# Source for "Build a Large Language Model From Scratch"
# - https://www.manning.com/books/build-a-large-language-model-from-scratch
# Code: https://github.com/rasbt/LLMs-from-scratch
"""
Script that processes the Project Gutenberg files into fewer larger files.
"""

6
ch05/03_bonus_pretraining_on_gutenberg/pretraining_simple.py
View File

@@ -1,4 +1,8 @@
# -*- coding: utf-8 -*-
# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
# Source for "Build a Large Language Model From Scratch"
# - https://www.manning.com/books/build-a-large-language-model-from-scratch
# Code: https://github.com/rasbt/LLMs-from-scratch
"""
Script for pretraining a small GPT-2 124M parameter model
on books from Project Gutenberg.

5
ch05/03_bonus_pretraining_on_gutenberg/previous_chapters.py
View File

@@ -1,3 +1,8 @@
# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
# Source for "Build a Large Language Model From Scratch"
# - https://www.manning.com/books/build-a-large-language-model-from-scratch
# Code: https://github.com/rasbt/LLMs-from-scratch
# This file collects all the relevant code that we covered thus far
# throughout Chapters 2-4.
# This file can be run as a standalone script.