books/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-10 12:33:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

updated Dockerfile and Additional Classification Finetuning Experiments (#195)

Browse Source 
* accuracy to .2f

* added curl
This commit is contained in:
Daniel Kleine
2024-06-06 03:17:49 +02:00
committed by GitHub
parent c6efa78325
commit ba1c1e74aa
2 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
ch06/02_bonus_additional-experiments/README.md
View File

@@ -19,7 +19,7 @@ For example,
| 6 | gpt2-medium (355M) | pretrained | last | last_block | longest train ex. (120) | 87.50% | 91.28% | 84.67% | 0.75 min | A100 | | 6 | gpt2-medium (355M) | pretrained | last | last_block | longest train ex. (120) | 87.50% | 91.28% | 84.67% | 0.75 min | A100 |
| 7 | gpt2-large (774M) | pretrained | last | last_block | longest train ex. (120) | 99.52% | 98.66% | 96.67% | 1.50 min | A100 | | 7 | gpt2-large (774M) | pretrained | last | last_block | longest train ex. (120) | 99.52% | 98.66% | 96.67% | 1.50 min | A100 |
| 8 | gpt2-xl (1558M) | pretrained | last | last_block | longest train ex. (120) | 99.81% | 99.33% | 98.33% | 2.83 min | A100 | | 8 | gpt2-xl (1558M) | pretrained | last | last_block | longest train ex. (120) | 99.81% | 99.33% | 98.33% | 2.83 min | A100 |
| 9 | gpt2-small (124M) | random | last | all | longest train ex. (120) | 100% | 96.64% | 93.67% | 0.69 min | A100 | | 9 | gpt2-small (124M) | random | last | all | longest train ex. (120) | 100.00% | 96.64% | 93.67% | 0.69 min | A100 |
| 10 | gpt2-small (124M) | pretrained | last | LoRA | longest train ex. (120) | 100.00% | 97.32% | 96.67% | 0.75 min | A100 | | 10 | gpt2-small (124M) | pretrained | last | LoRA | longest train ex. (120) | 100.00% | 97.32% | 96.67% | 0.75 min | A100 |
| 11 | gpt2-small (124M) | pretrained | last | last_block | context length (1024) | 83.08% | 87.92% | 78.33% | 2.46 min | A100 | | 11 | gpt2-small (124M) | pretrained | last | last_block | context length (1024) | 83.08% | 87.92% | 78.33% | 2.46 min | A100 |
| 12 | gpt2-small (124M) | pretrained | last | last_block | variable: no padding (batch size 1) | 100.00% | 98.66% | 98.00% | 1.75 min | A100 | | 12 | gpt2-small (124M) | pretrained | last | last_block | variable: no padding (batch size 1) | 100.00% | 98.66% | 98.00% | 1.75 min | A100 |

1
setup/03_optional-docker-environment/.devcontainer/Dockerfile
View File

@@ -4,6 +4,7 @@ RUN apt-get update && \
apt-get upgrade -y && \ apt-get upgrade -y && \
apt-get install -y rsync && \ apt-get install -y rsync && \
apt-get install -y git && \ apt-get install -y git && \
apt-get install -y curl && \
rm -rf /var/lib/apt/lists/* rm -rf /var/lib/apt/lists/*
COPY requirements.txt requirements.txt COPY requirements.txt requirements.txt