Readability and code quality improvements (#959)

* Consistent dataset naming

* consistent section headers

ch07/01_main-chapter-code/ch07.ipynb

@@ -89,6 +89,7 @@
     "id": "8bbc68e9-75b3-41f1-ac2c-e071c3cd0813"
    },
    "source": [
+    " \n",
     "## 7.1 Introduction to instruction finetuning"
    ]
   },
@@ -133,6 +134,7 @@
     "id": "5384f0cf-ef3c-4436-a5fa-59bd25649f86"
    },
    "source": [
+    " \n",
     "## 7.2 Preparing a dataset for supervised instruction finetuning"
    ]
   },
@@ -499,6 +501,7 @@
     "id": "fcaaf606-f913-4445-8301-632ae10d387d"
    },
    "source": [
+    " \n",
     "## 7.3 Organizing data into training batches"
    ]
   },
@@ -1492,6 +1495,7 @@
     "id": "d6aad445-8f19-4238-b9bf-db80767fb91a"
    },
    "source": [
+    " \n",
     "## 7.5 Loading a pretrained LLM"
    ]
   },
@@ -1724,6 +1728,7 @@
     "id": "70d27b9d-a942-4cf5-b797-848c5f01e723"
    },
    "source": [
+    " \n",
     "## 7.6 Finetuning the LLM on instruction data"
    ]
   },
@@ -1995,6 +2000,7 @@
     "id": "87b79a47-13f9-4d1f-87b1-3339bafaf2a3"
    },
    "source": [
+    " \n",
     "## 7.7 Extracting and saving responses"
    ]
   },
@@ -2251,6 +2257,7 @@
     "id": "obgoGI89dgPm"
    },
    "source": [
+    " \n",
     "## 7.8 Evaluating the finetuned LLM"
    ]
   },
@@ -2847,6 +2854,7 @@
     "id": "tIbNMluCDjVM"
    },
    "source": [
+    " \n",
     "### 7.9.1 What's next\n",
     "\n",
     "- This marks the final chapter of this book\n",
@@ -2857,12 +2865,26 @@
     "- An optional step that is sometimes followed after instruction finetuning, as described in this chapter, is preference finetuning\n",
     "- Preference finetuning process can be particularly useful for customizing a model to better align with specific user preferences; see the [../04_preference-tuning-with-dpo](../04_preference-tuning-with-dpo) folder if you are interested in this\n",
     "\n",
-    "- This GitHub repository also contains a large selection of additional bonus material you may enjoy; for more information, please see the [Bonus Material](https://github.com/rasbt/LLMs-from-scratch?tab=readme-ov-file#bonus-material) section on this repository's README page\n",
-    "\n",
+    "- This GitHub repository also contains a large selection of additional bonus material you may enjoy; for more information, please see the [Bonus Material](https://github.com/rasbt/LLMs-from-scratch?tab=readme-ov-file#bonus-material) section on this repository's README page"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0e2b7bc2-2e8d-483f-a8f5-e2aa093db189",
+   "metadata": {},
+   "source": [
+    " \n",
     "### 7.9.2 Staying up to date in a fast-moving field\n",
     "\n",
-    "- No code in this section\n",
-    "\n",
+    "- No code in this section"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e3d8327d-afb5-4d24-88af-e253889251cf",
+   "metadata": {},
+   "source": [
+    " \n",
     "### 7.9.3 Final words\n",
     "\n",
     "- I hope you enjoyed this journey of implementing an LLM from the ground up and coding the pretraining and finetuning functions\n",

ch07/02_dataset-utilities/create-passive-voice-entries.ipynb

@@ -88,7 +88,8 @@
    "id": "8bcdcb34-ac75-4f4f-9505-3ce0666c42d5",
    "metadata": {},
    "source": [
-    "## Test OpenAI API"
+    " \n",
+    "## 1. Test OpenAI API"
    ]
   },
   {
@@ -177,7 +178,8 @@
    "id": "162a4739-6f03-4092-a5c2-f57a0b6a4c4d",
    "metadata": {},
    "source": [
-    "## Create JSON Entries"
+    " \n",
+    "## 2. Create JSON Entries"
    ]
   },
   {
@@ -418,7 +420,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.13.5"
   }
  },
  "nbformat": 4,

ch07/04_preference-tuning-with-dpo/dpo-from-scratch.ipynb

@@ -1303,7 +1303,7 @@
    },
    "source": [
     " \n",
-    "## 2.4) Creating training, validation, and test set data loaders"
+    "## 2.4 Creating training, validation, and test set data loaders"
    ]
   },
   {

ch07/05_dataset-generation/reflection-gpt4.ipynb

@@ -306,7 +306,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "def instr_prompt_no_input(ins, outp):\n",
+    "def build_instruction_reflection_prompt_no_input(ins, outp):\n",
     "\n",
     "    sys_prompt = \"You are a helpful, precise but picky assistant for checking the quality of a given instruction.\"\n",
     "    prompt_template = \"[Instruction]\\n{ins}\\n\\n[The Start of Answer]\\n{outp}\\n\\n[The End of Answer]\\n\\n[System]\\n{criteria}\\n\\n\"\n",
@@ -356,7 +356,7 @@
    "id": "9572a1aa-532a-4a76-9fa3-3b59d996ba13",
    "metadata": {},
    "source": [
-    "- We can refine the instruction as follows, using `instr_prompt_no_input` function defined above:"
+    "- We can refine the instruction as follows, using `build_instruction_reflection_prompt_no_input` function defined above:"
    ]
   },
   {
@@ -405,7 +405,7 @@
    "source": [
     "entry = json_data[2]\n",
     "\n",
-    "system_prompt, prompt = instr_prompt_no_input(ins=entry[\"instruction\"], outp=entry[\"output\"])\n",
+    "system_prompt, prompt = build_instruction_reflection_prompt_no_input(ins=entry[\"instruction\"], outp=entry[\"output\"])\n",
     "output = run_chatgpt(prompt=prompt, client=client, system_prompt=system_prompt)\n",
     "\n",
     "print(output)"
@@ -430,7 +430,7 @@
    "source": [
     "import re\n",
     "\n",
-    "def extract_ins(text, no_input=True):\n",
+    "def extract_instruction_segment(text, no_input=True):\n",
     "    if '[New Instruction]' in text:\n",
     "        pattern = r'(\\[New Instruction\\])(.*?)(\\[End\\]|\\[New Answer\\]|New Answer:)'\n",
     "    else:\n",
@@ -445,7 +445,7 @@
     "    return seg_ins\n",
     "\n",
     "\n",
-    "def extract_oup(text, no_input=True):\n",
+    "def extract_output_segment(text, no_input=True):\n",
     "    if '[New Answer]' in text:\n",
     "        pattern = r'(\\[New Answer\\])(.*?)(\\[End\\]|$)'\n",
     "    else:\n",
@@ -462,8 +462,8 @@
     "def extract_instruction(text):\n",
     "    if text == '':\n",
     "        return []\n",
-    "    seg_ins = extract_ins(text, no_input=True)\n",
-    "    seg_oup = extract_oup(text, no_input=True)\n",
+    "    seg_ins = extract_instruction_segment(text, no_input=True)\n",
+    "    seg_oup = extract_output_segment(text, no_input=True)\n",
     "    return [seg_ins, seg_oup]"
    ]
   },
@@ -561,7 +561,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "def res_gen_prompt_no_input(ins, outp):\n",
+    "def build_response_reflection_prompt_no_input(ins, outp):\n",
     "\n",
     "    sys_prompt = \"You are a helpful, precise but picky assistant for checking the quality of the answer to a given instruction.\"\n",
     "    prompt_template = \"[Instruction]\\n{ins}\\n\\n[The Start of Answer]\\n{outp}\\n\\n[The End of Answer]\\n\\n[System]\\n{criteria}\\n\\n\"\n",
@@ -574,7 +574,7 @@
     "    return sys_prompt, prompt\n",
     "\n",
     "\n",
-    "def res_gen_prompt_input(ins, inp, outp):\n",
+    "def build_response_reflection_prompt_with_input(ins, inp, outp):\n",
     "\n",
     "    sys_prompt = \"You are a helpful and precise assistant for checking the quality of the answer to a given instruction and its input.\"\n",
     "    prompt_template = \"[Instruction]\\n{ins}\\n\\n[The Start of Input]\\n{inp}\\n\\n[The End of Input]\\n\\n[The Start of Answer]\\n{outp}\\n\\n[The End of Answer]\\n\\n[System]\\n{criteria}\\n\\n\"\n",
@@ -626,7 +626,7 @@
    "source": [
     "entry = json_data[2]\n",
     "\n",
-    "system_prompt, prompt = res_gen_prompt_no_input(ins=entry[\"instruction\"], outp=entry[\"output\"])\n",
+    "system_prompt, prompt = build_response_reflection_prompt_no_input(ins=entry[\"instruction\"], outp=entry[\"output\"])\n",
     "output = run_chatgpt(prompt=prompt, client=client, system_prompt=system_prompt)\n",
     "\n",
     "print(output)"
@@ -750,7 +750,7 @@
     "    for entry in tqdm(json_data):\n",
     "        \n",
     "        if not entry[\"input\"]:\n",
-    "            system_prompt, prompt = instr_prompt_no_input(ins=entry[\"instruction\"], outp=entry[\"output\"])\n",
+    "            system_prompt, prompt = build_instruction_reflection_prompt_no_input(ins=entry[\"instruction\"], outp=entry[\"output\"])\n",
     "            output = run_chatgpt(prompt=prompt, client=client, system_prompt=system_prompt)\n",
     "            new_instr, new_outp = extract_instruction(output)\n",
     "            new_entry = {\"instruction\": new_instr, \"input\": \"\", \"output\": new_outp}\n",
@@ -906,7 +906,7 @@
     "    for entry in tqdm(json_data):\n",
     "        \n",
     "        if not entry[\"input\"]:\n",
-    "            system_prompt, prompt = res_gen_prompt_no_input(ins=entry[\"instruction\"], outp=entry[\"output\"])\n",
+    "            system_prompt, prompt = build_response_reflection_prompt_no_input(ins=entry[\"instruction\"], outp=entry[\"output\"])\n",
     "            output = run_chatgpt(prompt=prompt, client=client, system_prompt=system_prompt)\n",
     "            new_response = extract_response(output)\n",
     "\n",
@@ -917,7 +917,7 @@
     "            new_json_data.append(new_entry)\n",
     "\n",
     "        else:\n",
-    "            system_prompt, prompt = res_gen_prompt_input(ins=entry[\"instruction\"], inp=entry[\"input\"], outp=entry[\"output\"])\n",
+    "            system_prompt, prompt = build_response_reflection_prompt_with_input(ins=entry[\"instruction\"], inp=entry[\"input\"], outp=entry[\"output\"])\n",
     "            output = run_chatgpt(prompt=prompt, client=client, system_prompt=system_prompt)\n",
     "            new_response = extract_response(output)\n",
     "\n",