mirror of
https://github.com/rasbt/LLMs-from-scratch.git
synced 2026-04-10 12:33:42 +00:00
feat: Add pixi environment (#534)
* feat: Add pixi environment * Add pixi manifest pixi.toml for Linux x86, macOS arm64, Windows 64. * ci: Update CI workflow and unify to one * Enable workflow dispatch. * Add concurrency limits. * Use pixi for CI workflow. * Unify to a single workflow for all OS tested * feat: Add pixi lock file * Ensure tensorflow-cpu installed on Windows * fix package check * fix package check * simplification plus uv and pip runners * some fixes to pixi and pip * create pixi.lock * fix pixi.lock issue * another attempt trying to fix get_packages * another attempt trying to fix get_packages * clean up python_environment_check.py * updated runner and docs * use bash * proper env activiation * proper env activiation --------- Co-authored-by: rasbt <mail@sebastianraschka.com>
This commit is contained in:
@@ -27,9 +27,11 @@ This section guides you through the Python setup and package installation proced
|
||||
> [!NOTE]
|
||||
> There are alternative ways to install Python and use `uv`. For example, you can install Python directly via `uv` and use `uv add` instead of `uv pip install` for even faster package management.
|
||||
>
|
||||
> If you prefer the native `uv` commands, refer to the [./native-uv.md tutorial](./native-uv.md). I also recommend checking the official [`uv` documentation](https://docs.astral.sh/uv/).
|
||||
> If you are a macOS or Linux user and prefer the native `uv` commands, refer to the [./native-uv.md tutorial](./native-uv.md). I also recommend checking the official [`uv` documentation](https://docs.astral.sh/uv/).
|
||||
>
|
||||
> While `uv add` offers additional speed advantages, I think that `uv pip` is slightly more user-friendly, making it a good starting point for beginners. However, if you're new to Python package management, the native `uv` interface is also a great opportunity to learn it from the start. It's also how I use `uv` now, but I realize it the barrier to entry is a bit higher if you are coming from `pip` and `conda`.
|
||||
> The `uv add` syntax also applies to Windows users. However, I found that some dependencies in the `pyproject.toml` cause problems on Windows. So, for Windows users, I recommend `pix` instead, which has a similar `pixi add` workflow like `uv add`. For more information, see the [./native-pixi.md tutorial](./native-pixi.md).
|
||||
>
|
||||
> While `uv add` and `pixi add` offer additional speed advantages, I think that `uv pip` is slightly more user-friendly, making it a good starting point for beginners. However, if you're new to Python package management, the native `uv` interface is also a great opportunity to learn it from the start. It's also how I use `uv` now, but I realize it the barrier to entry is a bit higher if you are coming from `pip` and `conda`.
|
||||
|
||||
|
||||
|
||||
@@ -153,9 +155,13 @@ uv pip install -U -r https://raw.githubusercontent.com/rasbt/LLMs-from-scratch/r
|
||||
|
||||
<img src="https://sebastianraschka.com/images/LLMs-from-scratch-images/setup/uv-setup/uv-install.png" width="700" height="auto" alt="Uv install">
|
||||
|
||||
|
||||
|
||||
> [!NOTE]
|
||||
> If you have problems with the following commands above due to certain dependencies (for example, if you are using Windows), you can always fall back to using regular pip:
|
||||
> `pip install -r requirements.txt`
|
||||
> or
|
||||
> `pip install -U -r https://raw.githubusercontent.com/rasbt/LLMs-from-scratch/refs/heads/main/requirements.txt`
|
||||
|
||||
<br>
|
||||
|
||||
|
||||
106
setup/01_optional-python-setup-preferences/native-pixi.md
Normal file
106
setup/01_optional-python-setup-preferences/native-pixi.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# Native pixi Python and package management
|
||||
|
||||
This tutorial is an alternative to the [`./native-uv.md`](native-uv.md) document for those who prefer `pixi`'s native commands over traditional environment and package managers like `conda` and `pip`.
|
||||
|
||||
Note that pixi uses `uv add` under the hood, as described in [`./native-uv.md`](native-uv.md).
|
||||
|
||||
Pixi and uv are both modern package and environment management tools for Python, but pixi is a polyglot package manager designed for managing not just Python but also other languages (similar to conda), while uv is a Python-specific tool optimized for ultra-fast dependency resolution and package installation.
|
||||
|
||||
Someone might choose pixi over uv if they need a polyglot package manager that supports multiple languages (not just Python) or prefer a declarative environment management approach similar to conda. For more information, please visit the official [pixi documentation](https://pixi.sh/latest/).
|
||||
|
||||
In this tutorial, I am using a computer running macOS, but this workflow is similar for Linux machines and may work for other operating systems as well.
|
||||
|
||||
|
||||
## 1. Install pixi
|
||||
|
||||
Pixi can be installed as follows, depending on your operating system.
|
||||
|
||||
<br>
|
||||
|
||||
**macOS and Linux**
|
||||
|
||||
```bash
|
||||
curl -fsSL https://pixi.sh/install.sh | sh
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```bash
|
||||
wget -qO- https://pixi.sh/install.sh | sh
|
||||
```
|
||||
|
||||
<br>
|
||||
|
||||
**Windows**
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy ByPass -c "irm -useb https://pixi.sh/install.ps1 | iex"
|
||||
```
|
||||
|
||||
> [!NOTE]
|
||||
> For more installation options, please refer to the official [pixi documentation](https://pixi.sh/latest/).
|
||||
|
||||
|
||||
|
||||
## 1. Install Python
|
||||
|
||||
You can install Python using pixi:
|
||||
|
||||
```bash
|
||||
pixi add python=3.10
|
||||
```
|
||||
|
||||
> [!NOTE]
|
||||
> I recommend installing a Python version that is at least 2 versions older than the most recent release to ensure PyTorch compatibility. For example, if the most recent version is Python 3.13, I recommend installing version 3.10 or 3.11. You can find out the most recent Python version by visiting [python.org](https://www.python.org).
|
||||
|
||||
|
||||
## 3. Install Python packages and dependencies
|
||||
|
||||
To install all required packages from a `pixi.toml` file (such as the one located at the top level of this GitHub repository), run the following command, assuming the file is in the same directory as your terminal session:
|
||||
|
||||
```bash
|
||||
pixi install
|
||||
```
|
||||
|
||||
> [!NOTE]
|
||||
> If you encounter issues with dependencies (for example, if you are using Windows), you can always fall back to pip: `pixi run pip install -U -r requirements.txt`
|
||||
|
||||
By default, `pixi install` will create a separate virtual environment specific to the project.
|
||||
|
||||
You can install new packages that are not specified in `pixi.toml` via `pixi add`, for example:
|
||||
|
||||
```bash
|
||||
pixi add packaging
|
||||
```
|
||||
|
||||
And you can remove packages via `pixi remove`, for example,
|
||||
|
||||
```bash
|
||||
pixi remove packaging
|
||||
```
|
||||
|
||||
|
||||
## 4. Run Python code
|
||||
|
||||
Your environment should now be ready to run the code in the repository.
|
||||
|
||||
Optionally, you can run an environment check by executing the `python_environment_check.py` script in this repository:
|
||||
|
||||
```bash
|
||||
pixi run python setup/02_installing-python-libraries/python_environment_check.py
|
||||
```
|
||||
|
||||
<br>
|
||||
|
||||
**Launching JupyterLab**
|
||||
|
||||
You can launch a JupyterLab instance via:
|
||||
|
||||
```bash
|
||||
pixi run jupyter lab
|
||||
```
|
||||
|
||||
|
||||
---
|
||||
|
||||
Any questions? Please feel free to reach out in the [Discussion Forum](https://github.com/rasbt/LLMs-from-scratch/discussions).
|
||||
@@ -116,28 +116,6 @@ uv run python setup/02_installing-python-libraries/python_environment_check.py
|
||||
<img src="https://sebastianraschka.com/images/LLMs-from-scratch-images/setup/uv-setup/uv-run-check.png?1" width="700" height="auto" alt="Uv install">
|
||||
|
||||
|
||||
|
||||
Or, if you don't want to type `uv run python` ever time you execute code, manually activate the virtual environment first.
|
||||
|
||||
On macOS/Linux:
|
||||
|
||||
```bash
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
On Windows (PowerShell):
|
||||
|
||||
```bash
|
||||
.venv\Scripts\activate
|
||||
```
|
||||
|
||||
Then, run:
|
||||
|
||||
|
||||
```bash
|
||||
python setup/02_installing-python-libraries/python_environment_check.py
|
||||
```
|
||||
|
||||
<br>
|
||||
|
||||
**Launching JupyterLab**
|
||||
@@ -150,19 +128,7 @@ uv run jupyter lab
|
||||
|
||||
**Skipping the `uv run` command**
|
||||
|
||||
If you find typing `uv run` cumbersome and want to run scripts via
|
||||
|
||||
```bash
|
||||
python script.py
|
||||
```
|
||||
|
||||
and launch JupyterLab via
|
||||
|
||||
```bash
|
||||
juputer lab
|
||||
```
|
||||
|
||||
instead, you can activated the environment manually.
|
||||
If you find typing `uv run` cumbersome, you can manually activate the virtual environment as described below.
|
||||
|
||||
On macOS/Linux:
|
||||
|
||||
@@ -176,6 +142,20 @@ On Windows (PowerShell):
|
||||
.venv\Scripts\activate
|
||||
```
|
||||
|
||||
Then, you can run scripts via
|
||||
|
||||
```bash
|
||||
python script.py
|
||||
```
|
||||
|
||||
and launch JupyterLab via
|
||||
|
||||
```bash
|
||||
juputer lab
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user