feat: Add pixi environment (#534)

* feat: Add pixi environment

* Add pixi manifest pixi.toml for Linux x86, macOS arm64, Windows 64.

* ci: Update CI workflow and unify to one

* Enable workflow dispatch.
* Add concurrency limits.
* Use pixi for CI workflow.
* Unify to a single workflow for all OS tested

* feat: Add pixi lock file

* Ensure tensorflow-cpu installed on Windows

* fix package check

* fix package check

* simplification plus uv and pip runners

* some fixes to pixi and pip

* create pixi.lock

* fix pixi.lock issue

* another attempt trying to fix get_packages

* another attempt trying to fix get_packages

* clean up python_environment_check.py

* updated runner and docs

* use bash

* proper env activiation

* proper env activiation

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
This commit is contained in:
Matthew Feickert
2025-02-17 10:33:53 -07:00
committed by GitHub
parent 16738b61fd
commit a8b8eb4731
14 changed files with 13169 additions and 222 deletions

View File

@@ -27,9 +27,11 @@ This section guides you through the Python setup and package installation proced
> [!NOTE]
> There are alternative ways to install Python and use `uv`. For example, you can install Python directly via `uv` and use `uv add` instead of `uv pip install` for even faster package management.
>
> If you prefer the native `uv` commands, refer to the [./native-uv.md tutorial](./native-uv.md). I also recommend checking the official [`uv` documentation](https://docs.astral.sh/uv/).
> If you are a macOS or Linux user and prefer the native `uv` commands, refer to the [./native-uv.md tutorial](./native-uv.md). I also recommend checking the official [`uv` documentation](https://docs.astral.sh/uv/).
>
> While `uv add` offers additional speed advantages, I think that `uv pip` is slightly more user-friendly, making it a good starting point for beginners. However, if you're new to Python package management, the native `uv` interface is also a great opportunity to learn it from the start. It's also how I use `uv` now, but I realize it the barrier to entry is a bit higher if you are coming from `pip` and `conda`.
> The `uv add` syntax also applies to Windows users. However, I found that some dependencies in the `pyproject.toml` cause problems on Windows. So, for Windows users, I recommend `pix` instead, which has a similar `pixi add` workflow like `uv add`. For more information, see the [./native-pixi.md tutorial](./native-pixi.md).
>
> While `uv add` and `pixi add` offer additional speed advantages, I think that `uv pip` is slightly more user-friendly, making it a good starting point for beginners. However, if you're new to Python package management, the native `uv` interface is also a great opportunity to learn it from the start. It's also how I use `uv` now, but I realize it the barrier to entry is a bit higher if you are coming from `pip` and `conda`.
@@ -153,9 +155,13 @@ uv pip install -U -r https://raw.githubusercontent.com/rasbt/LLMs-from-scratch/r
<img src="https://sebastianraschka.com/images/LLMs-from-scratch-images/setup/uv-setup/uv-install.png" width="700" height="auto" alt="Uv install">
&nbsp;
> [!NOTE]
> If you have problems with the following commands above due to certain dependencies (for example, if you are using Windows), you can always fall back to using regular pip:
> `pip install -r requirements.txt`
> or
> `pip install -U -r https://raw.githubusercontent.com/rasbt/LLMs-from-scratch/refs/heads/main/requirements.txt`
<br>

View File

@@ -0,0 +1,106 @@
# Native pixi Python and package management
This tutorial is an alternative to the [`./native-uv.md`](native-uv.md) document for those who prefer `pixi`'s native commands over traditional environment and package managers like `conda` and `pip`.
Note that pixi uses `uv add` under the hood, as described in [`./native-uv.md`](native-uv.md).
Pixi and uv are both modern package and environment management tools for Python, but pixi is a polyglot package manager designed for managing not just Python but also other languages (similar to conda), while uv is a Python-specific tool optimized for ultra-fast dependency resolution and package installation.
Someone might choose pixi over uv if they need a polyglot package manager that supports multiple languages (not just Python) or prefer a declarative environment management approach similar to conda. For more information, please visit the official [pixi documentation](https://pixi.sh/latest/).
In this tutorial, I am using a computer running macOS, but this workflow is similar for Linux machines and may work for other operating systems as well.
&nbsp;
## 1. Install pixi
Pixi can be installed as follows, depending on your operating system.
<br>
**macOS and Linux**
```bash
curl -fsSL https://pixi.sh/install.sh | sh
```
or
```bash
wget -qO- https://pixi.sh/install.sh | sh
```
<br>
**Windows**
```powershell
powershell -ExecutionPolicy ByPass -c "irm -useb https://pixi.sh/install.ps1 | iex"
```
> [!NOTE]
> For more installation options, please refer to the official [pixi documentation](https://pixi.sh/latest/).
&nbsp;
## 1. Install Python
You can install Python using pixi:
```bash
pixi add python=3.10
```
> [!NOTE]
> I recommend installing a Python version that is at least 2 versions older than the most recent release to ensure PyTorch compatibility. For example, if the most recent version is Python 3.13, I recommend installing version 3.10 or 3.11. You can find out the most recent Python version by visiting [python.org](https://www.python.org).
&nbsp;
## 3. Install Python packages and dependencies
To install all required packages from a `pixi.toml` file (such as the one located at the top level of this GitHub repository), run the following command, assuming the file is in the same directory as your terminal session:
```bash
pixi install
```
> [!NOTE]
> If you encounter issues with dependencies (for example, if you are using Windows), you can always fall back to pip: `pixi run pip install -U -r requirements.txt`
By default, `pixi install` will create a separate virtual environment specific to the project.
You can install new packages that are not specified in `pixi.toml` via `pixi add`, for example:
```bash
pixi add packaging
```
And you can remove packages via `pixi remove`, for example,
```bash
pixi remove packaging
```
&nbsp;
## 4. Run Python code
Your environment should now be ready to run the code in the repository.
Optionally, you can run an environment check by executing the `python_environment_check.py` script in this repository:
```bash
pixi run python setup/02_installing-python-libraries/python_environment_check.py
```
<br>
**Launching JupyterLab**
You can launch a JupyterLab instance via:
```bash
pixi run jupyter lab
```
---
Any questions? Please feel free to reach out in the [Discussion Forum](https://github.com/rasbt/LLMs-from-scratch/discussions).

View File

@@ -116,28 +116,6 @@ uv run python setup/02_installing-python-libraries/python_environment_check.py
<img src="https://sebastianraschka.com/images/LLMs-from-scratch-images/setup/uv-setup/uv-run-check.png?1" width="700" height="auto" alt="Uv install">
Or, if you don't want to type `uv run python` ever time you execute code, manually activate the virtual environment first.
On macOS/Linux:
```bash
source .venv/bin/activate
```
On Windows (PowerShell):
```bash
.venv\Scripts\activate
```
Then, run:
```bash
python setup/02_installing-python-libraries/python_environment_check.py
```
<br>
**Launching JupyterLab**
@@ -150,19 +128,7 @@ uv run jupyter lab
**Skipping the `uv run` command**
If you find typing `uv run` cumbersome and want to run scripts via
```bash
python script.py
```
and launch JupyterLab via
```bash
juputer lab
```
instead, you can activated the environment manually.
If you find typing `uv run` cumbersome, you can manually activate the virtual environment as described below.
On macOS/Linux:
@@ -176,6 +142,20 @@ On Windows (PowerShell):
.venv\Scripts\activate
```
Then, you can run scripts via
```bash
python script.py
```
and launch JupyterLab via
```bash
juputer lab
```
&nbsp;