Like a lot of people, I put up with frustrations because I don’t know there’s an alternative. This happened with my Mamba environment for data science. Then I found Pixi. This new tool solved my headaches. It might solve yours, and you don’t need to be a a data scientist to use it.
What is Pixi?
A package manager for developers of all stripes
Pixi is a package manager that runs on top of the operating system’s package manager, such as apt on Ubuntu-and-Debian-based systems or WinGet on Windows. It’s intended for developers.
If most modern operating systems have a package manager, why would you want to install a different package management system? If you’re a developer, a lot of the packages you depend on, such as libraries, interpreters, debuggers, or compilers, tend to be older than the current versions.
If you want a newer version than what’s in the package repositories, you would previously have had to download it and possibly compile it separately from the main system. This could make it difficult to upgrade in the future, since you need to repeat the process when new versions of your tools come out.
Prefix, the developers of Pixi, have created a video demo to show Pixi in action:
Pixi is project-based. You set up a development environment for the project you’re working on, and you can tear it down once you’re finished. You can create a specific environment with certain versions, and Pixi will freeze this setup in a “lockfile.” You can share your environment with other people and they should be able to build an app or run a script in the same environment you created it in.
Why I ditched Mamba
Updating was too slow
I’d previously used Mamba to set up a development environment for my stats, or data analysis, or data science, or machine learning, or whatever the hot term is now, in Mamba. It was fine for a while, but I noticed some glaring problems.
The biggest issue was how long it took to update my environments. If you use a Linux package manager, they’re usually quick to determine which packages to download. When I wanted to update my environment, it would take minutes, often tens of minutes, to “solve” the updates. I thought that was ridiculous.
I also had to “activate” my stats environment whenever I wanted to use it. Sometimes I would get a flash of inspiration and want to fire up IPython or a Jupyter notebook, but I would get an error message at the terminal because it hadn’t activated it.
While browsing the NumPy environment, I found a solution that I didn’t know I was looking for. Pixi was listed as one of the best ways to install for new users. I decided to try it.
Installing Pixi is simple:
Open a terminal window
Navigate to the installation page
Copy the installation script
Paste it into the terminal
Now you’ll have Pixi installed and ready to create development environments.
Creating an environment
Your projects live in directories now
With Pixi installed, I can create an environment. I’ll demonstrate by creating a generic “scientific python” environment that includes NumPy, SciPy, SymPy, IPython, Jupyter, and matplotlib.
pixi init sci
This command creates a directory (if it doesn’t already exist) and creates a .pixi directory that’s hidden on Linux, as well as the pixi.toml and pixi.lock.
The former lists the installed files and exposed executables, and the lock file defines the installed packages down to the specific versions. This makes it possible to duplicate the exact development environment on another machine.
With the environment created, I can install packages:
pixi add ipython jupyter numpy scipy sympy matplotlib
Fast updates
Pixi updates your environments easily and quickly
After installing Pixi and creating an environment, I could update it much more quickly. These updates took seconds compared to minutes. This might be due to the client being written in Rust, a fast, compiled, memory-safe replacement for C.
The upgrade process is similar to using other Linux package managers such as APT.
To update the package, you run this command from the project directory:
pixi update
To upgrade the avialable packages, type:
pixi upgrade
Multiple projects at once
Switch environments without breaking a sweat
WIth the directory structure, you can easily have different environments with different projects. You can even have different environments in a single project.
Apart from my stats environment, I tried to install SageMath, which is available in the repositories. It uses the same package repository as Mamba, so I know what I’m getting.
pixi create sage
cd sage
pixi add sage
To use a Pixi environment, you can run a shell with the full environment:
pixi shell
When you’re finished, you can exit by typing “exit” or pressing Ctrl + d to get back to your regular shell.
Alternatively, you can run just one command using the “pixi run” command. I can do that for SageMath
pixi run sage
I could still set up a stats/data science global workflow
My favorite tools at my fingertips
In my stats environment, I can run IPython or Jupyter in a similar way to Sage:
pixi run ipython
pixi shell
jupyter notebook
More convenient would be being able to run these without having to switch to another environment. Ordinarily, I would have to use a system package manager to do this. Pixi lets me set up global environments to have at my fingertips. This is handy for someone who works with data analysis.
To set this up, I can create an environment using stats with NumPy, SciPy, SymPy, and my favorite libraries including Seaborn and statsmodels.
I can do that with this command:
pixi global install –environment stats –expose jupyter –expose ipython jupyter numpy pandas matplotlib statsmodels sympy ipython
This is similar to using a standard package manager. The “–expose” sections make external executables able to use the libraries, since they run in isolated environments when installed with Pixi.
I also find the “isympy” script very handy to use with SymPy, so I have that “exposed” as well. I edited the pixi.toml file using this command:
pixi global edit
Under the “exposed = ” section, I added:
isympy = “isympy”
Now I could have the ultimate desk calculator at my fingertips.
Finally, an easy way to manage my development environment
Pixi solved a lot of the problems I had with Mamba. It seems to be yet another thing that I wished I’d found out about sooner.