‹‹‹ Blog Overview

Prerequisites

I am working with Linux Mint 21 which can be considered near-identical to Ubuntu 22.04 LTS for the purpose of this discussion. The overall procedure however also works on at least Linux Mint 20 a.k.a Ubuntu 20.04 LTS except for minor differences in package versions.

I need Python 3 to be present plus a couple of related infrastructure packages:

sudo apt install python3.10 python3.10-dev python3.10-venv

Furthermore, Ubuntu comes with a handy shortcut for installing the most basic relevant development tools:

sudo apt install build-essential

It is highly recommended to perform the following steps within a Python virtual environment of some sort. I will go with one generated by venv. Let's create and activate it:

python3 -m venv env
source env/bin/activate

Once it is created, it is always a good idea to update pip and setuptools. I usually also install IDE-related tools such as the Python language server:

pip install -vU pip setuptools
pip install -v "python-lsp-server[all]"

Building "llvmlite"

Before I can build numba from source, I need to build llvmlite from source first. Technically, this step is only relevant when working with the latest numba code from its git repository since it usually depends on the latest code from the llvmlite git repository. For older versions of numba, a simple pip install llvmlite=={required_version} might be sufficient. The required version can be deduced from entries within numbas setup.py file.

The build procedure is well documented. Note that officially, an Anaconda- or conda-forge-based build environment is recommended. However, at least on Ubuntu, building llvmlite works perfectly fine within a "regular" virtual environment, i.e. without conda and conda packages, if done right.

llvmlite requires LLVM 11. However, Ubuntu 22.04 ships with LLVM 13. Fortunately, earlier versions of LLVM are available as additional packages, so I can quickly install LLVM 11 next to LLVM 13:

sudo apt install llvm-11-dev

Now it is time to obtain llvmlite's source code:

git clone https://github.com/numba/llvmlite.git
cd llvmlite

Since we are running with a version of LLVM that is not the standard on the operating system underneath, we need to tell llvmlite which version to use - or, where to find the right llvm-config binary. This can be done via the LLVM_CONFIG environment variable. For LLVM 11 on Ubuntu 22.04, the correct path is /usr/bin/llvm-config-11. With this in mind, I can proceed to build llvmlite:

LLVM_CONFIG=/usr/bin/llvm-config-11 python setup.py build

It is a good idea to test the result:

python -m llvmlite.tests

If the tests succeed, I can install llvmlite:

LLVM_CONFIG=/usr/bin/llvm-config-11 python setup.py install

I can now leave llvmlite's folder:

cd ..

Building "numba"

Next up is actually building numba. The procedure is explained in detail within the numba documentation.

numba can compile code for CUDA targets. I usually have a CUDA-capable card and the CUDA-toolkit as well as the proprietary Nvidia driver present. The procedure may differ without those.

numba has a few fundamental requirements which can quickly be installed via pip:

pip install -v numpy scipy jinja2 cffi

Let's get the numba source code next:

git clone https://github.com/numba/numba.git
cd numba

This would be the right time to switch to another branch or pull any changes offered by the numba developers for you to test.

numba can now be built by running:

python setup.py build_ext --inplace

As before, it is a good idea to test the result. Running the entire test suite will take a lot of time, easily multiple hours. The second -m flag enables running multiple tests in parallel which can provide a decent speedup on high-core-count systems:

python -m numba.runtests -m

Last but not least, numba can be installed:

python setup.py install

Done.

cd ..

You can now test your use-case against the custom-built version of numba.

‹‹‹ Blog Overview