‹‹‹ Blog Overview

Compiling and Testing NUMBA from Source
On Ubuntu 22.04 LTS / Linux Mint 21

My recent journey into Numba has made me explore rarely used parts and edge-cases of the library. It inevitably made me discover bugs and inconsistencies. To the numba developers' credit, they are reacting impressively fast and offer bug-fixes for you to test in dedicated branches. It is therefore absolutely worth supporting those efforts by testing their fixes against your use-cases. This raises a question: How does one test these? Or, in other words, how does one build numba from source?

  • CUDA
  • JIT
  • Python
  • contributing
  • numba


I am working with Linux Mint 21 which can be considered near-identical to Ubuntu 22.04 LTS for the purpose of this discussion. The overall procedure however also works on at least Linux Mint 20 a.k.a Ubuntu 20.04 LTS except for minor differences in package versions.

I need Python 3 to be present plus a couple of related infrastructure packages:

sudo apt install python3.10 python3.10-dev python3.10-venv
Source code 1

Furthermore, Ubuntu comes with a handy shortcut for installing the most basic relevant development tools:

sudo apt install build-essential
Source code 2

It is highly recommended to perform the following steps within a Python virtual environment of some sort. I will go with one generated by venv. Let's create and activate it:

python3 -m venv env
source env/bin/activate
Source code 3

Once it is created, it is always a good idea to update pip and setuptools. I usually also install IDE-related tools such as the Python language server:

pip install -vU pip setuptools
pip install -v "python-lsp-server[all]"
Source code 4

Building "llvmlite"

Before I can build numba from source, I need to build llvmlite from source first. Technically, this step is only relevant when working with the latest numba code from its git repository since it usually depends on the latest code from the llvmlite git repository. For older versions of numba, a simple pip install llvmlite=={required_version} might be sufficient. The required version can be deduced from entries within numbas setup.py file.

The build procedure is well documented. Note that officially, an Anaconda- or conda-forge-based build environment is recommended. However, at least on Ubuntu, building llvmlite works perfectly fine within a "regular" virtual environment, i.e. without conda and conda packages, if done right.

llvmlite requires LLVM 11. However, Ubuntu 22.04 ships with LLVM 13. Fortunately, earlier versions of LLVM are available as additional packages, so I can quickly install LLVM 11 next to LLVM 13:

sudo apt install llvm-11-dev
Source code 5

Now it is time to obtain llvmlite's source code:

git clone https://github.com/numba/llvmlite.git
cd llvmlite
Source code 6

Since we are running with a version of LLVM that is not the standard on the operating system underneath, we need to tell llvmlite which version to use - or, where to find the right llvm-config binary. This can be done via the LLVM_CONFIG environment variable. For LLVM 11 on Ubuntu 22.04, the correct path is /usr/bin/llvm-config-11. With this in mind, I can proceed to build llvmlite:

LLVM_CONFIG=/usr/bin/llvm-config-11 python setup.py build
Source code 7

It is a good idea to test the result:

python -m llvmlite.tests
Source code 8

If the tests succeed, I can install llvmlite:

LLVM_CONFIG=/usr/bin/llvm-config-11 python setup.py install
Source code 9

I can now leave llvmlite's folder:

cd ..
Source code 10

Building "numba"

Next up is actually building numba. The procedure is explained in detail within the numba documentation.

numba can compile code for CUDA targets. I usually have a CUDA-capable card and the CUDA-toolkit as well as the proprietary Nvidia driver present. The procedure may differ without those.

numba has a few fundamental requirements which can quickly be installed via pip:

pip install -v numpy scipy jinja2 cffi
Source code 11

Let's get the numba source code next:

git clone https://github.com/numba/numba.git
cd numba
Source code 12

This would be the right time to switch to another branch or pull any changes offered by the numba developers for you to test.

numba can now be built by running:

python setup.py build_ext --inplace
Source code 13

As before, it is a good idea to test the result. Running the entire test suite will take a lot of time, easily multiple hours. The second -m flag enables running multiple tests in parallel which can provide a decent speedup on high-core-count systems:

python -m numba.runtests -m
Source code 14

Last but not least, numba can be installed:

python setup.py install
Source code 15


cd ..
Source code 16

You can now test your use-case against the custom-built version of numba.

‹‹‹ Blog Overview