Factsheet
Numba
python - How do I use numba on a member function of a class? - Stack Overflow
Is Numba useful for Python game dev?
Numba: A High Performance Python Compiler
Videos
» pip install numba
Wow. Just wow. Was doing something math-heavy, even very recursion-heavy which is why I wasn't using numpy, and it took a few minutes to get the results. I'd tried numba once before with something and saw some very minute improvements, but figured I'd try it again. I've been doing number crunching in Python for years, and never really thought it'd make a difference.
boy was I wrong. tqdm says it took 2:48 to do the pure python. 0:05 to do with numba (I suppose including compile). I am shooketh. Entirely and completely, to my core. Even if I parallelized it completely which was my first thought, at best I'd get to 21 seconds. I can't believe it. I am sure if there's any response to this it'll be an "I told you so" but really, wow.
Numba.
Edit: because the code was pretty basic, I re-wrote the math part in C. tqdm says I'm at 1 second now. It was 3 seconds ish without the compile time (recall 5 seconds with). ~247k it/s vs ~151k it/s. Still a lot easier with the python and the jit, but the C module wasn't as painless as I thought it'd be.
I was in a very similar situation and I found a way to use a Numba-JITed function inside of a class.
The trick is to use a static method, since this kind of methods are not called prepending the object instance to the argument list. The downside of not having access to self is that you cannot use variables defined outside of the method. So you have to pass them to the static method from a calling method that has access to self. In my case I did not need to define a wrapper method. I just had to split the method I wanted to JIT compile into two methods.
In the case of your example, the solution would be:
from numba import jit
class MyClass:
def __init__(self):
self.k = 1
def calculation(self):
k = self.k
return self.complicated([1,2,3],k)
@staticmethod
@jit(nopython=True)
def complicated(x,k):
for a in x:
b = a**2 .+ a**3 .+ k
You have several options:
Use a jitclass (http://numba.pydata.org/numba-doc/0.30.1/user/jitclass.html) to "numba-ize" the whole thing.
Or make the member function a wrapper and pass the member variables through:
import numba as nb
@nb.jit
def _complicated(x, k):
for a in x:
b = a**2.+a**3.+k
class myClass(object):
def __init__(self):
self.k = 1
def complicated(self,x):
_complicated(x, self.k)
I am not sure how it will behave with PyGame but I intend to use it in combination with some lower level frameworks like OpenGL or SDL and build a small proof of concept game engine with it. What do you think?
In normal loops, Numba seems to do really great in improving loop speeds. Combining mathematical calculations in one loop and having another loop for drawing rendering should give big performance gains, especially if we are adding batching in equation.
I modified your code a bit and was able to get the function to run. I removed the Particle class and changed all the list instances to numpy arrays.
This is the output for the Zavg, Zrelm and counter:
Zavg: [0.07047501 0.06735052 0.06728123 ... 0.39516435 0.3947497 0.39433495]
Zrelm: [-0.04179043 -0.04461464 -0.0394889 ... -0.11080628 -0.11087257
-0.11093883]
Counter: 538
Here is the modified code:
import numpy as np
import matplotlib.pyplot as plt
import random
from math import *
from random import *
from numba import jit
"Dynamics"
@jit(nopython=True)
def f(SP, Zavgi, Zreli, alf, dt, n):
"Time"
counter = 0;
sum1, sum2 = 0, 0;
Zavg = np.array([Zavgi]);
Zrelm = np.array([Zreli]);
T_plot = [0];
for i in range(1, iter + 1):
t = i * dt;
T_plot.append(t)
Z = [];
Up = [];
Down = [];
c, s = cos(t), sin(t);
c1, s1 = cos(t - dt), sin(t - dt);
for j in range(n - 1):
collchk = ((c * (SP[j][0]) + s * (SP[j][1])) - (c * (SP[j + 1][0]) + s * (SP[j + 1][1]))) * (
c1 * (SP[j][0]) + s1 * (SP[j][1]) - (c1 * (SP[j + 1][0]) + s1 * (SP[j + 1][1])));
prel = ((c * (SP[j][1]) - s * (SP[j][0])) - (c * (SP[j + 1][1]) - s * (SP[j + 1][0]))) / 2;
rcoeff = 1 / (1 + (prel * alf) ** 2);
rand_value = random();
if collchk < 0:
SP[j], SP[j + 1] = SP[j + 1], SP[j];
if rcoeff > rand_value:
counter = counter + 1
SP[j][2], SP[j + 1][2] = SP[j + 1][2], SP[j][2];
if SP[j][2] == 1:
Up.append(c * (SP[j][0]) + s * (SP[j][1]))
else:
Down.append(c * (SP[j][0]) + s * (SP[j][1]))
Z.append(c * (SP[j][0]) + s * (SP[j][1]))
Zrel = np.sum(np.array(Up)) / len(Up) - np.sum(np.array(Down)) / len(Down);
Zrelm = np.append(Zrelm, Zrel)
Zm = np.sum(np.array(Z)) / len(Z)
Zavg = np.append(Zavg, Zm)
return Zavg, Zrelm, counter, T_plot
if __name__ == '__main__':
n = 1000
mu = np.random.uniform(0, 1, n)
r = [sqrt(-2 * log(1 - i)) for i in mu]
eta = np.random.uniform(0, 1, n)
theta = 2 * pi * eta;
cuz = [cos(i) for i in theta]
suz = [sin(i) for i in theta]
Zinitial = [a * b for a, b in zip(r, cuz)];
Pinitial = [a * b for a, b in zip(r, suz)];
iter = 10 ** (5);
dt = 1 / (2 * n);
alf = sqrt(n);
SP = np.array(sorted(np.array([ np.array([i,j,choice([0,1])]) for i, j in zip(Zinitial, Pinitial)]),
key=lambda x: x[0]))
Upi = [];
Downi = [];
count_plot = [];
for j in range(len(SP)):
if SP[j][2] == 1:
Upi.append(SP[j][0])
else:
Downi.append(SP[j][0])
Zavgi = sum(Zinitial) / len(Zinitial)
Zreli = sum(Upi) / len(Upi) - sum(Downi) / len(Downi)
Zavg, Zrelm, counter, T_plot = f(SP, Zavgi, Zreli, alf, dt, n)
print(Zavg, Zrelm, counter)
plt.plot(T_plot, Zrelm)
plt.show()
This is how the plot looks like:

The error appears because Numba try to access to global variable which the type is not known at compile time. Indeed, SP is a pure-Python list called reflected list which can contain items of different types. Numba does not support such kind of list anymore. Instead, Numba supports typed lists which are compatible with reflected lists. This means you need to build a new typed list (with a well-defined type) and copy the reflected list items to the typed list. This process can be quite expensive compared to the overall computation. Thus, this is often better not to use lists when arrays can be used instead (typically when you cannot know the size of the final list): Numpy array are significantly faster, more compact and functions using them can be compiled faster.
Additionally, Numba has no idea of was is the type Particle. Numba only supports Numpy built-in types by default. There is an experimental support for jitted classes but I advise you to work with basic array since it is generally faster and also more flexible as you can use Numpy vectorized functions on the target arrays as opposed to an object-based array (which are AFAIK inefficiently stored in Numpy arrays and slow).
Moreover, you should really avoid using global variables, especially the ones that are mutated. Global variables are slower to access in CPython and are generally seen as a bad software engineering practice. For Numba, they are considered as compile time constant so it can cause some surprising results if the variables are mutated at runtime.
