You can reshape the array to a four dimensional array that reflects the desired block structure, and then sum along both axes within each block. Example:

>>> a = np.arange(24).reshape(4, 6)
>>> a
array([[ 0,  1,  2,  3,  4,  5],
       [ 6,  7,  8,  9, 10, 11],
       [12, 13, 14, 15, 16, 17],
       [18, 19, 20, 21, 22, 23]])
>>> a.reshape(2, 2, 2, 3).sum(3).sum(1)
array([[ 24,  42],
       [ 96, 114]])

If a has the shape m, n, the reshape should have the form

a.reshape(m_bins, m // m_bins, n_bins, n // n_bins)
Answer from Sven Marnach on Stack Overflow
Top answer
1 of 5
14

You can reshape the array to a four dimensional array that reflects the desired block structure, and then sum along both axes within each block. Example:

>>> a = np.arange(24).reshape(4, 6)
>>> a
array([[ 0,  1,  2,  3,  4,  5],
       [ 6,  7,  8,  9, 10, 11],
       [12, 13, 14, 15, 16, 17],
       [18, 19, 20, 21, 22, 23]])
>>> a.reshape(2, 2, 2, 3).sum(3).sum(1)
array([[ 24,  42],
       [ 96, 114]])

If a has the shape m, n, the reshape should have the form

a.reshape(m_bins, m // m_bins, n_bins, n // n_bins)
2 of 5
2

At first I was also going to suggest that you use np.histogram2d rather than reinventing the wheel, but then I realized that it would be overkill to use that and would need some hacking still.

If I understand correctly, you just want to sum over submatrices of your input. That's pretty easy to brute force: going over your output submatrix and summing up each subblock of your input:

import numpy as np

def submatsum(data,n,m):
    # return a matrix of shape (n,m)
    bs = data.shape[0]//n,data.shape[1]//m  # blocksize averaged over
    return np.reshape(np.array([np.sum(data[k1*bs[0]:(k1+1)*bs[0],k2*bs[1]:(k2+1)*bs[1]]) for k1 in range(n) for k2 in range(m)]),(n,m))

# set up dummy data
N,M = 4,6
data_matrix = np.reshape(np.arange(N*M),(N,M))

# set up size of 2x3-reduced matrix, assume congruity
n,m = N//2,M//3
reduced_matrix = submatsum(data_matrix,n,m)

# check output
print(data_matrix)
print(reduced_matrix)

This prints

print(data_matrix)
[[ 0  1  2  3  4  5]
 [ 6  7  8  9 10 11]
 [12 13 14 15 16 17]
 [18 19 20 21 22 23]]

print(reduced_matrix)
[[ 24  42]
 [ 96 114]]

which is indeed the result for summing up submatrices of shape (2,3).

Note that I'm using // for integer division to make sure it's python3-compatible, but in case of python2 you can just use / for division (due to the numbers involved being integers).

🌐
SciPython
scipython.com › blog › binning-a-2d-array-in-numpy
Binning a 2D array in NumPy
August 4, 2016 - In [x]: arr.reshape(2, 2, 3, 2).mean(-1).mean(1) Out[x]: array([[ 3.5, 5.5, 7.5], [ 15.5, 17.5, 19.5]]) This is the $2\times 3$ binned array that we wanted.
Discussions

python - Binning data based on one column in 2D array and estimate mean in each bin using cython - Stack Overflow
In order to optimize the speed of my code which is very vital for the speed of my MCMC, I want to substitute some of the bottlenecks of my python code with cython. Since I am working with a huge two More on stackoverflow.com
🌐 stackoverflow.com
binning - Numpy rebinning a 2D array - Stack Overflow
I am looking for a fast formulation to do a numerical binning of a 2D numpy array. By binning I mean calculate submatrix averages or cumulative values. For ex. x = numpy.arange(16).reshape(4, 4) wo... More on stackoverflow.com
🌐 stackoverflow.com
python - binning data live into a 2D array - Stack Overflow
I am calculating two distances and binning them in intervals of 0.1 in a 2D array. Currently I am doing this. However it takes a lot of time for large number of points import numpy as np from scipy. More on stackoverflow.com
🌐 stackoverflow.com
August 3, 2017
arrays - How do I median bin a 2D image in python? - Stack Overflow
I have a 2D numarray, of size WIDTHxHEIGHT. I would like to bin the array by finding the median of each bin so that the resultant array is WIDTH/binsize x HEIGHT/binsize. Assume that both WIDTH and More on stackoverflow.com
🌐 stackoverflow.com
Top answer
1 of 2
2

It might be surprising, but summing up some values in the matrix isn't an easy task. This and this answers of mine give some insights, so I'm not going to repeat in much detail, but to get the best performance one must utilize the cache and SIMD/pipeline-nature floating operations on modern CPUs in the best possible way.

Numpy gets all of the above right and it is quite hard to beat. It is not impossible, but one must really be very intimate with low level optimization to be successful - and one should not expect much of improvement. Naive implementation won't be able to beat Numpy at all.

Here are my tries using cython and numba, where all of the speed-up comes from parallelization.

Let's start with the baseline-algorithm:

def bin2d(a,K):
    m_bins = a.shape[0]//K
    n_bins = a.shape[1]//K
    return a.reshape(m_bins, K, n_bins, K).sum(3).sum(1)

and measure it's performance:

import numpy as np
N,K=2000,50
a=np.arange(N*N, dtype=np.float64).reshape(N,N)
%timeit bin2d(a,K)
# 2.76 ms ± 107 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Cython:

Here is my implementation with Cython, which uses OpenMP for parallelization. To keep things simple I perform summation in-place, thus passed array will be changed (which is not the case for numpy's version):

%%cython -a  -c=/openmp --link-args=/openmp -c=/arch:AVX512
import numpy as np
import cython
from cython.parallel import prange
cdef extern from *:
    """
    //assumes direct memory and row-major-order (i.e. rows are continuous)
    double calc_bin(double *ptr, int N, int y_offset){
       double *row = ptr;
       for(int y=1;y<N;y++){
           row+=y_offset; //next row
           //there is no dependencies, so the summation can be done in parallel
           for(int x=0;x<N;x++){
               ptr[x]+=row[x];
           }
       }
       double res=0.0;
       for(int x=0;x<N;x++){
           //could be made slightly faster (summation is not in parallel), but is it needed?
           res+=ptr[x];
       }
       return res;
    }
    """
    double calc_bin(double *ptr, int N, int y_offset) nogil


@cython.boundscheck(False)
@cython.wraparound(False)
def cy_bin2d_parallel(double[:, ::1] a, int K):
    cdef int y_offset = a.shape[0]
    cdef int m_bins = a.shape[0]//K
    cdef int n_bins = a.shape[1]//K
    cdef double[:,:] res = np.empty((m_bins, n_bins), dtype=np.float64)
    cdef int i,j,k
    for k in prange(m_bins*n_bins, nogil=True):
        i = k//m_bins
        j = k%m_bins
        res[i,j] = calc_bin(&a[i*K, j*K], K, y_offset)
    return res.base

And now (after verifying that the results are the same):

a=np.arange(N*N, dtype=np.float64).reshape(N,N)
%timeit cy_bin2d_parallel(a,K)
# 1.51 ms ± 25.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
# without parallelization: 2.93 ms

which is about 30% faster. Using -c=/arch:AVX512 (otherwise compiled only with /Ox with MSVC) as proposed by @max9111, made the single-threaded version about 20% faster but brought only about 5% for the parallel version.

Numba:

There is the same algorithm compiled with numba (which often can beat cython due to better performance of the clang-compiler) - but the result is slightly slower than cython but beats numpy by about 20%:

import numba as nb
@nb.njit(parallel=True)
def nb_bin2d_parallel(a, K):
    m_bins = a.shape[0]//K
    n_bins = a.shape[1]//K
    res = np.zeros((m_bins, n_bins), dtype=np.float64)

    for k in nb.prange(m_bins*n_bins):
        i = k//m_bins
        j = k%m_bins
        for y in range(i*K+1, (i+1)*K):
            for x in range(j*K, (j+1)*K):
                a[i*K, x] += a[y,x]
        s=0.0
        for x in range(j*K, (j+1)*K):
            s+=a[i*K, x]
        res[i,j] = s
    return res

and now:

a=np.arange(N*N, dtype=np.float64).reshape(N,N)
%timeit nb_bin2d_parallel(a,K)
# 1.98 ms ± 162 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
# without parallelization: 5.8 ms

In a nutshell: I guess it is possible to beat the above, but there is no free lunch anymore as numpy does its job pretty well. The most potential is probably in the parallelization, but it is limited due to the memory-bandwidth-bound nature of the problem (and probably one should use a smarter strategy as mine - for 50*50 summation one possible still can see an overhead for creation/management of the threads).


There is another faster (at least for sizes of about 2000) attempt with Cython, which perform the summation not for small 50-element parts, but for the whole row thus reducing the overhead (but may have more cache misses once the row is larger than 1-2k):

%%cython -a --verbose -c=/openmp --link-args=/openmp -c=/arch:AVX512
import numpy as np
import cython
from cython.parallel import prange
cdef extern from *:
    """
    void calc_bin_row(double *ptr, int N, int y_offset, double* out){
       double *row = ptr;
       for(int y=1;y<N;y++){
           row+=y_offset; //next row
           for(int x=0;x<y_offset;x++){
               ptr[x]+=row[x];
           }
       }
       double res=0.0;
       int i=0;
       int k=0;
       for(int x=0;x<y_offset;x++){//could be made faster, but is it needed?
           res+=ptr[x];
           k++;
           if(k==N){
              k=0;
              out[i]=res;
              i++;
              res=0.0;
           }
       }
    }
    """
    void calc_bin_row(double *ptr, int N, int y_offset, double* out) nogil


@cython.boundscheck(False)
@cython.wraparound(False)
def cy_bin2d_parallel_rowise(double[:, ::1] a, int K):
    cdef int y_offset = a.shape[1]
    cdef int m_bins = a.shape[0]//K
    cdef int n_bins = a.shape[1]//K
    cdef double[:,:] res = np.empty((m_bins, n_bins), dtype=np.float64)
    cdef int i,j,k
    for k in prange(0, y_offset, K, nogil=True):
        calc_bin_row(&a[k, 0], K, y_offset, &res[k//K, 0])
    return res.base

which is already faster singlethreaded (2ms) and about 60% (1.27 ms ± 50.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)) faster with parallelization.

2 of 2
1

This is basically a comment on @ead answer.

Normally the best you can do on a aligned summation is too use scalars as much as possible (which maps to registers). This function also doesn't modify the input array, which may not be wanted.

Code and Timings

@nb.njit(parallel=True,fastmath=True,cache=True)
def nb_bin2d_parallel_2(a, K):
    #There is no bounds-checking, make sure that the dimensions are OK
    assert a.shape[0]%K==0
    assert a.shape[1]%K==0

    m_bins = a.shape[0]//K
    n_bins = a.shape[1]//K
    #Works for all datatypes, but overflow especially in small integer types
    #may occur
    res = np.zeros((m_bins, n_bins), dtype=a.dtype)

    for i in nb.prange(res.shape[0]):
        for ii in range(i*K,(i+1)*K):
            for j in range(res.shape[1]):
                TMP=res[i,j]
                for jj in range(j*K,(j+1)*K):
                    TMP+=a[ii,jj]
                res[i,j]=TMP
    return res

N,K=2000,50
a=np.arange(N*N, dtype=np.float64).reshape(N,N)

#warmup (Numba compilation is on the first call)
res_1=nb_bin2d_parallel(a, K)
res_2=cy_bin2d_parallel(a,K)
res_3=bin2d(a,K)
res_4=nb_bin2d_parallel_2(a, K)

%timeit bin2d(a,K)
#2.51 ms ± 25.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
%timeit nb_bin2d_parallel(a, K)
#1.33 ms ± 33.3 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit nb_bin2d_parallel_2(a, K)
#1.05 ms ± 8.96 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit cy_bin2d_parallel(a,K) #arch:AVX2
#996 µs ± 7.94 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

N,K=4000,50
a=np.arange(N*N, dtype=np.float64).reshape(N,N)

%timeit bin2d(a,K)
#10.8 ms ± 56.5 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
%timeit nb_bin2d_parallel(a, K)
#5.13 ms ± 46.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
%timeit nb_bin2d_parallel_2(a, K)
#3.99 ms ± 31.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
%timeit cy_bin2d_parallel(a,K) #arch:AVX2
#4.31 ms ± 168 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
🌐
SciPy
docs.scipy.org › doc › scipy › reference › generated › scipy.stats.binned_statistic_2d.html
binned_statistic_2d — SciPy v1.18.0 Manual
Note that the returned linearized bin indices are used for an array with extra bins on the outer binedges to capture values outside of the defined bin bounds. If ‘True’: The returned binnumber is a shape (2,N) ndarray where each row indicates bin placements for each dimension respectively.
Top answer
1 of 3
21

You can use a higher dimensional view of your array and take the average along the extra dimensions:

In [12]: a = np.arange(36).reshape(6, 6)

In [13]: a
Out[13]: 
array([[ 0,  1,  2,  3,  4,  5],
       [ 6,  7,  8,  9, 10, 11],
       [12, 13, 14, 15, 16, 17],
       [18, 19, 20, 21, 22, 23],
       [24, 25, 26, 27, 28, 29],
       [30, 31, 32, 33, 34, 35]])

In [14]: a_view = a.reshape(3, 2, 3, 2)

In [15]: a_view.mean(axis=3).mean(axis=1)
Out[15]: 
array([[  3.5,   5.5,   7.5],
       [ 15.5,  17.5,  19.5],
       [ 27.5,  29.5,  31.5]])

In general, if you want bins of shape (a, b) for an array of (rows, cols), your reshaping of it should be .reshape(rows // a, a, cols // b, b). Note also that the order of the .mean is important, e.g. a_view.mean(axis=1).mean(axis=3) will raise an error, because a_view.mean(axis=1) only has three dimensions, although a_view.mean(axis=1).mean(axis=2) will work fine, but it makes it harder to understand what is going on.

As is, the above code only works if you can fit an integer number of bins inside your array, i.e. if a divides rows and b divides cols. There are ways to deal with other cases, but you will have to define the behavior you want then.

2 of 3
1

See the SciPy Cookbook on rebinning, which provides this snippet:

def rebin(a, *args):
    '''rebin ndarray data into a smaller ndarray of the same rank whose dimensions
    are factors of the original dimensions. eg. An array with 6 columns and 4 rows
    can be reduced to have 6,3,2 or 1 columns and 4,2 or 1 rows.
    example usages:
    >>> a=rand(6,4); b=rebin(a,3,2)
    >>> a=rand(6); b=rebin(a,2)
    '''
    shape = a.shape
    lenShape = len(shape)
    factor = asarray(shape)/asarray(args)
    evList = ['a.reshape('] + \
             ['args[%d],factor[%d],'%(i,i) for i in range(lenShape)] + \
             [')'] + ['.sum(%d)'%(i+1) for i in range(lenShape)] + \
             ['/factor[%d]'%i for i in range(lenShape)]
    print ''.join(evList)
    return eval(''.join(evList))
Top answer
1 of 2
2

So you are looking for median over strided reshape:

import numpy as np
a = np.arange(24).reshape(4,6)

def median_binner(a,bin_x,bin_y):
    m,n = np.shape(a)
    strided_reshape = np.lib.stride_tricks.as_strided(a,shape=(bin_x,bin_y,m//bin_x,n//bin_y),strides = a.itemsize*np.array([(m / bin_x) * n, (n / bin_y), n, 1]))
    return np.array([np.median(col) for row in strided_reshape for col in row]).reshape(bin_x,bin_y)



print "Original Matrix:"
print a
print "\n"
bin_tester1 = median_binner(a,2,3)
print "2x3 median bin :"
print bin_tester1
print "\n"
bin_tester2 = median_binner(a,2,2)
print "2x2 median bin :"
print bin_tester2

result:

Original Matrix:
[[ 0  1  2  3  4  5]
 [ 6  7  8  9 10 11]
 [12 13 14 15 16 17]
 [18 19 20 21 22 23]]


2x3 median bin :
[[  3.5   5.5   7.5]
 [ 15.5  17.5  19.5]]


2x2 median bin :
[[  4.   7.]
 [ 16.  19.]]

Read this in order to completely understand the following line in the code:

strided_reshape = np.lib.stride_tricks.as_strided(a,shape=(bin_x,bin_y,m//bin_x,n//bin_y),strides = a.itemsize*np.array([(m / bin_x) * n, (n / bin_y), n, 1])) .

2 of 2
0

I was dealing with the same issue. I have found the answer of Kennet Celeste very useful but there are some caveats. First the stride reshape is fast but the loop then is slow. The trick is to get all the data you compute median from to the same location in the memory and use somehow vectorized numpy operation.

If you don't want to fiddle with the stride reshape you can go for np.swapaxes function. So let's say I have an array X of the size xdim x ydim and want to bin it by window bin_x x bin_y

import numpy as np
#Some sample values
xdim= 5039
ydim = 6637
bin_x = 5
bin_y = 7
X = np.random.rand(ydim, xdim)
#now compute reduced dimensions so that bin_x divides xdim_red
xdim_red = xdim - xdim % bin_x
ydim_red = ydim - ydim % bin_y
#and dimensions after binning
xdim_bin = xdim_red // bin_x
ydim_bin = ydim_red // bin_y
#crop X to the end of the indices
X = X[0:ydim_red, 0:xdim_red]
#Here alternative to stride reshape
X.shape = (ydim_bin, bin_y, xdim_bin, bin_x)
X_reshaped = X.swapaxes(1, 2)
#The following can be done on stride_reshape array as well and finally joins the chunks of the memory we need to get together  
X_reshaped = X_reshaped.reshape((ydim_bin, xdim_bin, bin_x*bin_y))
#There could be faster implementation but this at least use batc
g = np.median(X_reshaped, axis=-1)
Find elsewhere
🌐
PyPI
pypi.org › project › vorbin
vorbin · PyPI
While signal and noise are typically used to compute the binning, the S/N can also be calculated on-the-fly using the sn_func keyword. To create bins with approximately equal total signal, one can assume Poissonian noise by setting noise = np.sqrt(signal). ... The desired signal-to-noise ratio for the final 2D-binned data.
      » pip install vorbin
    
Published: Sep 14, 2025
Version: 3.2.1
🌐
NumPy
numpy.org › doc › stable › reference › generated › numpy.histogram2d.html
numpy.histogram2d — NumPy v2.5 Manual
A combination [int, array] or [array, int], where int is the number of bins and array is the bin edges.
🌐
Reddit
reddit.com › r/learnpython › help with binning data in a numpy 2d array based on first column values, maybe using pandas.cut?
r/learnpython on Reddit: Help with binning data in a numpy 2d array based on first column values, maybe using pandas.cut?
August 27, 2020 - Subreddit for posting questions and asking for general advice about your python code. Members · Online • · DarkAvenger12 · ADMIN MOD · I have a numpy 2d array of size N x 3: my_arr = np.array([[x1,y1,z1], [x2,y2,z2], ... , [xN,yN,zN]]) Ultimately I want to bin each row in my_arr based on the x value (I already have pre-made bins).
🌐
SciPy
docs.scipy.org › doc › scipy-0.14.0 › reference › generated › scipy.stats.binned_statistic_2d.html
scipy.stats.binned_statistic_2d — SciPy v0.14.0 Reference Guide
Compute a bidimensional binned statistic for a set of data · This is a generalization of a histogram2d function. A histogram divides the space into bins, and returns the count of the number of points in each bin. This function allows the computation of the sum, mean, median, or other statistic ...
Top answer
1 of 2
4

You may use scipy.stats.binned_statistic to get the mean of the data in each bin. The bins would best be created via numpy.logspace. You may then plot those means e.g. as horiziontal lines spanning the bin width or as scatter at the mean position.

import numpy as np; np.random.seed(42)
from scipy.stats import binned_statistic
import matplotlib.pyplot as plt

x = np.logspace(0,5,300)
y = np.logspace(0,5,300)+np.random.rand(300)*1.e3


fig, ax = plt.subplots()
ax.scatter(x,y, s=9)

s, edges, _ = binned_statistic(x,y, statistic='mean', bins=np.logspace(0,5,6))

ys = np.repeat(s,2)
xs = np.repeat(edges,2)[1:-1]
ax.hlines(s,edges[:-1],edges[1:], color="crimson", )

for e in edges:
    ax.axvline(e, color="grey", linestyle="--")

ax.scatter(edges[:-1]+np.diff(edges)/2, s, c="limegreen", zorder=3)

ax.set_xscale("log")
ax.set_yscale("log")
plt.show()

2 of 2
1

You can achieve this with pandas. The idea is to assign each X value to an interval using np.digitize. Since you are using a log scale, it makes sense to use np.logspace to choose intervals of exponentially changing lengths. Finally, you can group X values in each interval and compute mean Y values.


import pandas as pd
import numpy as np

x_max = 10

xs = np.exp(x_max * np.random.rand(1000))
ys = np.exp(np.random.rand(1000))

df = pd.DataFrame({
    'X': xs,
    'Y': ys,
})

df['Xbins'] = np.digitize(df.X, np.logspace(0, x_max, 30, base=np.exp(1)))
df['Ymean'] = df.groupby('Xbins').Y.transform('mean')
df.plot(kind='scatter', x='X', y='Ymean')
🌐
Stack Exchange
scicomp.stackexchange.com › questions › 36854 › bin-2d-array-such-that-each-bin-contains-equal-number-of-samples
python - bin 2d array such that each bin contains equal number of samples? - Computational Science Stack Exchange
February 15, 2021 - You could compute the Voronoi diagram. Each cell in this diagram contains exactly one point and taken together the cells cover the domain. If you need, say, $n$ points in each bin, it is then a matter of picking a cell, merging it with its $n-1$ ...
🌐
Python Data Science Handbook
jakevdp.github.io › PythonDataScienceHandbook › 04.05-histograms-and-binnings.html
Histograms, Binnings, and Density | Python Data Science Handbook
The two-dimensional histogram creates a tesselation of squares across the axes. Another natural shape for such a tesselation is the regular hexagon. For this purpose, Matplotlib provides the plt.hexbin routine, which will represents a two-dimensional dataset binned within a grid of hexagons:
🌐
Readthedocs
physt.readthedocs.io › en › latest › 2d_histograms.html
2D Histograms in physt — Physt 0.9.0 documentation
Histogram2D('Some histogram', bins=(8, 4), total=1000, dtype=int64) [4]: # Frequencies are a 2D-array histogram.frequencies ·
🌐
NumPy
numpy.org › devdocs › reference › generated › numpy.histogram2d.html
numpy.histogram2d — NumPy v2.6.dev0 Manual
A combination [int, array] or [array, int], where int is the number of bins and array is the bin edges.