Sorry for the necromancy, but this answer can be substantially improved upon, using the invaluable np.einsum.
import numpy as np
D,M,N,R = 1,2,3,4
A = np.random.rand(M,N,R)
B = np.random.rand(N,D,R)
print np.einsum('mnr,ndr->mdr', A, B).shape
Note that it has several advantages: first of all, its fast. np.einsum is well-optimized generally, but moreover, np.einsum is smart enough to avoid the creation of an MxNxR temporary array, but performs the contraction over N directly.
But perhaps more importantly, its very readable. There is no doubt that this code is correct; and you could make it a lot more complicated without any trouble.
Note that the dummy 'D' axis can simply be dropped from B and the einsum statement if you wish.
Answer from Eelco Hoogendoorn on Stack OverflowI currently have a loop where I have a 3D array A[2x2x1000] and a 2D array B[2x2]. I want to multiply A by B for every value in the 3rd dimension without using a loop.
My current loop does the job but it runs slowly.
for i in range(1000):
C[:,:,i] = A[:,:,i]*BI've come across np.einsum, np.matmul and np.tensordot but I'm not completely sure how these work and how to use them.
Any help would be much appreciated.
A loopless 3D matrix multiplication in python - Stack Overflow
Does python have native capability (ie via numpy) for multiplying 3D arrays like Matlab's pagemtimes?
python - 3d Matrix multiplication in numpy - Stack Overflow
python - Numpy elementwise product of 3d array - Stack Overflow
Sorry for the necromancy, but this answer can be substantially improved upon, using the invaluable np.einsum.
import numpy as np
D,M,N,R = 1,2,3,4
A = np.random.rand(M,N,R)
B = np.random.rand(N,D,R)
print np.einsum('mnr,ndr->mdr', A, B).shape
Note that it has several advantages: first of all, its fast. np.einsum is well-optimized generally, but moreover, np.einsum is smart enough to avoid the creation of an MxNxR temporary array, but performs the contraction over N directly.
But perhaps more importantly, its very readable. There is no doubt that this code is correct; and you could make it a lot more complicated without any trouble.
Note that the dummy 'D' axis can simply be dropped from B and the einsum statement if you wish.
numpy.tensordot() is the right way to do it:
a = numpy.arange(24).reshape(2, 3, 4)
b = numpy.arange(12).reshape(3, 1, 4)
c = numpy.tensordot(a, b, axes=[1, 0]).diagonal(axis1=1, axis2=3)
Edit: The first version of this was faulty, and this version computes more han it should and throws away most of it. Maybe a Python loop over the last axis is the better way to do it.
Another Edit: I've come to the conclusion that numpy.tensordot() is not the best solution here.
c = (a[:,:,None] * b).sum(axis=1)
will be more efficient (though even harder to grasp).
Matlab has a really useful function called pagemtimes that essentially does
def pagemtimes(A,B):
import numpy as np
#A and B are 3D arrays with q layers
#each layer of A and B is compatible
#for matrix multiplication
szA=np.shape(A)
szB=np.shape(B)
C=np.zeros((szA[0],szB[1],szB[2]))
for i in range(sz[2])
C[:,:,i]=np.matmul(A[:,:,i],B[:,:,i])
return CDoes numpy have a function that has this functionality? I haven't been able to find anything. So if not, is there a library that has leverages compiled code for this function?
A for loop works just fine, but I'd like to accelerate things as much as possible. I'd normally take care of writing and compiling my own routine (done this countless times with Matlab). But I'm relatively new to python and I'm working on a very fast turn around project that doesn't have space for me to learn this at the moment.
Thanks in advance!
For array of dimension 3 (or, a rank-3 tensor) that you have, you can use np.einsum doc for more complex matrix multiplications. In your particular case, you can use the following
>>> import numpy as np
>>> x = np.random.randint(0, 3, (3, 3, 3)) # shape (3, 3, 3)
>>> y = np.random.randint(0, 3, (3, 3, 3)) # shape (3, 3, 3)
>>> np.einsum('ijk,ikl->ijl', x, y) # still shape (3, 3, 3)
In particular, the einsum expression 'ijk,ikl->ijl' means for each ith matrix, do a regular matrix multiplication jk,kl->jl and put the result in the ith entry in the resulting tensor (ndarray). A more general form of this process could be
np.einsum('...jk,...kl->...jl', x, y)
where you can have arbitrary number of dimensions in front of each tensor (ndarray) that you have.
See the following for a complete example:
>>> import numpy as np
>>> x = np.random.randint(0, 3, (3, 3, 3)) # shape (3, 3, 3)
>>> x
array([[[0, 0, 1],
[2, 2, 1],
[2, 1, 1]],
[[2, 0, 2],
[2, 2, 1],
[2, 2, 2]],
[[2, 2, 2],
[1, 1, 2],
[0, 2, 2]]])
>>> y = np.random.randint(0, 3, (3, 3, 3)) # shape (3, 3, 3)
>>> y
array([[[0, 0, 1],
[2, 1, 0],
[0, 0, 2]],
[[1, 2, 0],
[2, 0, 1],
[2, 2, 1]],
[[0, 2, 1],
[0, 1, 0],
[0, 2, 1]]])
>>> np.einsum('ijk,ikl->ijl', x, y)
array([[[ 0, 0, 2],
[ 4, 2, 4],
[ 2, 1, 4]],
[[ 6, 8, 2],
[ 8, 6, 3],
[10, 8, 4]],
[[ 0, 10, 4],
[ 0, 7, 3],
[ 0, 6, 2]]])
>>> np.einsum('...ij,...jk->...ik', x, y)
array([[[ 0, 0, 2],
[ 4, 2, 4],
[ 2, 1, 4]],
[[ 6, 8, 2],
[ 8, 6, 3],
[10, 8, 4]],
[[ 0, 10, 4],
[ 0, 7, 3],
[ 0, 6, 2]]])
np.matmul(A,B) works just fine. What error(s) did you get?
In [263]: A,B = np.arange(24).reshape(2,3,4), np.arange(8).reshape(2,4,1)
The einsum solution:
In [264]: np.einsum('ijk,ikl->ijl',A,B)
Out[264]:
array([[[ 14],
[ 38],
[ 62]],
[[302],
[390],
[478]]])
In [265]: _.shape
Out[265]: (2, 3, 1)
The matmul solution:
In [266]: A@B
Out[266]:
array([[[ 14],
[ 38],
[ 62]],
[[302],
[390],
[478]]])
Your loop:
In [267]: [np.matmul(a, b) for a, b in zip(A, B)]
Out[267]:
[array([[14],
[38],
[62]]),
array([[302],
[390],
[478]])]
matmuldocs:
If either argument is N-D, N > 2, it is treated as a stack of
matrices residing in the last two indexes and broadcast accordingly.
It seems you are doing matrix-multiplications for each slice along the first axis. For the same, you can use np.einsum like so -
np.einsum('ijk,ikl->ijl',A,B)
We can also use np.matmul -
np.matmul(A,B)
On Python 3.x, this matmul operation simplifies with @ operator -
A @ B
Benchmarking
Approaches -
def einsum_based(A,B):
return np.einsum('ijk,ikl->ijl',A,B)
def matmul_based(A,B):
return np.matmul(A,B)
def forloop(A,B):
N = A.shape[0]
C = np.zeros((N,2,2))
for i in range(N):
C[i] = np.dot(A[i], B[i])
return C
Timings -
In [44]: N = 10000
...: A = np.random.rand(N,2,2)
...: B = np.random.rand(N,2,2)
In [45]: %timeit einsum_based(A,B)
...: %timeit matmul_based(A,B)
...: %timeit forloop(A,B)
100 loops, best of 3: 3.08 ms per loop
100 loops, best of 3: 3.04 ms per loop
100 loops, best of 3: 10.9 ms per loop
You just need to perform the operation on the first dimension of your tensors, which is labeled by 0:
c = tensordot(a, b, axes=(0,0))
This will work as you wish. Also you don't need a list of axes, because it's just along one dimension you're performing the operation. With axes([1,2],[2,1]) you're cross multiplying the 2nd and 3rd dimensions. If you write it in index notation (Einstein summing convention) this corresponds to c[i,j] = a[i,k,l]*b[j,k,l], thus you're contracting the indices you want to keep.
EDIT: Ok, the problem is that the tensor product of a two 3d object is a 6d object. Since contractions involve pairs of indices, there's no way you'll get a 3d object by a tensordot operation. The trick is to split your calculation in two: first you do the tensordot on the index to do the matrix operation and then you take a tensor diagonal in order to reduce your 4d object to 3d. In one command:
d = np.diagonal(np.tensordot(a,b,axes=()), axis1=0, axis2=2)
In tensor notation d[i,j,k] = c[i,j,i,k] = a[i,j,l]*b[i,l,k].
I have two 3d numpy arrays arr1 and arr2 of sizes (n, m, p) and (n, p, k) respectively. np.matmul(arr1, arr2) does a batched matrix multiplication of the last two dimensions to yield an array of shape (n, m, k).
However, I've noticed that using a for loop with np.dot is actually faster.
res = np.empty((n, m, k))
for idx in range(n):
res[idx] = np.dot(arr1[idx], arr2[idx])Why is this the case? I would have assumed that np.matmul would have been optimized for this. I also tried using np.einsum('ijk,ikl->ijl', arr1, arr2). It's faster than np.matmul but still slower than the for loop and np.dot.
Any ideas why this is the case? Are there other faster ways to compute this matrix product? Assume that no GPU or parallelization is allowed.
Edit: A user in comments reported getting faster speeds with np.matmul. I was also able to confirm the same on a different machine. This has likely happened due to an issue with the specific machine I was using. Either its a numpy version bug (1.23.5) or something else.
The easiest is:
C = numpy.sum(A * B, -1)
I think this might work too:
C = numpy.einsum("...i,...i->...", A, B)
try this: numpy.sum( A*B, axis=2 )
this is similar to the other suggestion but perhaps clearer (axes are numbered from 0, so axis=2 is the 3rd axis or H out of MxNxH)