Broadcasting involves 2 steps
give all arrays the same number of dimensions
expand the
1dimensions to match the other arrays
With your inputs
(41,6) (41,)
one is 2d, the other 1d; broadcasting can change the 1d to (1, 41), but it does not automatically expand in the other direction (41,1).
(41,6) (1,41)
Neither (41,41) or (6,41) matches the other.
So you need to change your y to (41,1) or the x to (6,41)
x.T*y
x*y[:,None]
I'm assuming, of course, that you want element by element multiplication, not the np.dot matrix product.
Broadcasting involves 2 steps
give all arrays the same number of dimensions
expand the
1dimensions to match the other arrays
With your inputs
(41,6) (41,)
one is 2d, the other 1d; broadcasting can change the 1d to (1, 41), but it does not automatically expand in the other direction (41,1).
(41,6) (1,41)
Neither (41,41) or (6,41) matches the other.
So you need to change your y to (41,1) or the x to (6,41)
x.T*y
x*y[:,None]
I'm assuming, of course, that you want element by element multiplication, not the np.dot matrix product.
Not exactly sure, what you are trying to achieve. Maybe you could give an example of your input and your expected output. One possibility is:
import numpy as np
x = np.array([[1, 2], [1, 2], [1, 2]])
y = np.array([1, 2, 3])
res = x * np.transpose(np.array([y,]*2))
This will multiply each column of x with y, so the result of the above example is:
array([[1, 2],
[2, 4],
[3, 6]])
numpy - Multidimensional matrix multiplication in python - Stack Overflow
python - Multiplying specific dimension matrices - Stack Overflow
How can arrays with different dimensions be multiplied? - Python - Data Science Dojo Discussions
Python Numpy matrix multiplication in high dimension - Stack Overflow
Videos
If you have numpy arrays you can use the np.dot function for this:
np.dot(A, B)
It will do exactly what you want, i.e. "contract" the last axis of A with the first axis of B:
For 2-D arrays it is equivalent to matrix multiplication, and for 1-D arrays to inner product of vectors (without complex conjugation). For N dimensions it is a sum product over the last axis of a and the second-to-last of b:
dot(a, b)[i,j,k,m] = sum(a[i,j,:] * b[k,:,m])
First of all any multidimensional array is a n times b split of a flatten array. The 2,3,2 dimension array, [ [ [A,B],[C,D],[E,F] ], [ [G,H],[I,J],[K,L] ] ] is simply divide A,B,C,D,E,F,G,H,I,J,K,L into 2 pieces , then 3 pieces, then 2 pieces again and there is your multidimensional array. Vectors are here two dimensional.
Secondly, two dimensional matrix multiplication is also fixed length vector multiplication with a combination. (n,m)*(m,l) dimensional matrix multiplication is actually term by term multiplication and sum of results of l different m vectors and n different m vectors. l times n combination of two different m vectors.
When you get that you simply can convert 20 dimension matrix to two dimension matrix with same vector size and multiply them and reshape it back.
Quick Example:
import numpy as np
a=np.random.random(120).reshape(2,5,3,4)
b=np.random.random(120).reshape(5,3,4,2)
br=np.rollaxis(b,3,2).reshape(30,4).T
# converted all combinations to a series of 4 dimension vectors here
test1=np.matmul(a.reshape(30,4),br).reshape(2,5,3,5,3,2)
test2=np.dot(a,b)
np.all(test1==test2)
returns
array(True)
lets do it basically for all combinations (dimensions)
sa=a.shape
sb=b.shape
res=np.ndarray([sa[0],sa[1],sb[0],sb[1],sa[2],sb[3]])
for i in range(a.shape[0]):
for j in range(a.shape[1]):
for k in range(b.shape[0]):
for n in range(b.shape[1]):
x=np.matmul(a[i,j],b[k,n])
np.append(res,x)
print(np.any(res==np.dot(a,b).reshape(2,5,5,3,3,2)))
returns
True
USE CASE:
Suppose we have a case such as for 10 different materials and 20 different geometries 3 different physical quantity in 3 dimensions vectors will be matrix multiplied for a geometry and physics processing neural network layer which will be calculated with genetic selective algorithm which has neural connection coefficient vectors of 5 different groups of population each having 100 different gene sequence (population) and 20 neural nodes. In a such case you may benefit calculating it at once or you can somehow serialize this calculation to two flat arrays and send it to your gpu or cpu according to your concurrent free ram amount. In such case you may want to understand how this calculations work.
In any case of multidimensional matrix calculation combinations of vectors are calculated term by term You may want to multiply first term with seconds vectors last term and sum the rest. It is up to you but understanding how it works is important. So here is a simple illustration I have used to undestand this.
[ [ [A,B],[C,D],[E,F] ], [ [G,H],[I,J],[K,L] ] ] --> ABCDEFGHIJKL [ [ [1,2],[3,4],[5,6] ], [ [7,8],[9,0],[€,$] ] ] --> 1234567890€$
use operator(multiply) term by term shifting first array by amount of vector size (2)
ABCDEFGHIJKL CDEFGHIJKLAB FGHIJKLABCDE ...
1234567890€$ 1234567890€$ 1234567890€$ ...
Here comes all combinations
append all of them and reshape and use another operator (+)
[A+2*B],[C*3+D*4],[E*5,F*6] ...... [I*€+J*$]
Hope this helps and saves time to grasp this.
You may want to look into np.einsum. As an example:
>>> mat = np.arange(80).reshape(4, 4, 5)
>>> vec = np.arange(12).reshape(3, 4)
>>> np.einsum('ij,jkl,ik->il', vec, mat, vec)
array([[ 2100, 2136, 2172, 2208, 2244],
[20900, 21384, 21868, 22352, 22836],
[58900, 60344, 61788, 63232, 64676]])
If I didn't get the indices wrong, I have 5 matrices of shape 4x4, and 3 vectors of length 4, and am computing the 3x5 quadratic forms of each matrix with each vector.
Note that np.dot behaves differently with different dimension arrays, from here:
For 2-D arrays it is equivalent to matrix multiplication,
for 1-D arrays to inner product of vectors (without complex conjugation).
For N dimensions it is a sum product over the last axis of a and the second-to-last of b
In your example z has 3 dimensions.
Also, there is an important distinction between a numpy array and a numpy matrix:
You are using arrays in your example. Numpy matrices are 2D only and matrix multiplication is achived with the * operator. (With arrays this perfroms element by element multiplication)
This is a typical application for np.tensordot():
sum = np.tensordot(A, B, [[0,2],[0,1]])
Timing
Using the following code:
import numpy as np
n_examples = 100
A = np.random.randn(n_examples, 20,30)
B = np.random.randn(n_examples, 30,5)
def sol1():
sum = np.zeros([20,5])
for i in range(len(A)):
sum += np.dot(A[i],B[i])
return sum
def sol2():
return np.array(map(np.dot, A,B)).sum(0)
def sol3():
return np.einsum('nmk,nkj->mj',A,B)
def sol4():
return np.tensordot(A, B, [[2,0],[1,0]])
def sol5():
return np.tensordot(A, B, [[0,2],[0,1]])
Results:
timeit sol1()
1000 loops, best of 3: 1.46 ms per loop
timeit sol2()
100 loops, best of 3: 4.22 ms per loop
timeit sol3()
1000 loops, best of 3: 1.87 ms per loop
timeit sol4()
10000 loops, best of 3: 205 µs per loop
timeit sol5()
10000 loops, best of 3: 172 µs per loop
on my computer the tensordot() was the fastest solution and changing the order that the axes are evaluated did not change the results neither the performance.
Ha, it can be done in just one line: np.einsum('nmk,nkj->mj',A,B).
See Einstein summation: http://docs.scipy.org/doc/numpy/reference/generated/numpy.einsum.html
Not the same problem but the idea is quite much the same, see discussions and alternative methods in this topic we just discussed: numpy multiply matrices preserve third axis
Don't name your variable sum, you override the build-in sum.
As @Jaime pointed out, the loop is actually faster for dimensions of these size. In fact a solution based on map and sum is, albeit simpler, even slower:
In [19]:
%%timeit
SUM = np.zeros([20,5])
for i in range(len(A)):
SUM += np.dot(A[i],B[i])
10000 loops, best of 3: 115 µs per loop
In [20]:
%timeit np.array(map(np.dot, A,B)).sum(0)
1000 loops, best of 3: 445 µs per loop
In [21]:
%timeit np.einsum('nmk,nkj->mj',A,B)
1000 loops, best of 3: 259 µs per loop
Thing are different with larger dimension:
n_examples = 1000
A = np.random.randn(n_examples, 20,1000)
B = np.random.randn(n_examples, 1000,5)
And:
In [46]:
%%timeit
SUM = np.zeros([20,5])
for i in range(len(A)):
SUM += np.dot(A[i],B[i])
1 loops, best of 3: 191 ms per loop
In [47]:
%timeit np.array(map(np.dot, A,B)).sum(0)
1 loops, best of 3: 164 ms per loop
In [48]:
%timeit np.einsum('nmk,nkj->mj',A,B)
1 loops, best of 3: 451 ms per loop
This isn't making any sense to me:
import numpy as np
# Builds matrices
A = np.matrix('1 -2 3 7; 2 1 1 4; -3 2 -2 10')
B = np.matrix('0 1 -3 -2; 10 -1 2 -3; 5 1 -1 4')
C = np.matrix('4 0 -2 3; 3 6 9 7; 2 2 5 1; 9 4 6 -2')
print(A@B)That returns a value error, even though they're the same size.
A@C
That one works though, even though A and C are different sizes. Why can't NumPy do A@B?
I'm trying to multiply A = 20x1x10 with B = 10x20 and I'm supposed to get a 20x1x20 matrix as the output. So far I've tried
C = numpy.matmul(A,B) # and C = A @ B
But all of them seem to result a 20x20x20 matrix. Any idea how to proceed from here?
For better context, A = jacobian output of derivative of an activation function in a neural network, with shape of (N samples * 1 * values) and B = changes in output of the layer