@Ney @hpaulj is correct, you need to experiment, but I suspect you don't realize that summation for some arrays can occur along axes. Observe the following which reading the documentation
>>> a
array([[0, 0, 0],
[0, 1, 0],
[0, 2, 0],
[1, 0, 0],
[1, 1, 0]])
>>> np.sum(a, keepdims=True)
array([[6]])
>>> np.sum(a, keepdims=False)
6
>>> np.sum(a, axis=1, keepdims=True)
array([[0],
[1],
[2],
[1],
[2]])
>>> np.sum(a, axis=1, keepdims=False)
array([0, 1, 2, 1, 2])
>>> np.sum(a, axis=0, keepdims=True)
array([[2, 4, 0]])
>>> np.sum(a, axis=0, keepdims=False)
array([2, 4, 0])
You will notice that if you don't specify an axis (1st two examples), the numerical result is the same, but the keepdims = True returned a 2D array with the number 6, whereas, the second incarnation returned a scalar.
Similarly, when summing along axis 1 (across rows), a 2D array is returned again when keepdims = True.
The last example, along axis 0 (down columns), shows a similar characteristic... dimensions are kept when keepdims = True.
Studying axes and their properties is critical to a full understanding of the power of NumPy when dealing with multidimensional data.
@Ney @hpaulj is correct, you need to experiment, but I suspect you don't realize that summation for some arrays can occur along axes. Observe the following which reading the documentation
>>> a
array([[0, 0, 0],
[0, 1, 0],
[0, 2, 0],
[1, 0, 0],
[1, 1, 0]])
>>> np.sum(a, keepdims=True)
array([[6]])
>>> np.sum(a, keepdims=False)
6
>>> np.sum(a, axis=1, keepdims=True)
array([[0],
[1],
[2],
[1],
[2]])
>>> np.sum(a, axis=1, keepdims=False)
array([0, 1, 2, 1, 2])
>>> np.sum(a, axis=0, keepdims=True)
array([[2, 4, 0]])
>>> np.sum(a, axis=0, keepdims=False)
array([2, 4, 0])
You will notice that if you don't specify an axis (1st two examples), the numerical result is the same, but the keepdims = True returned a 2D array with the number 6, whereas, the second incarnation returned a scalar.
Similarly, when summing along axis 1 (across rows), a 2D array is returned again when keepdims = True.
The last example, along axis 0 (down columns), shows a similar characteristic... dimensions are kept when keepdims = True.
Studying axes and their properties is critical to a full understanding of the power of NumPy when dealing with multidimensional data.
An example showing keepdims in action when working with higher dimensional arrays. Let's see how the shape of the array changes as we do different reductions:
import numpy as np
a = np.random.rand(2,3,4)
a.shape
# => (2, 3, 4)
# Note: axis=0 refers to the first dimension of size 2
# axis=1 refers to the second dimension of size 3
# axis=2 refers to the third dimension of size 4
a.sum(axis=0).shape
# => (3, 4)
# Simple sum over the first dimension, we "lose" that dimension
# because we did an aggregation (sum) over it
a.sum(axis=0, keepdims=True).shape
# => (1, 3, 4)
# Same sum over the first dimension, but instead of "loosing" that
# dimension, it becomes 1.
a.sum(axis=(0,2)).shape
# => (3,)
# Here we "lose" two dimensions
a.sum(axis=(0,2), keepdims=True).shape
# => (1, 3, 1)
# Here the two dimensions become 1 respectively
Dba = np.sum(dtanh,axis=1 or keepdims = True)
np.sum with keepdims fails when input is a 1d array
python - Numpy sum keepdims error - Stack Overflow
What is the role of keepdims in Numpy (Python)? - Stack Overflow
The keepdims argument was added in NumPy 1.7. At least the docstring of np.sum (1.6) hasn't listed it as one of the arguments:
numpy.sum(a, axis=None, dtype=None, out=None)
However the 1.7 docstring already listed it:
numpy.sum(a, axis=None, dtype=None, out=None, keepdims=False)
Given that NumPy 1.6 was released in 2012 you probably should update your NumPy package.
However you could also use np.expand_dims in case you can't (or don't want to) update NumPy:
np.expand_dims(np.sum(exp_scores, axis=1), axis=1)
The argument is valid in the latest version of numpy as explained here. Here is the full list of argument for numpy.sum:
numpy.sum(a, axis=None, dtype=None, out=None, keepdims=False)
This was added since version 1.7 as you can see in the source code here. So, you need to upgrade your numpy installation.
Consider a small 2d array:
In [180]: A=np.arange(12).reshape(3,4)
In [181]: A
Out[181]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
Sum across rows; the result is a (3,) array
In [182]: A.sum(axis=1)
Out[182]: array([ 6, 22, 38])
But to sum (or divide) A by the sum requires reshaping
In [183]: A-A.sum(axis=1)
...
ValueError: operands could not be broadcast together with shapes (3,4) (3,)
In [184]: A-A.sum(axis=1)[:,None] # turn sum into (3,1)
Out[184]:
array([[ -6, -5, -4, -3],
[-18, -17, -16, -15],
[-30, -29, -28, -27]])
If I use keepdims, "the result will broadcast correctly against" A.
In [185]: A.sum(axis=1, keepdims=True) # (3,1) array
Out[185]:
array([[ 6],
[22],
[38]])
In [186]: A-A.sum(axis=1, keepdims=True)
Out[186]:
array([[ -6, -5, -4, -3],
[-18, -17, -16, -15],
[-30, -29, -28, -27]])
If I sum the other way, I don't need the keepdims. Broadcasting this sum is automatic: A.sum(axis=0)[None,:]. But there's no harm in using keepdims.
In [190]: A.sum(axis=0)
Out[190]: array([12, 15, 18, 21]) # (4,)
In [191]: A-A.sum(axis=0)
Out[191]:
array([[-12, -14, -16, -18],
[ -8, -10, -12, -14],
[ -4, -6, -8, -10]])
If you prefer, these actions might make more sense with np.mean, normalizing the array over columns or rows. In any case it can simplify further math between the original array and the sum/mean.
You can keep the dimension with "keepdims=True" if you sum a matrix
For example:
import numpy as np
x = np.array([[1,2,3],[4,5,6]])
x.shape
# (2, 3)
np.sum(x, keepdims=True).shape
# (1, 1)
np.sum(x, keepdims=True)
# array([[21]]) <---the reault is still a 1x1 array
np.sum(x, keepdims=False).shape
# ()
np.sum(x, keepdims=False)
# 21 <--- the result is an integer with no dimesion