numpy is implemented as a mix of C code and Python code. The source is available for browsing on github, and can be downloaded as a git repository. But digging your way into the C source takes some work. A lot of the files are marked as .c.src, which means they pass through one or more layers of perprocessing before compiling.
And Python is written in a mix of C and Python as well. So don't try to force things into C++ terms.
It's probably better to draw on your MATLAB experience, with adjustments to allow for Python. And numpy has a number of quirks that go beyond Python. It is using Python syntax, but because it has its own C code, it isn't simply a Python class.
I use Ipython as my usual working environment. With that I can use foo? to see the documentation for foo (same as the Python help(foo), and foo?? to see the code - if it is writen in Python (like the MATLAB/Octave type(foo))
Python objects have attributes, and methods. Also properties which look like attributes, but actually use methods to get/set. Usually you don't need to be aware of the difference between attributes and properties.
x.ndim # as noted, has a get, but no set; see also np.ndim(x)
x.shape # has a get, but can also be set; see also np.shape(x)
x.<tab> in Ipython shows me all the completions for a ndarray. There are 4*18. Some are methods, some attributes. x._<tab> shows a bunch more that start with __. These are 'private' - not meant for public consumption, but that's just semantics. You can look at them and use them if needed.
Off hand x.shape is the only ndarray property that I set, and even with that I usually use reshape(...) instead. Read their docs to see the difference. ndim is the number of dimensions, and it doesn't make sense to change that directly. It is len(x.shape); change the shape to change ndim. Likewise x.size shouldn't be something you change directly.
Some of these properties are accessible via functions. np.shape(x) == x.shape, similar to MATLAB size(x). (MATLAB doesn't have . attribute syntax).
x.__array_interface__ is a handy property, that gives a dictionary with a number of the attributes
In [391]: x.__array_interface__
Out[391]:
{'descr': [('', '<f8')],
'version': 3,
'shape': (50,),
'typestr': '<f8',
'strides': None,
'data': (165646680, False)}
The docs for ndarray(shape, dtype=float, buffer=None, offset=0,
strides=None, order=None), the __new__ method lists these attributes:
`Attributes
----------
T : ndarray
Transpose of the array.
data : buffer
The array's elements, in memory.
dtype : dtype object
Describes the format of the elements in the array.
flags : dict
Dictionary containing information related to memory use, e.g.,
'C_CONTIGUOUS', 'OWNDATA', 'WRITEABLE', etc.
flat : numpy.flatiter object
Flattened version of the array as an iterator. The iterator
allows assignments, e.g., ``x.flat = 3`` (See `ndarray.flat` for
assignment examples; TODO).
imag : ndarray
Imaginary part of the array.
real : ndarray
Real part of the array.
size : int
Number of elements in the array.
itemsize : int
The memory use of each array element in bytes.
nbytes : int
The total number of bytes required to store the array data,
i.e., ``itemsize * size``.
ndim : int
The array's number of dimensions.
shape : tuple of ints
Shape of the array.
strides : tuple of ints
The step-size required to move from one element to the next in
memory. For example, a contiguous ``(3, 4)`` array of type
``int16`` in C-order has strides ``(8, 2)``. This implies that
to move from element to element in memory requires jumps of 2 bytes.
To move from row-to-row, one needs to jump 8 bytes at a time
(``2 * 4``).
ctypes : ctypes object
Class containing properties of the array needed for interaction
with ctypes.
base : ndarray
If the array is a view into another array, that array is its `base`
(unless that array is also a view). The `base` array is where the
array data is actually stored.
All of these should be treated as properties, though I don't think numpy actually uses the property mechanism. In general they should be considered to be 'read-only'. Besides shape, I only recall changing data (pointer to a data buffer), and strides.
what's the difference between a numpy array with shape (x,) and (x, 1)?
python - numpy dimensions - Stack Overflow
python - Puzzled on the ndim from Numpy - Stack Overflow
python - Explaining the differences between dim, shape, rank, dimension and axis in numpy - Stack Overflow
Videos
numpy is implemented as a mix of C code and Python code. The source is available for browsing on github, and can be downloaded as a git repository. But digging your way into the C source takes some work. A lot of the files are marked as .c.src, which means they pass through one or more layers of perprocessing before compiling.
And Python is written in a mix of C and Python as well. So don't try to force things into C++ terms.
It's probably better to draw on your MATLAB experience, with adjustments to allow for Python. And numpy has a number of quirks that go beyond Python. It is using Python syntax, but because it has its own C code, it isn't simply a Python class.
I use Ipython as my usual working environment. With that I can use foo? to see the documentation for foo (same as the Python help(foo), and foo?? to see the code - if it is writen in Python (like the MATLAB/Octave type(foo))
Python objects have attributes, and methods. Also properties which look like attributes, but actually use methods to get/set. Usually you don't need to be aware of the difference between attributes and properties.
x.ndim # as noted, has a get, but no set; see also np.ndim(x)
x.shape # has a get, but can also be set; see also np.shape(x)
x.<tab> in Ipython shows me all the completions for a ndarray. There are 4*18. Some are methods, some attributes. x._<tab> shows a bunch more that start with __. These are 'private' - not meant for public consumption, but that's just semantics. You can look at them and use them if needed.
Off hand x.shape is the only ndarray property that I set, and even with that I usually use reshape(...) instead. Read their docs to see the difference. ndim is the number of dimensions, and it doesn't make sense to change that directly. It is len(x.shape); change the shape to change ndim. Likewise x.size shouldn't be something you change directly.
Some of these properties are accessible via functions. np.shape(x) == x.shape, similar to MATLAB size(x). (MATLAB doesn't have . attribute syntax).
x.__array_interface__ is a handy property, that gives a dictionary with a number of the attributes
In [391]: x.__array_interface__
Out[391]:
{'descr': [('', '<f8')],
'version': 3,
'shape': (50,),
'typestr': '<f8',
'strides': None,
'data': (165646680, False)}
The docs for ndarray(shape, dtype=float, buffer=None, offset=0,
strides=None, order=None), the __new__ method lists these attributes:
`Attributes
----------
T : ndarray
Transpose of the array.
data : buffer
The array's elements, in memory.
dtype : dtype object
Describes the format of the elements in the array.
flags : dict
Dictionary containing information related to memory use, e.g.,
'C_CONTIGUOUS', 'OWNDATA', 'WRITEABLE', etc.
flat : numpy.flatiter object
Flattened version of the array as an iterator. The iterator
allows assignments, e.g., ``x.flat = 3`` (See `ndarray.flat` for
assignment examples; TODO).
imag : ndarray
Imaginary part of the array.
real : ndarray
Real part of the array.
size : int
Number of elements in the array.
itemsize : int
The memory use of each array element in bytes.
nbytes : int
The total number of bytes required to store the array data,
i.e., ``itemsize * size``.
ndim : int
The array's number of dimensions.
shape : tuple of ints
Shape of the array.
strides : tuple of ints
The step-size required to move from one element to the next in
memory. For example, a contiguous ``(3, 4)`` array of type
``int16`` in C-order has strides ``(8, 2)``. This implies that
to move from element to element in memory requires jumps of 2 bytes.
To move from row-to-row, one needs to jump 8 bytes at a time
(``2 * 4``).
ctypes : ctypes object
Class containing properties of the array needed for interaction
with ctypes.
base : ndarray
If the array is a view into another array, that array is its `base`
(unless that array is also a view). The `base` array is where the
array data is actually stored.
All of these should be treated as properties, though I don't think numpy actually uses the property mechanism. In general they should be considered to be 'read-only'. Besides shape, I only recall changing data (pointer to a data buffer), and strides.
Regarding your first question, Python has syntactic sugar for properties, including fine-grained control of getting, setting, deleting them, as well as restricting any of the above.
So, for example, if you have
class Foo(object):
@property
def shmip(self):
return 3
then you can write Foo().shmip to obtain 3, but, if that is the class definition, you've disabled setting Foo().shmip = 4.
In other words, those are read-only properties.
hey, im doing an ai thing in school and my code didnt work as expected, and after 5 hours i found out i reshaped an array from (206,) to (206,1) and that made the results wrong. and from what i understand, the shape means the length of each dimension, and length is not 0 indexed so a size of 1 would be equal to just 1D no?
The shape of an array is a tuple of its dimensions. An array with one dimension has a shape of (n,). A two dimension array has a shape of (n,m) (like your case 2 and 3) and a three dimension array has a shape of (n,m,k) and so on.
Therefore, whilst the shape of your second and third example are different, the no. dimensions is two in both cases:
>>> a= np.array([[1,2,3],[4,5,6]])
>>> a.shape
(2, 3)
>>> b=np.arange(15).reshape(3,5)
>>> b.shape
(3, 5)
If you wanted to add another dimension to your examples you would have to do something like this:
a= np.array([[[1,2,3]],[[4,5,6]]])
or
np.arange(15).reshape(3,5,1)
You can keep adding dimensions in this way:
One dimension:
>>> a = np.zeros((2))
array([ 0., 0.])
>>> a.shape
(2,)
>>> a.ndim
1
Two dimensions:
>>> b = np.zeros((2,2))
array([[ 0., 0.],
[ 0., 0.]])
>>> b.shape
(2,2)
>>> b.ndim
2
Three dimensions:
>>> c = np.zeros((2,2,2))
array([[[ 0., 0.],
[ 0., 0.]],
[[ 0., 0.],
[ 0., 0.]]])
>>> c.shape
(2,2,2)
>>> c.ndim
3
Four dimensions:
>>> d = np.zeros((2,2,2,2))
array([[[[ 0., 0.],
[ 0., 0.]],
[[ 0., 0.],
[ 0., 0.]]],
[[[ 0., 0.],
[ 0., 0.]],
[[ 0., 0.],
[ 0., 0.]]]])
>>> d.shape
(2,2,2,2)
>>> d.ndim
4
While @atomh33ls has a fantastic written answer, I generated this visual to help communicate the differences between a numpy.ndarray.shape and a numpy.ndarray.ndim.
In this visual, the ndim increases as each new slider moves above 1. This is first seen when m is set to 1 defining a 1-dimensional ndarray with shape (1,).
Note that I use m, n, and k in order to represent the length of each additional dimension.

The tuple you give to zeros and other similar functions gives the 'shape' of the array (and is available for any array as a.shape). The number of dimensions is the number of entries in that shape.
If you were instead printing len(a.shape) and len(b.shape) you would expect the result you get. The ndim is always equal to this (because that's how it's defined).
The link you give shows the same behavior, except that the reshape method takes arbitary numbers of positional arguments rather than a tuple. But in the tutorial's example:
>>> a = arange(15).reshape(3, 5)
>>> a
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14]])
>>> a.shape
(3, 5)
>>> a.ndim
2
The number of arguments given to reshape, and the number of elements in the tuple given back by a.shape, is 2. If this was instead:
>>> a = np.arange(30).reshape(3, 5, 2)
>>> a.ndim
3
Then we see the same behavior: the ndim is the number of shape entries we gave to numpy.
zerogives an array of zeros with dimensions you specify:
>>> b = np.zeros((2,3,4), dtype=np.int16)
>>> b
array([[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]],
[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]]], dtype=int16)
For b you give three dimensions but for a you give four. Count the opening [.
>>> a = np.zeros((5,2,3,4), dtype=np.int16)
>>> a
array([[[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]],
[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]]],
...
shape echos back the dimensions you specified:
>>> a.shape
(5, 2, 3, 4)
>>> b.shape
(2, 3, 4)
Dimensionality of NumPy arrays must be understood in the data structures sense, not the mathematical sense, i.e. it's the number of scalar indices you need to obtain a scalar value.(*)
E.g., this is a 3-d array:
>>> X = np.arange(24).reshape(2, 3, 4)
>>> X
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]],
[[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]])
Indexing once gives a 2-d array (matrix):
>>> X[0]
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
Indexing twice gives a 1-d array (vector), and indexing three times gives a scalar.
The rank of X is its number of dimensions:
>>> X.ndim
3
>>> np.rank(X)
3
Axis is roughly synonymous with dimension; it's used in broadcasting operations:
>>> X.sum(axis=0)
array([[12, 14, 16, 18],
[20, 22, 24, 26],
[28, 30, 32, 34]])
>>> X.sum(axis=1)
array([[12, 15, 18, 21],
[48, 51, 54, 57]])
>>> X.sum(axis=2)
array([[ 6, 22, 38],
[54, 70, 86]])
To be honest, I find this definition of "rank" confusing since it matches neither the name of the attribute ndim nor the linear algebra definition of rank.
Now regarding np.dot, what you have to understand is that there are three ways to represent a vector in NumPy: 1-d array, a column vector of shape (n, 1) or a row vector of shape (1, n). (Actually, there are more ways, e.g. as a (1, n, 1)-shaped array, but these are quite rare.) np.dot performs vector multiplication when both arguments are 1-d, matrix-vector multiplication when one argument is 1-d and the other is 2-d, and otherwise it performs a (generalized) matrix multiplication:
>>> A = np.random.randn(2, 3)
>>> v1d = np.random.randn(2)
>>> np.dot(v1d, A)
array([-0.29269547, -0.52215117, 0.478753 ])
>>> vrow = np.atleast_2d(v1d)
>>> np.dot(vrow, A)
array([[-0.29269547, -0.52215117, 0.478753 ]])
>>> vcol = vrow.T
>>> np.dot(vcol, A)
Traceback (most recent call last):
File "<ipython-input-36-98949c6de990>", line 1, in <module>
np.dot(vcol, A)
ValueError: matrices are not aligned
The rule "sum product over the last axis of a and the second-to-last of b" matches and generalizes the common definition of matrix multiplication.
(*) Arrays of dtype=object are a bit of an exception, since they treat any Python object as a scalar.
np.dot is a generalization of matrix multiplication.
In regular matrix multiplication, an (N,M)-shape matrix multiplied with a (M,P)-shaped matrix results in a (N,P)-shaped matrix. The resultant shape can be thought of as being formed by squashing the two shapes together ((N,M,M,P)) and then removing the middle numbers, M (to produce (N,P)). This is the property that np.dot preserves while generalizing to arrays of higher dimension.
When the docs say,
"For N dimensions it is a sum product over the last axis of a and the second-to-last of b".
it is speaking to this point. An array of shape (u,v,M) dotted with an array of shape (w,x,y,M,z) would result in an array of shape (u,v,w,x,y,z).
Let's see how this rule looks when applied to
In [25]: V = np.arange(2); V
Out[25]: array([0, 1])
In [26]: M = np.arange(4).reshape(2,2); M
Out[26]:
array([[0, 1],
[2, 3]])
First, the easy part:
In [27]: np.dot(M, V)
Out[27]: array([1, 3])
There is no surprise here; this is just matrix-vector multiplication.
Now consider
In [28]: np.dot(V, M)
Out[28]: array([2, 3])
Look at the shape of V and M:
In [29]: V.shape
Out[29]: (2,)
In [30]: M.shape
Out[30]: (2, 2)
So np.dot(V,M) is like matrix multiplication of a (2,)-shaped matrix with a (2,2)-shaped matrix, which should result in a (2,)-shaped matrix.
The last (and only) axis of V and the second-to-last axis of M (aka the first axis of M) are multiplied and summed over, leaving only the last axis of M.
If you want to visualize this: np.dot(V, M) looks as though V has 1 row and 2 columns:
[[0, 1]] * [[0, 1],
[2, 3]]
and so, when V is multiplied by M, np.dot(V, M) equals
[[0*0 + 1*2], [2,
[0*1 + 1*3]] = 3]
However, I don't really recommend trying to visualize NumPy arrays this way -- at least I never do. I focus almost exclusively on the shape.
(2,) * (2,2)
\ /
\ /
(2,)
You just think about the "middle" axes being dotted, and disappearing from the resultant shape.
np.sum(arr, axis=0) tells NumPy to sum the elements in arr eliminating the 0th axis. If arr is 2-dimensional, the 0th axis are the rows. So for example, if arr looks like this:
In [1]: arr = np.arange(6).reshape(2,3); arr
Out[1]:
array([[0, 1, 2],
[3, 4, 5]])
then np.sum(arr, axis=0) will sum along the columns, thus eliminating the 0th axis (i.e. the rows).
In [2]: np.sum(arr, axis=0)
Out[2]: array([3, 5, 7])
The 3 is the result of 0+3, the 5 equals 1+4, the 7 equals 2+5.
Notice arr had shape (2,3), and after summing, the 0th axis is removed so the result is of shape (3,). The 0th axis had length 2, and each sum is composed of adding those 2 elements. The shape (2,3) "becomes" (3,). You can know the resultant shape in advance! This can help guide your thinking.
To test your understanding, consider np.sum(arr, axis=1). Now the 1-axis is removed. So the resultant shape will be (2,), and element in the result will be the sum of 3 values.
In [3]: np.sum(arr, axis=1)
Out[3]: array([ 3, 12])
The 3 equals 0+1+2, and the 12 equals 3+4+5.
So we see that summing an axis eliminates that axis from the result. This has bearing on np.dot, since the calculation performed by np.dot is a sum of products. Since np.dot performs a summing operation along certain axes, that axis is removed from the result. That is why applying np.dot to arrays of shape (2,) and (2,2) results in an array of shape (2,). The first 2 in both arrays is summed over, eliminating both, leaving only the second 2 in the second array.