There are a few alternatives. The below assumes you are iterating over a 1d NumPy array.
Iterate with range
for j in range(theta.shape[0]): # or range(len(theta))
some_function(j, theta[j], theta)
Note this is the only of the 3 solutions which will work with numba. This is noteworthy since iterating over a NumPy array explicitly is usually only efficient when combined with numba or another means of pre-compilation.
Iterate with enumerate
for idx, j in enumerate(theta):
some_function(idx, j, theta)
The most efficient of the 3 solutions for 1d arrays. See benchmarking below.
Iterate with np.ndenumerate
for idx, j in np.ndenumerate(theta):
some_function(idx[0], j, theta)
Notice the additional indexing step in idx[0]. This is necessary since the index (like shape) of a 1d NumPy array is given as a singleton tuple. For a 1d array, np.ndenumerate is inefficient; its benefits only show for multi-dimensional arrays.
Performance benchmarking
# Python 3.7, NumPy 1.14.3
np.random.seed(0)
arr = np.random.random(10**6)
def enumerater(arr):
for index, value in enumerate(arr):
index, value
pass
def ranger(arr):
for index in range(len(arr)):
index, arr[index]
pass
def ndenumerater(arr):
for index, value in np.ndenumerate(arr):
index[0], value
pass
%timeit enumerater(arr) # 131 ms
%timeit ranger(arr) # 171 ms
%timeit ndenumerater(arr) # 579 ms
Answer from jpp on Stack OverflowThere are a few alternatives. The below assumes you are iterating over a 1d NumPy array.
Iterate with range
for j in range(theta.shape[0]): # or range(len(theta))
some_function(j, theta[j], theta)
Note this is the only of the 3 solutions which will work with numba. This is noteworthy since iterating over a NumPy array explicitly is usually only efficient when combined with numba or another means of pre-compilation.
Iterate with enumerate
for idx, j in enumerate(theta):
some_function(idx, j, theta)
The most efficient of the 3 solutions for 1d arrays. See benchmarking below.
Iterate with np.ndenumerate
for idx, j in np.ndenumerate(theta):
some_function(idx[0], j, theta)
Notice the additional indexing step in idx[0]. This is necessary since the index (like shape) of a 1d NumPy array is given as a singleton tuple. For a 1d array, np.ndenumerate is inefficient; its benefits only show for multi-dimensional arrays.
Performance benchmarking
# Python 3.7, NumPy 1.14.3
np.random.seed(0)
arr = np.random.random(10**6)
def enumerater(arr):
for index, value in enumerate(arr):
index, value
pass
def ranger(arr):
for index in range(len(arr)):
index, arr[index]
pass
def ndenumerater(arr):
for index, value in np.ndenumerate(arr):
index[0], value
pass
%timeit enumerater(arr) # 131 ms
%timeit ranger(arr) # 171 ms
%timeit ndenumerater(arr) # 579 ms
You can use numpy.ndenumerate for example
import numpy as np
test_array = np.arange(2, 3, 0.1)
for index, value in np.ndenumerate(test_array):
print(index[0], value)
For more information refer to https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndenumerate.html
python - Iterating over a numpy array - Stack Overflow
How to Iterating NumPy Array?
Iterating through a numpy array
Why iterate over an array using the index?
As I've become more familiar with the NumPy and general scientific array processing ecosystem, it's become clearer to me that iteration is usually outright wrong due to how slow it is in comparison, and writing to use a vectorized operation is best whenever possible. Though the style is not as obvious/Pythonic at first, I've (anecdotally) gained ridiculous speedups with vectorized operations; more than 1000x in a case of swapping out a form like some row iteration .apply(lambda)
@MSeifert's answer much better provides this and will be significantly more performant on a dataset of any real size
More general Answer by @cs95 covering and comparing alternatives to iteration in Pandas
Original Answer
You can iterate through the values in your array with numpy.ndenumerate to get the indices of the values in your array.
Using the documentation above:
A = np.array([[1,2,3],[4,5,6],[7,8,9]])
for index, values in np.ndenumerate(A):
print(index, values) # operate here
You can do it using np.ndenumerate but generally you don't need to iterate over an array.
You can simply create a meshgrid (or open grid) to get all indices at once and you can then process them (vectorized) much faster.
For example
>>> x, y = np.mgrid[slice(A.shape[0]), slice(A.shape[1])]
>>> x
array([[0, 0, 0],
[1, 1, 1],
[2, 2, 2]])
>>> y
array([[0, 1, 2],
[0, 1, 2],
[0, 1, 2]])
and these can be processed like any other array. So if your function that needs the indices can be vectorized you shouldn't do the manual loop!
For example to calculate the lattice distance for each point to a point say (2, 3):
>>> abs(x - 2) + abs(y - 3)
array([[5, 4, 3],
[4, 3, 2],
[3, 2, 1]])
For distances an ogrid would be faster. Just replace np.mgrid with np.ogrid:
>>> x, y = np.ogrid[slice(A.shape[0]), slice(A.shape[1])]
>>> np.hypot(x - 2, y - 3) # cartesian distance this time! :-)
array([[ 3.60555128, 2.82842712, 2.23606798],
[ 3.16227766, 2.23606798, 1.41421356],
[ 3. , 2. , 1. ]])
I think you're looking for the ndenumerate.
>>> import numpy as np
>>> a = np.array([[1,2],[3,4],[5,6]])
>>> for (i, j), value in np.ndenumerate(a):
... print(i, j)
...
0 0
0 1
1 0
1 1
2 0
2 1
Regarding the performance. It is a bit slower than a list comprehension.
X = np.zeros((100, 100, 100))
%timeit list([((i, j, k), X[i, j, k]) for i in range(X.shape[0]) for j in range(X.shape[1]) for k in range(X.shape[2])])
1 loop, best of 3: 376 ms per loop
%timeit list(np.ndenumerate(X))
1 loop, best of 3: 570 ms per loop
If you are worried about the performance you could optimise a bit further by looking at the implementation of ndenumerate, which does 2 things, converting to an array and looping. If you know you have an array, you can call the .coords attribute of the flat iterator.
a = X.flat
%timeit list([(a.coords, x) for x in a.flat])
1 loop, best of 3: 305 ms per loop
If you only need the indices, you could try numpy.ndindex:
>>> a = numpy.arange(9).reshape(3, 3)
>>> [(x, y) for x, y in numpy.ndindex(a.shape)]
[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2)]
I know this is a dumb question, but I've scoured Google for the documentation, Stack Overflow, etc. but I can't figure out how to do this. I'm really used to C so I might be approaching this wrong.
Basically, I have a multidimensional array; specifically, it's n x 2. Another way of looking at it is that it's an array of tuples.
So suppose there are the points (1, 9), (2, 8), and (3, 7).
Now, my assumption is that [[1, 2, 3], [9, 8, 7]] is an nx2 array, and that [[1,9], [2, 8], [3, 7]] is a 2xn array, correct?
So my question is, in each case (given an nx2 and a 2xn array), how would I iterate through each ordered pair?