This is a little faster (and looks nicer)
np.argmax(aa>5)
Since argmax will stop at the first True ("In case of multiple occurrences of the maximum values, the indices corresponding to the first occurrence are returned.") and doesn't save another list.
In [2]: N = 10000
In [3]: aa = np.arange(-N,N)
In [4]: timeit np.argmax(aa>N/2)
100000 loops, best of 3: 52.3 us per loop
In [5]: timeit np.where(aa>N/2)[0][0]
10000 loops, best of 3: 141 us per loop
In [6]: timeit np.nonzero(aa>N/2)[0][0]
10000 loops, best of 3: 142 us per loop
Answer from askewchan on Stack OverflowThis is a little faster (and looks nicer)
np.argmax(aa>5)
Since argmax will stop at the first True ("In case of multiple occurrences of the maximum values, the indices corresponding to the first occurrence are returned.") and doesn't save another list.
In [2]: N = 10000
In [3]: aa = np.arange(-N,N)
In [4]: timeit np.argmax(aa>N/2)
100000 loops, best of 3: 52.3 us per loop
In [5]: timeit np.where(aa>N/2)[0][0]
10000 loops, best of 3: 141 us per loop
In [6]: timeit np.nonzero(aa>N/2)[0][0]
10000 loops, best of 3: 142 us per loop
given the sorted content of your array, there is an even faster method: searchsorted.
import time
N = 10000
aa = np.arange(-N,N)
%timeit np.searchsorted(aa, N/2)+1
%timeit np.argmax(aa>N/2)
%timeit np.where(aa>N/2)[0][0]
%timeit np.nonzero(aa>N/2)[0][0]
# Output
100000 loops, best of 3: 5.97 µs per loop
10000 loops, best of 3: 46.3 µs per loop
10000 loops, best of 3: 154 µs per loop
10000 loops, best of 3: 154 µs per loop
Yes, given an array, array, and a value, item to search for, you can use np.where as:
itemindex = numpy.where(array == item)
The result is a tuple with first all the row indices, then all the column indices.
For example, if an array is two dimensions and it contained your item at two locations then
array[itemindex[0][0]][itemindex[1][0]]
would be equal to your item and so would be:
array[itemindex[0][1]][itemindex[1][1]]
If you need the index of the first occurrence of only one value, you can use nonzero (or where, which amounts to the same thing in this case):
>>> t = array([1, 1, 1, 2, 2, 3, 8, 3, 8, 8])
>>> nonzero(t == 8)
(array([6, 8, 9]),)
>>> nonzero(t == 8)[0][0]
6
If you need the first index of each of many values, you could obviously do the same as above repeatedly, but there is a trick that may be faster. The following finds the indices of the first element of each subsequence:
>>> nonzero(r_[1, diff(t)[:-1]])
(array([0, 3, 5, 6, 7, 8]),)
Notice that it finds the beginning of both subsequence of 3s and both subsequences of 8s:
[1, 1, 1, 2, 2, 3, 8, 3, 8, 8]
So it's slightly different than finding the first occurrence of each value. In your program, you may be able to work with a sorted version of t to get what you want:
>>> st = sorted(t)
>>> nonzero(r_[1, diff(st)[:-1]])
(array([0, 3, 5, 7]),)
Although it is way too late for you, but for future reference: Using numba (1) is the easiest way until numpy implements it. If you use anaconda python distribution it should already be installed. The code will be compiled so it will be fast.
@jit(nopython=True)
def find_first(item, vec):
"""return the index of the first occurence of item in vec"""
for i in xrange(len(vec)):
if item == vec[i]:
return i
return -1
and then:
>>> a = array([1,7,8,32])
>>> find_first(8,a)
2
I've made a benchmark for several methods:
argwherenonzeroas in the question.tostring()as in @Rob Reilink's answer- python loop
- Fortran loop
The Python and Fortran code are available. I skipped the unpromising ones like converting to a list.
The results on log scale. X-axis is the position of the needle (it takes longer to find if it's further down the array); last value is a needle that's not in the array. Y-axis is the time to find it.

The array had 1 million elements and tests were run 100 times. Results still fluctuate a bit, but the qualitative trend is clear: Python and f2py quit at the first element so they scale differently. Python gets too slow if the needle is not in the first 1%, whereas f2py is fast (but you need to compile it).
To summarize, f2py is the fastest solution, especially if the needle appears fairly early.
It's not built in which is annoying, but it's really just 2 minutes of work. Add this to a file called search.f90:
subroutine find_first(needle, haystack, haystack_length, index)
implicit none
integer, intent(in) :: needle
integer, intent(in) :: haystack_length
integer, intent(in), dimension(haystack_length) :: haystack
!f2py intent(inplace) haystack
integer, intent(out) :: index
integer :: k
index = -1
do k = 1, haystack_length
if (haystack(k)==needle) then
index = k - 1
exit
endif
enddo
end
If you're looking for something other than integer, just change the type. Then compile using:
f2py -c -m search search.f90
after which you can do (from Python):
import search
print(search.find_first.__doc__)
a = search.find_first(your_int_needle, your_int_array)
Benchmarking results
Well so I have decided to compare the proposed solutions in a typical case scenario. The three approaches are:
@timeit
def gener_sol(a):
# My OP proposal
return next(idx for idx in zip((i for i, x in enumerate(a) if x), (a.size-i-1 for i, x in enumerate(a[::-1]) if x)))
@timeit
def nonzero_sol(a):
# kalzso's proposal
return np.nonzero(a)[0][[0, -1]]
@timeit
def argmax_sol(a):
# jan992's proposal
return np.argmax(a), a.size - np.argmax(a[::-1]) - 1
NOTE: @timeit is a custom wrapper I have to measure functions execution time. Beyond the question whether it is perfect or not, it is the same for all of them so it should be a fair comparison.
And here goes the piece of code to execute them in a loop:
for rep in range(10000):
N = 2**20 # Array size
depth = N // 10 # max depth from the edges for first and last True
arr = np.random.randint(0, 2, N).astype(bool)
# First and last True values are randomly located at both ends
first = random.randint(0, depth)
arr[:first] = False
arr[first] = True
last = random.randint(0, depth)
arr[-last:] = False
arr[-last] = True
arg_res = argmax_sol(arr)
non_res = nonzero_sol(arr)
gen_res = gener_sol(arr)
# Ensure they all return the same
assert arg_res == tuple(non_res) == gen_res
And the results are the following:
N=2^20 (10000 runs)
argmax_sol....: 10000 runs, 6.38 s (638.01 μs/run)
nonzero_sol...: 10000 runs, 13.124 s (1.312 ms/run)
gener_sol.....: 10000 runs, 42.695 s (4.27 ms/run)
N=2^25 (100 runs)
argmax_sol....: 100 runs, 1.891 s (18.908 ms/run)
nonzero_sol...: 100 runs, 4.125 s (41.25 ms/run)
gener_sol.....: 100 runs, 13.799 s (137.99 ms/run)
So, in this scenario the argmax solution is the clear winner, and the generator the clear loser.
However, I noted that the depth at which the first and last indices are located (up to N/10 in this case) can have a significant impact. A reasonable value in my case is depth=10000 (or even less). Therefore, if I repeat the experiment I obtain:
N=2^20 (10000 runs) depth = 10000
gener_sol.....: 100 runs, 39.967 ms (399.671 μs/run)
argmax_sol....: 100 runs, 58.851 ms (588.505 μs/run)
nonzero_sol...: 100 runs, 138.077 ms (1.381 ms/run)
As you see, in this case the generator yields better results. And the improvement is even larger if depth can be bound to even a smaller number.
See for instance:
N=2^20 (10000 runs) depth=1000
gener_sol.....: 10000 runs, 0.947 s (94.753 μs/run)
argmax_sol....: 10000 runs, 6.082 s (608.196 μs/run)
nonzero_sol...: 10000 runs, 14.282 s (1.428 ms/run)
N=2^25 (1000 runs) depth=1000
gener_sol.....: 1000 runs, 0.053 s (53.08 μs/run)
argmax_sol....: 1000 runs, 18.84 s (18.84 ms/run)
nonzero_sol...: 1000 runs, 44.058 s (44.058 ms/run)
wow, that's a dramatic improvement! ~6 and ~350 times faster than numpy's argmax for N=2^20 and 2^25, respectively. So one should not underestimate the power of for loops/generators under appropriate circumstances.
Finally, and to leave a good word for kalszo's nonzero solution, I do have found that there is a case in which it outperforms the others, and that is when the array is False everywhere except in two locations (the ones to find).
Therefore, changing the line:
# arr = np.random.randint(0, 2, N).astype(bool)
arr = np.zeros(N).astype(bool)
N=2^20 depth=N//10, only two True values
nonzero_sol...: 10000 runs, 1.612 s (161.186 μs/run)
argmax_sol....: 10000 runs, 6.482 s (648.178 μs/run)
gener_sol.....: 10000 runs, 44.625 s (4.463 ms/run)
and with shallower depths:
N=2^20 (10000 runs) depth=10000, only two True values
nonzero_sol...: 10000 runs, 1.571 s (157.093 μs/run)
gener_sol.....: 10000 runs, 4.208 s (420.791 μs/run)
argmax_sol....: 10000 runs, 6.235 s (623.512 μs/run)
N=2^25 (1000 runs) depth=10000, only two True values
gener_sol.....: 1000 runs, 0.433 s (433.125 μs/run)
nonzero_sol...: 1000 runs, 5.427 s (5.427 ms/run)
argmax_sol....: 1000 runs, 19.128 s (19.128 ms/run)
N=2^20 (10000 runs) depth=1000, only two True values
gener_sol.....: 10000 runs, 0.962 s (96.286 μs/run)
nonzero_sol...: 10000 runs, 1.637 s (163.687 μs/run)
argmax_sol....: 10000 runs, 6.522 s (652.228 μs/run)
again, the small depth relative to the array size is a key factor in favor of the generator expression.
In conclusion, the right solution depends very much on the scenario. For my particular case in which:
- Arrays are large (2^20-2^25)
- The array has multiple True values at unknown locations
- The first and last True values are relatively close to the edges
the generator expression has the best performance with up to one or two orders of magnitude.
Use numpy's nonzero function, which returns the indices of all non-zero elements in the input array:
a = np.array([False, False, False, True, ..., True, False, False])
# Find the indices of the non-zero elements
indices = np.nonzero(a)
# The first and last indices will be the first and last True values
first = indices[0][0]
last = indices[0][-1]
This one should work with no for loops, for 1D or 2D:
def firstByRow(a, f = lambda x: x >= 10):
b = (np.cumsum(f(a), axis = -1) == 1).T
b[1:] = b[1:] * np.equal(b[1:], np.diff((f(a)).astype(int), axis = -1).T)
return b.T
Not sure if it would be faster than a slightly loopier code though, as it does both cumsum and diff
EDIT:
You can also do this, which is probably faster (leveraging that np.unique(return_index = True) picks the first occurrence):
def firstByAxis(a, f = lambda x: x >= 10, axis = 0):
c = np.where(f(a))
i = np.unique(c[axis], return_index = True)[1]
b = np.zeros_like(a)
b[tuple(np.take(c, i, axis = -1))] = 1
return b
You can try the following:
>>> import numpy as np
>>> a_2d = np.array([[0,4,10],[0,11,10]])
>>> r, c = np.where( a_2d >= 10 )
>>> mask = r+c == (r+c).min()
>>> highMask = np.zeros(np.shape(a_2d))
>>> highMask[r[mask], c[mask]] = 1
>>> highMask
array([[ 0., 0., 1.],
[ 0., 1., 0.]])
There is no such thing as the 'first' one in a 2D array. In a 2D array, the minimum indices will form a line on the 2D axis, the both of which will have minimum indices values. For a 3D matrix, this will be a surface, etc ..
Example of such a line would be:
0 0 0 0 0 1
0 0 0 0 1 0
0 0 0 1 0 0
0 0 1 0 0 0
0 1 0 0 0 0
1 0 0 0 0 0
All of which are equidistant from the [0,0] location ...