Indices of first occurrences
Use np.argmax along that axis (zeroth axis for columns here) on the mask of non-zeros to get the indices of first matches (True values) -
(arr!=0).argmax(axis=0)
Extending to cover generic axis specifier and for cases where no non-zeros are found along that axis for an element, we would have an implementation like so -
def first_nonzero(arr, axis, invalid_val=-1):
mask = arr!=0
return np.where(mask.any(axis=axis), mask.argmax(axis=axis), invalid_val)
Note that since argmax() on all False values returns 0, so if the invalid_val needed is 0, we would have the final output directly with mask.argmax(axis=axis).
Sample runs -
In [296]: arr # Different from given sample for variety
Out[296]:
array([[1, 0, 0],
[1, 1, 0],
[0, 1, 0],
[0, 0, 0]])
In [297]: first_nonzero(arr, axis=0, invalid_val=-1)
Out[297]: array([ 0, 1, -1])
In [298]: first_nonzero(arr, axis=1, invalid_val=-1)
Out[298]: array([ 0, 0, 1, -1])
Extending to cover all comparison operations
To find the first zeros, simply use arr==0 as mask for use in the function. For first ones equal to a certain value val, use arr == val and so on for all cases of comparisons possible here.
Indices of last occurrences
To find the last ones matching a certain comparison criteria, we need to flip along that axis and use the same idea of using argmax and then compensate for the flipping by offsetting from the axis length, as shown below -
def last_nonzero(arr, axis, invalid_val=-1):
mask = arr!=0
val = arr.shape[axis] - np.flip(mask, axis=axis).argmax(axis=axis) - 1
return np.where(mask.any(axis=axis), val, invalid_val)
Sample runs -
In [320]: arr
Out[320]:
array([[1, 0, 0],
[1, 1, 0],
[0, 1, 0],
[0, 0, 0]])
In [321]: last_nonzero(arr, axis=0, invalid_val=-1)
Out[321]: array([ 1, 2, -1])
In [322]: last_nonzero(arr, axis=1, invalid_val=-1)
Out[322]: array([ 0, 1, 1, -1])
Again, all cases of comparisons possible here are covered by using the corresponding comparator to get mask and then using within the listed function.
Indices of first occurrences
Use np.argmax along that axis (zeroth axis for columns here) on the mask of non-zeros to get the indices of first matches (True values) -
(arr!=0).argmax(axis=0)
Extending to cover generic axis specifier and for cases where no non-zeros are found along that axis for an element, we would have an implementation like so -
def first_nonzero(arr, axis, invalid_val=-1):
mask = arr!=0
return np.where(mask.any(axis=axis), mask.argmax(axis=axis), invalid_val)
Note that since argmax() on all False values returns 0, so if the invalid_val needed is 0, we would have the final output directly with mask.argmax(axis=axis).
Sample runs -
In [296]: arr # Different from given sample for variety
Out[296]:
array([[1, 0, 0],
[1, 1, 0],
[0, 1, 0],
[0, 0, 0]])
In [297]: first_nonzero(arr, axis=0, invalid_val=-1)
Out[297]: array([ 0, 1, -1])
In [298]: first_nonzero(arr, axis=1, invalid_val=-1)
Out[298]: array([ 0, 0, 1, -1])
Extending to cover all comparison operations
To find the first zeros, simply use arr==0 as mask for use in the function. For first ones equal to a certain value val, use arr == val and so on for all cases of comparisons possible here.
Indices of last occurrences
To find the last ones matching a certain comparison criteria, we need to flip along that axis and use the same idea of using argmax and then compensate for the flipping by offsetting from the axis length, as shown below -
def last_nonzero(arr, axis, invalid_val=-1):
mask = arr!=0
val = arr.shape[axis] - np.flip(mask, axis=axis).argmax(axis=axis) - 1
return np.where(mask.any(axis=axis), val, invalid_val)
Sample runs -
In [320]: arr
Out[320]:
array([[1, 0, 0],
[1, 1, 0],
[0, 1, 0],
[0, 0, 0]])
In [321]: last_nonzero(arr, axis=0, invalid_val=-1)
Out[321]: array([ 1, 2, -1])
In [322]: last_nonzero(arr, axis=1, invalid_val=-1)
Out[322]: array([ 0, 1, 1, -1])
Again, all cases of comparisons possible here are covered by using the corresponding comparator to get mask and then using within the listed function.
The problem, apparently 2D, can be solved by applying to the each row a function that finds the first non-zero element (exactly as in the question).
arr = np.array([[1,1,0],[1,1,0],[0,0,1],[0,0,0]])
def first_nonzero_index(array):
"""Return the index of the first non-zero element of array. If all elements are zero, return -1."""
fnzi = -1 # first non-zero index
indices = np.flatnonzero(array)
if (len(indices) > 0):
fnzi = indices[0]
return fnzi
np.apply_along_axis(first_nonzero_index, axis=1, arr=arr)
# result
array([ 0, 0, 2, -1])
Explanation
The np.flatnonzero(array) method (as suggested in the comments by Henrik Koberg) returns "indices that are non-zero in the flattened version of array". The function calculates these indices and returns the first (or -1 if all elements are zero).
The apply_along_axis applys a function to 1-D slices along the given axis. Here since the axis is 1, the function is applied to the rows.
If we can assume that all rows of the input array contain at leas one non-zero element, the solution can be written calculated in one line:
np.apply_along_axis(lambda a: np.flatnonzero(a)[0], axis=1, arr=arr)
Possible variations
- If we were interested in the last non-zero element, that could be obrained by changing indices[0] into indices[-1] in the function.
- To get the first non-zero by row, we would change axes=1 into axis=0 in np.apply_along_axis
ORIGINAL ANSWER
Here is an alternative using numpy.argwhere which returns the index of the non zero elements of an array:
array = np.array([0,0,0,1,2,3,0,0])
nonzero_indx = np.argwhere(array).squeeze()
start, end = (nonzero_indx[0], nonzero_indx[-1])
print(array[start], array[end])
gives:
1 3
It is reasonably fast to use np.where:
>>> a
array([[0, 0, 0, 1, 1, 1, 1],
[0, 0, 0, 1, 1, 1, 1],
[0, 0, 0, 0, 1, 1, 1],
[0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 0, 0, 0]])
>>> np.where(a>0)
(array([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 3, 4, 5]), array([3, 4, 5, 6, 3, 4, 5, 6, 4, 5, 6, 6, 6, 6]))
That delivers tuples with to coordinates of the values greater than 0.
You can also use np.where to test each sub array:
def first_true1(a):
""" return a dict of row: index with value in row > 0 """
di={}
for i in range(len(a)):
idx=np.where(a[i]>0)
try:
di[i]=idx[0][0]
except IndexError:
di[i]=None
return di
Prints:
{0: 3, 1: 3, 2: 4, 3: 6, 4: 6, 5: 6, 6: None}
ie, row 0: index 3>0; row 4: index 4>0; row 6: no index greater than 0
As you suspect, argmax may be faster:
def first_true2():
di={}
for i in range(len(a)):
idx=np.argmax(a[i])
if idx>0:
di[i]=idx
else:
di[i]=None
return di
# same dict is returned...
If you can deal with the logic of not having a None for rows of all naughts, this is faster still:
def first_true3():
di={}
for i, j in zip(*np.where(a>0)):
if i in di:
continue
else:
di[i]=j
return di
And here is a version that uses axis in argmax (as suggested in your comments):
def first_true4():
di={}
for i, ele in enumerate(np.argmax(a,axis=1)):
if ele==0 and a[i][0]==0:
di[i]=None
else:
di[i]=ele
return di
For speed comparisons (on your example array), I get this:
rate/sec usec/pass first_true1 first_true2 first_true3 first_true4
first_true1 23,818 41.986 -- -34.5% -63.1% -70.0%
first_true2 36,377 27.490 52.7% -- -43.6% -54.1%
first_true3 64,528 15.497 170.9% 77.4% -- -18.6%
first_true4 79,287 12.612 232.9% 118.0% 22.9% --
If I scale that to a 2000 X 2000 np array, here is what I get:
rate/sec usec/pass first_true3 first_true1 first_true2 first_true4
first_true3 3 354380.107 -- -0.3% -74.7% -87.8%
first_true1 3 353327.084 0.3% -- -74.6% -87.7%
first_true2 11 89754.200 294.8% 293.7% -- -51.7%
first_true4 23 43306.494 718.3% 715.9% 107.3% --
argmax() use C level loop, it's much faster than Python loop, so I think even you write a smart algorithm in Python, it's hard to beat argmax(), You can use Cython to speedup:
@cython.boundscheck(False)
@cython.wraparound(False)
def find(int[:,:] a):
cdef int h = a.shape[0]
cdef int w = a.shape[1]
cdef int i, j
cdef int idx = 0
cdef list r = []
for i in range(h):
for j in range(idx, w):
if a[i, j] == 1:
idx = j
r.append(idx)
break
else:
r.append(-1)
return r
On my PC for 2000x2000 matrix, it's 100us vs 3ms.
Use next with enumerate:
>>> myList = [0.0 , 0.0, 0.0, 2.0, 2.0]
>>> next((i for i, x in enumerate(myList) if x), None) # x!= 0 for strict match
3
Use filter
Python 2:
myList = [0.0, 0.0, 0.0, 2.0, 2.0]
myList2 = [0.0, 0.0]
myList.index(filter(lambda x: x!=0, myList)[0]) # 3
myList2.index(filter(lambda x: x!=0, myList2)[0]) # IndexError
Python 3: (Thanks for Matthias's comment):
myList.index(next(filter(lambda x: x!=0, myList))) # 3
myList2.index(next(filter(lambda x: x!=0, myList2))) # StopIteration
# from Ashwini Chaudhary's answer
next((i for i, x in enumerate(myList) if x), None) # 3
next((i for i, x in enumerate(myList2) if x), None) # None
You have to handle special case.