I think both the fastest and most concise way to do this is to use NumPy's built-in Fancy indexing. If you have an ndarray named arr, you can replace all elements >255 with a value x as follows:
Use with AIarr[arr > 255] = x
I ran this on my machine with a 500 x 500 random matrix, replacing all values >0.5 with 5, and it took an average of 7.59ms.
Use with AIIn [1]: import numpy as np
In [2]: A = np.random.rand(500, 500)
In [3]: timeit A[A > 0.5] = 5
100 loops, best of 3: 7.59 ms per loop
Answer from mdml on Stack OverflowI think both the fastest and most concise way to do this is to use NumPy's built-in Fancy indexing. If you have an ndarray named arr, you can replace all elements >255 with a value x as follows:
Use with AIarr[arr > 255] = x
I ran this on my machine with a 500 x 500 random matrix, replacing all values >0.5 with 5, and it took an average of 7.59ms.
Use with AIIn [1]: import numpy as np
In [2]: A = np.random.rand(500, 500)
In [3]: timeit A[A > 0.5] = 5
100 loops, best of 3: 7.59 ms per loop
If you want a new array result containing a copy of arr whenever arr < 255, and 255 otherwise:
Use with AIresult = np.minimum(arr, 255)
More generally, for a lower and/or upper bound:
Use with AIresult = np.clip(arr, 0, 255)
If you just want to access the values over 255, or something more complicated, @mtitan8's answer is more general, but np.clip and np.minimum (or np.maximum) are nicer and much faster for your case:
Use with AIIn [292]: timeit np.minimum(a, 255)
100000 loops, best of 3: 19.6 µs per loop
In [293]: %%timeit
.....: c = np.copy(a)
.....: c[a>255] = 255
.....:
10000 loops, best of 3: 86.6 µs per loop
If you want to do it in-place (i.e., modify arr instead of creating result) you can use the out parameter of np.minimum:
Use with AInp.minimum(arr, 255, out=arr)
or
Use with AInp.clip(arr, 0, 255, arr)
(the out= name is optional since the arguments in the same order as the function's definition.)
For in-place modification, the boolean indexing speeds up a lot (without having to make and then modify the copy separately), but is still not as fast as minimum:
Use with AIIn [328]: %%timeit
.....: a = np.random.randint(0, 300, (100,100))
.....: np.minimum(a, 255, a)
.....:
100000 loops, best of 3: 303 µs per loop
In [329]: %%timeit
.....: a = np.random.randint(0, 300, (100,100))
.....: a[a>255] = 255
.....:
100000 loops, best of 3: 356 µs per loop
For comparison, if you wanted to restrict your values with a minimum as well as a maximum, without clip you would have to do this twice, with something like
Use with AInp.minimum(a, 255, a)
np.maximum(a, 0, a)
or,
Use with AIa[a>255] = 255
a[a<0] = 0
python - How to replace values in a numpy array? - Data Science Stack Exchange
Replace elements in numpy array
python - Fast replacement of values in a numpy array - Stack Overflow
python - How to replace values in numpy array at the same time - Stack Overflow
I have a function that converts angular values (degrees) into cardinal directions (N, NNW, ect). Here is what I have so far:
import numpy as np
def to_cardinal(values: np.ndarray):
CARD_DIR = [
"N",
"NNE",
"NE",
"ENE",
"E",
"ESE",
"SE",
"SSE",
"S",
"SSW",
"SW",
"WSW",
"W",
"WNW",
"NW",
"NNW",
"N",
]
# If values over 360, scale down
angle = np.remainder(vals_array, 360.0)
# Divide by 22.5 and convert to int
# On another note, not sure if np.around is necessary?
index_arr = np.around((angle / 22.5)).astype(np.int16, copy=False)
return index_arr
I return index_arr here because I'm not sure how to continue. Here is a sample input/output:
input = np.array((10, 350, 190, 100, 270, 30)) output = to_cardinal(input) print(output) >>> (0, 16, 8, 4, 12, 1)
These values are the indices of the CARD_DIR list that the degree values corresponds to. In words, I need to construct an array (or replace from index_arr) where each element is now the string from CARD_DIR instead of the integer that represents the index. I can imagine doing this in a loop, but how could this be vectorized?
In psuedo-code:
new_array = np.empty(size = index_arr.size)
for i, value in enumerate(index_arr):
new_array[i] = CARD_DIR[value]Also, not sure if np.around is necessary when casting a float to an int? Does the int type conversion round correctly already?
Edit: Figured it out, numpy is magical. See comment below! Still interested in type-casting from floats to ints, though.
I believe there's even more efficient method, but for now, try
from numpy import copy
newArray = copy(theArray)
for k, v in d.iteritems(): newArray[theArray==k] = v
Microbenchmark and test for correctness:
#!/usr/bin/env python2.7
from numpy import copy, random, arange
random.seed(0)
data = random.randint(30, size=10**5)
d = {4: 0, 9: 5, 14: 10, 19: 15, 20: 0, 21: 1, 22: 2, 23: 3, 24: 0}
dk = d.keys()
dv = d.values()
def f1(a, d):
b = copy(a)
for k, v in d.iteritems():
b[a==k] = v
return b
def f2(a, d):
for i in xrange(len(a)):
a[i] = d.get(a[i], a[i])
return a
def f3(a, dk, dv):
mp = arange(0, max(a)+1)
mp[dk] = dv
return mp[a]
a = copy(data)
res = f2(a, d)
assert (f1(data, d) == res).all()
assert (f3(data, dk, dv) == res).all()
Result:
$ python2.7 -m timeit -s 'from w import f1,f3,data,d,dk,dv' 'f1(data,d)'
100 loops, best of 3: 6.15 msec per loop
$ python2.7 -m timeit -s 'from w import f1,f3,data,d,dk,dv' 'f3(data,dk,dv)'
100 loops, best of 3: 19.6 msec per loop
Assuming the values are between 0 and some maximum integer, one could implement a fast replace by using the numpy-array as int->int dict, like below
mp = numpy.arange(0,max(data)+1)
mp[replace.keys()] = replace.values()
data = mp[data]
where first
data = [ 0 1 6 5 1 2 7 6 2 3 8 7 3 4 9 8 5 6 11 10 6 7 12 11 7
8 13 12 8 9 14 13 10 11 16 15 11 12 17 16 12 13 18 17 13 14 19 18 15 16
21 20 16 17 22 21 17 18 23 22 18 19 24 23]
and replacing with
replace = {4: 0, 9: 5, 14: 10, 19: 15, 20: 0, 21: 1, 22: 2, 23: 3, 24: 0}
we obtain
data = [ 0 1 6 5 1 2 7 6 2 3 8 7 3 0 5 8 5 6 11 10 6 7 12 11 7
8 13 12 8 5 10 13 10 11 16 15 11 12 17 16 12 13 18 17 13 10 15 18 15 16
1 0 16 17 2 1 17 18 3 2 18 15 0 3]
It's better to use np.select if you've multiple conditions:
a = np.array([7, 1, 2, 0, 2, 3, 4, 0, 5])
a = np.select([a == 7, a == 2], [2, 3], a)
OUTPUT:
[2 1 3 0 3 3 4 0 5]
Numpy provide the comparison to a scalar with the standard == operator, such that arr == v return a boolean array, often called a mask. The masked selection arr[arr == v] takes the subset of arr where the condition is met, so this snippet should work.
import numpy as np
arr = np.array([7, 1, 2, 0, 2, 3, 4, 0, 5])
arr[arr == 7] = 2
arr
array([2, 1, 2, 0, 2, 3, 4, 0, 5])
Well, I suppose what you need is
a[a==2] = 10 #replace all 2's with 10's
Instead of replacing the values one by one, it is possible to remap the entire array like this:
import numpy as np
a = np.array([1,2,2,1]).reshape(2,2)
# palette must be given in sorted order
palette = [1, 2]
# key gives the new values you wish palette to be mapped to.
key = np.array([0, 10])
index = np.digitize(a.ravel(), palette, right=True)
print(key[index].reshape(a.shape))
yields
[[ 0 10]
[10 0]]
Credit for the above idea goes to @JoshAdel. It is significantly faster than my original answer:
import numpy as np
import random
palette = np.arange(8)
key = palette**2
a = np.array([random.choice(palette) for i in range(514*504)]).reshape(514,504)
def using_unique():
palette, index = np.unique(a, return_inverse=True)
return key[index].reshape(a.shape)
def using_digitize():
index = np.digitize(a.ravel(), palette, right=True)
return key[index].reshape(a.shape)
if __name__ == '__main__':
assert np.allclose(using_unique(), using_digitize())
I benchmarked the two versions this way:
In [107]: %timeit using_unique()
10 loops, best of 3: 35.6 ms per loop
In [112]: %timeit using_digitize()
100 loops, best of 3: 5.14 ms per loop
You could use the direct indices like so:
a[:, :, 3:] = b
Non-overwriting method:
a[:,:,-2:]fetches the zeros at the end; usea[:,:,:3].According to the documentation,
np.hstack(x)is equivalent tonp.concatenate(x, axis=1). Since you want to join the matrices on their innermost rows, you should useaxis=2.
Code:
>>> np.concatenate((a[:,:,:3], b), axis=2)
array([[[1, 2, 3, 1, 2],
[4, 5, 6, 2, 3],
[7, 8, 9, 3, 4]],
[[1, 3, 5, 4, 1],
[2, 4, 6, 5, 2],
[1, 1, 1, 6, 3]]])