I think both the fastest and most concise way to do this is to use NumPy's built-in Fancy indexing. If you have an ndarray named arr, you can replace all elements >255 with a value x as follows:
arr[arr > 255] = x
I ran this on my machine with a 500 x 500 random matrix, replacing all values >0.5 with 5, and it took an average of 7.59ms.
In [1]: import numpy as np
In [2]: A = np.random.rand(500, 500)
In [3]: timeit A[A > 0.5] = 5
100 loops, best of 3: 7.59 ms per loop
Answer from mdml on Stack OverflowI think both the fastest and most concise way to do this is to use NumPy's built-in Fancy indexing. If you have an ndarray named arr, you can replace all elements >255 with a value x as follows:
arr[arr > 255] = x
I ran this on my machine with a 500 x 500 random matrix, replacing all values >0.5 with 5, and it took an average of 7.59ms.
In [1]: import numpy as np
In [2]: A = np.random.rand(500, 500)
In [3]: timeit A[A > 0.5] = 5
100 loops, best of 3: 7.59 ms per loop
If you want a new array result containing a copy of arr whenever arr < 255, and 255 otherwise:
result = np.minimum(arr, 255)
More generally, for a lower and/or upper bound:
result = np.clip(arr, 0, 255)
If you just want to access the values over 255, or something more complicated, @mtitan8's answer is more general, but np.clip and np.minimum (or np.maximum) are nicer and much faster for your case:
In [292]: timeit np.minimum(a, 255)
100000 loops, best of 3: 19.6 µs per loop
In [293]: %%timeit
.....: c = np.copy(a)
.....: c[a>255] = 255
.....:
10000 loops, best of 3: 86.6 µs per loop
If you want to do it in-place (i.e., modify arr instead of creating result) you can use the out parameter of np.minimum:
np.minimum(arr, 255, out=arr)
or
np.clip(arr, 0, 255, arr)
(the out= name is optional since the arguments in the same order as the function's definition.)
For in-place modification, the boolean indexing speeds up a lot (without having to make and then modify the copy separately), but is still not as fast as minimum:
In [328]: %%timeit
.....: a = np.random.randint(0, 300, (100,100))
.....: np.minimum(a, 255, a)
.....:
100000 loops, best of 3: 303 µs per loop
In [329]: %%timeit
.....: a = np.random.randint(0, 300, (100,100))
.....: a[a>255] = 255
.....:
100000 loops, best of 3: 356 µs per loop
For comparison, if you wanted to restrict your values with a minimum as well as a maximum, without clip you would have to do this twice, with something like
np.minimum(a, 255, a)
np.maximum(a, 0, a)
or,
a[a>255] = 255
a[a<0] = 0
python - How to replace values in a numpy array? - Data Science Stack Exchange
python - Fast replacement of values in a numpy array - Stack Overflow
Replace elements in numpy array
python - How to replace values in numpy array at the same time - Stack Overflow
I believe there's even more efficient method, but for now, try
from numpy import copy
newArray = copy(theArray)
for k, v in d.iteritems(): newArray[theArray==k] = v
Microbenchmark and test for correctness:
#!/usr/bin/env python2.7
from numpy import copy, random, arange
random.seed(0)
data = random.randint(30, size=10**5)
d = {4: 0, 9: 5, 14: 10, 19: 15, 20: 0, 21: 1, 22: 2, 23: 3, 24: 0}
dk = d.keys()
dv = d.values()
def f1(a, d):
b = copy(a)
for k, v in d.iteritems():
b[a==k] = v
return b
def f2(a, d):
for i in xrange(len(a)):
a[i] = d.get(a[i], a[i])
return a
def f3(a, dk, dv):
mp = arange(0, max(a)+1)
mp[dk] = dv
return mp[a]
a = copy(data)
res = f2(a, d)
assert (f1(data, d) == res).all()
assert (f3(data, dk, dv) == res).all()
Result:
$ python2.7 -m timeit -s 'from w import f1,f3,data,d,dk,dv' 'f1(data,d)'
100 loops, best of 3: 6.15 msec per loop
$ python2.7 -m timeit -s 'from w import f1,f3,data,d,dk,dv' 'f3(data,dk,dv)'
100 loops, best of 3: 19.6 msec per loop
Assuming the values are between 0 and some maximum integer, one could implement a fast replace by using the numpy-array as int->int dict, like below
mp = numpy.arange(0,max(data)+1)
mp[replace.keys()] = replace.values()
data = mp[data]
where first
data = [ 0 1 6 5 1 2 7 6 2 3 8 7 3 4 9 8 5 6 11 10 6 7 12 11 7
8 13 12 8 9 14 13 10 11 16 15 11 12 17 16 12 13 18 17 13 14 19 18 15 16
21 20 16 17 22 21 17 18 23 22 18 19 24 23]
and replacing with
replace = {4: 0, 9: 5, 14: 10, 19: 15, 20: 0, 21: 1, 22: 2, 23: 3, 24: 0}
we obtain
data = [ 0 1 6 5 1 2 7 6 2 3 8 7 3 0 5 8 5 6 11 10 6 7 12 11 7
8 13 12 8 5 10 13 10 11 16 15 11 12 17 16 12 13 18 17 13 10 15 18 15 16
1 0 16 17 2 1 17 18 3 2 18 15 0 3]
I have a function that converts angular values (degrees) into cardinal directions (N, NNW, ect). Here is what I have so far:
import numpy as np
def to_cardinal(values: np.ndarray):
CARD_DIR = [
"N",
"NNE",
"NE",
"ENE",
"E",
"ESE",
"SE",
"SSE",
"S",
"SSW",
"SW",
"WSW",
"W",
"WNW",
"NW",
"NNW",
"N",
]
# If values over 360, scale down
angle = np.remainder(vals_array, 360.0)
# Divide by 22.5 and convert to int
# On another note, not sure if np.around is necessary?
index_arr = np.around((angle / 22.5)).astype(np.int16, copy=False)
return index_arr
I return index_arr here because I'm not sure how to continue. Here is a sample input/output:
input = np.array((10, 350, 190, 100, 270, 30)) output = to_cardinal(input) print(output) >>> (0, 16, 8, 4, 12, 1)
These values are the indices of the CARD_DIR list that the degree values corresponds to. In words, I need to construct an array (or replace from index_arr) where each element is now the string from CARD_DIR instead of the integer that represents the index. I can imagine doing this in a loop, but how could this be vectorized?
In psuedo-code:
new_array = np.empty(size = index_arr.size)
for i, value in enumerate(index_arr):
new_array[i] = CARD_DIR[value]Also, not sure if np.around is necessary when casting a float to an int? Does the int type conversion round correctly already?
Edit: Figured it out, numpy is magical. See comment below! Still interested in type-casting from floats to ints, though.
It's better to use np.select if you've multiple conditions:
a = np.array([7, 1, 2, 0, 2, 3, 4, 0, 5])
a = np.select([a == 7, a == 2], [2, 3], a)
OUTPUT:
[2 1 3 0 3 3 4 0 5]
Numpy provide the comparison to a scalar with the standard == operator, such that arr == v return a boolean array, often called a mask. The masked selection arr[arr == v] takes the subset of arr where the condition is met, so this snippet should work.
import numpy as np
arr = np.array([7, 1, 2, 0, 2, 3, 4, 0, 5])
arr[arr == 7] = 2
arr
array([2, 1, 2, 0, 2, 3, 4, 0, 5])
It sounds like you want:
subset = array[:, 2:]
subset[subset < 0] = 0
or as a rather unreadable one-liner:
array[:, 2:][array[:, 2:] < 0] = 0
As a more complete example:
import numpy as np
array = np.random.normal(0, 1, (10, 5))
print array
# Note that "subset" is a view, so modifying it modifies "array"
subset = array[:, 2:]
subset[subset < 0] = 0
print array
You would need to clip subsets of the arrays.
something like this:
a[2:].clip(0, None)
You could do this a couple of ways. One would be in a for loop:
for list in list_of_lists:
list[2:] = list[2:].clip(0, None)
Or, using [:, 2:] which references your list of lists (:), and then all sublists of that (2:).
The result is basically what Joe Kingston has suggested:
list[:, 2:] = list[:, 2:].clip(0, None)
There is a numpy function for this, numpy.full or numpy.full_like:
>>> x = np.array(([1,2,2], [1,4,3]))
>>> np.full(x.shape, 10)
array([[10, 10, 10],
[10, 10, 10]])
# OR,
>>> np.full_like(x, 10)
array([[10, 10, 10],
[10, 10, 10]])
If you want to iterate you can either use itertools.product:
>>> from itertools import product
>>> def Iter_Replace(x):
indices = product(*map(range, x.shape))
for index in indices:
x[tuple(index)] = 10
return x
>>> x = np.array([[1,2,2], [1,4,3]])
>>> Iter_Replace(x)
array([[10, 10, 10],
[10, 10, 10]])
Or, use np.nditer
>>> x = np.array([[1,2,2], [1,4,3]])
>>> for index in np.ndindex(x.shape):
x[index] = 10
>>> x
array([[10, 10, 10],
[10, 10, 10]])
You have two errors. There are missing parenthesis in the first line of main:
x = np.array(([1,2,2], [1,4,3]))
And you must replace range(x) by range(len(x)) in the Iter_Replace function.
Well, I suppose what you need is
a[a==2] = 10 #replace all 2's with 10's
Instead of replacing the values one by one, it is possible to remap the entire array like this:
import numpy as np
a = np.array([1,2,2,1]).reshape(2,2)
# palette must be given in sorted order
palette = [1, 2]
# key gives the new values you wish palette to be mapped to.
key = np.array([0, 10])
index = np.digitize(a.ravel(), palette, right=True)
print(key[index].reshape(a.shape))
yields
[[ 0 10]
[10 0]]
Credit for the above idea goes to @JoshAdel. It is significantly faster than my original answer:
import numpy as np
import random
palette = np.arange(8)
key = palette**2
a = np.array([random.choice(palette) for i in range(514*504)]).reshape(514,504)
def using_unique():
palette, index = np.unique(a, return_inverse=True)
return key[index].reshape(a.shape)
def using_digitize():
index = np.digitize(a.ravel(), palette, right=True)
return key[index].reshape(a.shape)
if __name__ == '__main__':
assert np.allclose(using_unique(), using_digitize())
I benchmarked the two versions this way:
In [107]: %timeit using_unique()
10 loops, best of 3: 35.6 ms per loop
In [112]: %timeit using_digitize()
100 loops, best of 3: 5.14 ms per loop
You have two arrays inside an array. The below code should work:
array = [["id1", "1.0"],["id2", "0.0"]]
for item in array:
if item[1] == "0.0":
item[1] = "ClassA"
else:
item[1] = "ClassB"
you are missing ,s in the definition of your array. your array is the same as this: [["id11.0"], ["id20.0"]] (strings just get concatenated). if your arrays are numpy arrays then this is the way they are represented (printed). but that does not work as input...
starting from your code you could to this:
array = [["id1", "1.0"], ["id2", "0.0"]]
for i, (id_, number) in enumerate(array):
if number == "0.0":
array[i] = [id_, "ClassA"]
else:
array[i] = [id_, "ClassB"]
or, more elegant, use a list-comprehension:
array = [[id_, "ClassA"] if number == "0.0" else [id_, "ClassB"]
for id_, number in array ]