First of all, 1.79769313486e+308 is not the same as +inf. The former is the largest number which can be expressed with a 64-bit float, the latter is a special float.
If you just have very large numbers in your array, then:
A[A > 1e308] = 0
is sufficient. Thet'll replace oll elements above 1e308 with 0.
It is also possible to operate with the inf's. For example:
>>> fmax = np.finfo(np.float64).max
>>> pinf = float('+inf')
>>> ninf = float('-inf')
>>> fnan = float('nan')
>>> print fmax, pinf, ninf, fnan
1.79769313486e+308 inf -inf nan
So, these are completely different things. You may compare some of them:
>>> pinf > fmax
True
>>> ninf < 0.0
True
>>> pinf == pinf
True
>>> pinf == ninf
False
This looks good! However, nan acts differently:
>>> fnan > 0
False
>>> fnan < 0
False
>>> fnan == 0
False
>>> fnan < pinf
False
>>> fnan == fnan
False
You may use positive and negativi infinities with Numpy ndarraywithout any problems. This will work:
A[A == pinf] = 0.0
But if you have nans in the array, you'll get some complaints:
>>> np.array([fnan, pinf, ninf]) < 0
RuntimeWarning: invalid value encountered in less
[False, False, True]
So, it works but complains => do not use. The same without the nan:
>>> np.array([0.0, pinf, ninf]) < 0
[False, False, True]
If you want to do something with the nans (should you have them), use numpy.isnan:
A[np.isnan(A)] = 0.0
will change all nans into zeros.
And -- this you did not ask -- here is one to surprise your friends (*):
>>> [float('-0.0'), 0.0] * 3
[-0.0, 0.0, -0.0, 0.0, -0.0, 0.0]
Yep, float64 (and float32) have even a separate -0.0. In calculations it acts as an ordinary zero, though:
>>> float('-0.0') == 0.0
True
(*) Depending on the kind of people you call friends.
Answer from DrV on Stack OverflowFirst of all, 1.79769313486e+308 is not the same as +inf. The former is the largest number which can be expressed with a 64-bit float, the latter is a special float.
If you just have very large numbers in your array, then:
A[A > 1e308] = 0
is sufficient. Thet'll replace oll elements above 1e308 with 0.
It is also possible to operate with the inf's. For example:
>>> fmax = np.finfo(np.float64).max
>>> pinf = float('+inf')
>>> ninf = float('-inf')
>>> fnan = float('nan')
>>> print fmax, pinf, ninf, fnan
1.79769313486e+308 inf -inf nan
So, these are completely different things. You may compare some of them:
>>> pinf > fmax
True
>>> ninf < 0.0
True
>>> pinf == pinf
True
>>> pinf == ninf
False
This looks good! However, nan acts differently:
>>> fnan > 0
False
>>> fnan < 0
False
>>> fnan == 0
False
>>> fnan < pinf
False
>>> fnan == fnan
False
You may use positive and negativi infinities with Numpy ndarraywithout any problems. This will work:
A[A == pinf] = 0.0
But if you have nans in the array, you'll get some complaints:
>>> np.array([fnan, pinf, ninf]) < 0
RuntimeWarning: invalid value encountered in less
[False, False, True]
So, it works but complains => do not use. The same without the nan:
>>> np.array([0.0, pinf, ninf]) < 0
[False, False, True]
If you want to do something with the nans (should you have them), use numpy.isnan:
A[np.isnan(A)] = 0.0
will change all nans into zeros.
And -- this you did not ask -- here is one to surprise your friends (*):
>>> [float('-0.0'), 0.0] * 3
[-0.0, 0.0, -0.0, 0.0, -0.0, 0.0]
Yep, float64 (and float32) have even a separate -0.0. In calculations it acts as an ordinary zero, though:
>>> float('-0.0') == 0.0
True
(*) Depending on the kind of people you call friends.
To remove the very high values:
>>> a = numpy.array([1, 2, 1.8E308, 1.8E308, 42])
>>> a[a < 1E308] # use whatever threshold you like
array([ 1., 2., 42.])
To set them 0:
>>> a = numpy.array([1, 2, 1.8E308, 1.8E308, 42])
>>> a[a >= 1E308] = 0
>>> a
array([ 1., 2., 0., 0., 42.])
Videos
You can just replace NaN and infinite values with the following mask:
output[~np.isfinite(output)] = 0
>>> output
array([[1. , 0.5 , 1. , 1. , 0. ,
1. ],
[1. , 1. , 0.5 , 1. , 0.46524064,
1. ],
[1. , 1. , 1. , 0. , 1. ,
1. ]])
There is a special function just for that:
numpy.nan_to_num(x_arr, copy=False, nan=0.0, posinf=0.0, neginf=0.0)
The easiest way to do this is to use numpy.ma.masked_invalid():
a = numpy.log(numpy.arange(15))
a.sum()
# -inf
numpy.ma.masked_invalid(a).sum()
# 25.19122118273868
Use masked arrays:
>>> a = numpy.array([2, 0, 1.5, -3])
>>> b = numpy.ma.log(a)
>>> b
masked_array(data = [0.69314718056 -- 0.405465108108 --],
mask = [False True False True],
fill_value = 1e+20)
>>> b.sum()
1.0986122886681096
You have to save the operation in your dataframe. One way is to use the parameter inplace=True:
df_fund['dly_retn'].replace(np.inf, 0, inplace=True)
na_fund['dly_retn'].replace(np.inf, 0, inplace=True)
you can do it for an entire dataframe as follow:
new_df.replace(np.inf, 0, inplace=True)
Use numpy.delete(), which returns a new array with sub-arrays along an axis deleted.
numpy.delete(a, index)
For your specific question:
import numpy as np
a = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9])
index = [2, 3, 6]
new_a = np.delete(a, index)
print(new_a)
# Output: [1, 2, 5, 6, 8, 9]
Note that numpy.delete() returns a new array since array scalars are immutable, similar to strings in Python, so each time a change is made to it, a new object is created. I.e., to quote the delete() docs:
"A copy of arr with the elements specified by obj removed. Note that delete does not occur in-place..."
If the code I post has output, it is the result of running the code.
Use np.setdiff1d:
import numpy as np
>>> a = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> b = np.array([3,4,7])
>>> c = np.setdiff1d(a,b)
>>> c
array([1, 2, 5, 6, 8, 9])
To remove NaN values from a NumPy array x:
x = x[~numpy.isnan(x)]
Explanation
The inner function numpy.isnan returns a boolean/logical array which has the value True everywhere that x is not-a-number. Since we want the opposite, we use the logical-not operator ~ to get an array with Trues everywhere that x is a valid number.
Lastly, we use this logical array to index into the original array x, in order to retrieve just the non-NaN values.
filter(lambda v: v==v, x)
works both for lists and numpy array since v!=v only for NaN