The problem is not that you're failing to set a float64 dtype. The error message says:
Input contains NaN, infinity or a value too large for dtype('float32').
So try checking for those conditions first:
assert not np.any(np.isnan(x_arr))
assert np.all(np.isfinite(x_arr))
assert np.all(x_arr <= finfo('float32').max)
assert np.all(x_arr >= finfo('float32').min)
Answer from John Zwinck on Stack OverflowThe problem is not that you're failing to set a float64 dtype. The error message says:
Input contains NaN, infinity or a value too large for dtype('float32').
So try checking for those conditions first:
assert not np.any(np.isnan(x_arr))
assert np.all(np.isfinite(x_arr))
assert np.all(x_arr <= finfo('float32').max)
assert np.all(x_arr >= finfo('float32').min)
I got here because of the question in the title which I still feel remains unanswered.
To convert float32 to float64 in a numpy.ndarray object:
array32 = np.ndarray(shape=(2,2), dtype=np.float32, order='F')
print("Type of an object of 'array32': " + str(type(array32[0][0])))
# Convert to float64
array64 = array32.astype(np.float64)
print("Type of an object of 'array64': " + str(type(array64[0][0])))
# Convert back to float32
array32again = array64.astype(np.float32)
print("Type of an object of 'array32again': " + str(type(array32again[0][0])))
Will give you:
Type of an object of 'array32': <class 'numpy.float32'>
Type of an object of 'array64': <class 'numpy.float64'>
Type of an object of 'array32again': <class 'numpy.float32'>
It might not be a direct answer to the question but I think for practical purposes it should be adressed as a data science question.
First, a value of 1.4e258 feel rather suspicious as it is hard to picture where it could have a meaningfull physical signification. Such extreme values might screw your metrics and your model badly. The question should be about it being an outlier or not. The answer depends on your data, their meaning.
If it is an oultier (as in extreme value), the right approach might be to remove the instance entirely. This will probably boost the performance of your trained model on the rest of the instances. The downside is that the model won't perform well on that instance or similar extreme values. Practically, this requires that you make everyone using the model aware of its limitations on those extreme values.
If it is not an outlier you should consider modifying it to make it more informative, both to human and machine. That could mean using a more meaningful scale like a logarithmic one : it would be easier to manipulate for humans and might avoid significant calculation problems. Another approach would be to use some sort of renormalisation. For exemple if all your values are between 1e250 and 1e260 you could divide them by 1e255. If the variable that take such values depends on another one, you might renormalize by this value or one of its power, like if it is a volume, you might want to renormalise by a size variable to the 3rd power. That might help, both to avoid sklearn calculation problems and make more meaningful models.
It's a numerical precision problem. There is no current solution. The numbers are huge
I can replicate using this:
import numpy as np
from sklearn.ensemble import AdaBoostRegressor
X = np.repeat([1.4619664882428694e+258],100)
X = X.reshape(10,10)
y = np.ones((10,1))
model = AdaBoostRegressor()
model.fit(X,y)
ValueError: Input contains NaN, infinity or a value too large for dtype('float32').
np.all(np.isfinite(X))
True
I opened a request here: https://github.com/scikit-learn/scikit-learn/issues/15628
Let's see if I can address some of the confusion I'm seeing in the comments.
Make an array:
In [609]: x=np.arange(5)
In [610]: x
Out[610]: array([0, 1, 2, 3, 4])
In [611]: x.dtype
Out[611]: dtype('int32')
The default for arange is to make an int32.
astype is an array method; it can used on any array:
In [612]: x.astype(np.float32)
Out[612]: array([ 0., 1., 2., 3., 4.], dtype=float32)
arange also takes a dtype parameter
In [614]: np.arange(5, dtype=np.float32)
Out[614]: array([ 0., 1., 2., 3., 4.], dtype=float32)
whether it created the int array first and converted it, or made the float32 directly isn't any concern to me. This is a basic operation, done in compiled code.
I can also give it a float stop value, in which case it will give me a float array - the default float type.
In [615]: np.arange(5.0)
Out[615]: array([ 0., 1., 2., 3., 4.])
In [616]: _.dtype
Out[616]: dtype('float64')
zeros is similar; the default dtype is float64, but with a parameter I can change that. Since its primary task with to allocate memory, and it doesn't have to do any calculation, I'm sure it creates the desired dtype right away, without further conversion. But again, this is compiled code, and I shouldn't have to worry about what it is doing under the covers.
In [618]: np.zeros(5)
Out[618]: array([ 0., 0., 0., 0., 0.])
In [619]: _.dtype
Out[619]: dtype('float64')
In [620]: np.zeros(5,dtype=np.float32)
Out[620]: array([ 0., 0., 0., 0., 0.], dtype=float32)
randn involves a lot of calculation, and evidently it is compiled to work with the default float type. It does not take a dtype. But since the result is an array, it can be cast with astype.
In [623]: np.random.randn(3)
Out[623]: array([-0.64520949, 0.21554705, 2.16722514])
In [624]: _.dtype
Out[624]: dtype('float64')
In [625]: __.astype(np.float32)
Out[625]: array([-0.64520949, 0.21554704, 2.16722512], dtype=float32)
Let me stress that astype is a method of an array. It takes the values of the array and produces a new array with the desire dtype. It does not act retroactively (or in-place) on the array itself, or on the function that created that array.
The effect of astype is often (always?) the same as a dtype parameter, but the sequence of actions is different.
In https://stackoverflow.com/a/39625960/901925 I describe a sparse matrix creator that takes a dtype parameter, and implements it with an astype method call at the end.
When you do calculations such as dot or *, it tries to match the output dtype with inputs. In the case of mixed types it goes with the higher precision alternative.
In [642]: np.arange(5,dtype=np.float32)*np.arange(5,dtype=np.float64)
Out[642]: array([ 0., 1., 4., 9., 16.])
In [643]: _.dtype
Out[643]: dtype('float64')
In [644]: np.arange(5,dtype=np.float32)*np.arange(5,dtype=np.float32)
Out[644]: array([ 0., 1., 4., 9., 16.], dtype=float32)
There are casting rules. One way to look those up is with can_cast function:
In [649]: np.can_cast(np.float64,np.float32)
Out[649]: False
In [650]: np.can_cast(np.float32,np.float64)
Out[650]: True
It is possible in some calculations that it will cast the 32 to 64, do the calculation, and then cast back to 32. The purpose would be to avoid rounding errors. But I don't know how you find that out from the documentation or tests.
arr1 = np.array([25, 56, 12, 85, 34, 75])
arr2 = np.array([42, 3, 86, 32, 856, 46])
arr1.astype(np.complex)
print (arr1)
print(type(arr1[0]))
print(arr1.astype(np.complex))
arr2 = np.array(arr2,dtype='complex')
print(arr2)
print(type(arr2[0]))
OUTPUT for above
[25 56 12 85 34 75]
<class 'numpy.int64'>
[25.+0.j 56.+0.j 12.+0.j 85.+0.j 34.+0.j 75.+0.j]
[ 42.+0.j 3.+0.j 86.+0.j 32.+0.j 856.+0.j 46.+0.j]
<class 'numpy.complex128'>
It can be seen that astype changes the type temporally as we do in normal type casting but where as the generic method changes the type permanently