import numpy
a = numpy.arange(0.5, 1.5, 0.1, dtype=numpy.float64)
print(a)
a = numpy.round(a.astype(numpy.float64), 1)
print (a)
print (a.tolist())
Output:
[0.5 0.6 0.7 0.8 0.9 1. 1.1 1.2 1.3 1.4]
[0.5 0.6 0.7 0.8 0.9 1. 1.1 1.2 1.3 1.4]
[0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.4]
You can use numpy.round() with astype() method to round your float, getting something consistant upon tolist() conversion.
import numpy
a = numpy.arange(0.5, 1.5, 0.1, dtype=numpy.float64)
print(a)
a = numpy.round(a.astype(numpy.float64), 1)
print (a)
print (a.tolist())
Output:
[0.5 0.6 0.7 0.8 0.9 1. 1.1 1.2 1.3 1.4]
[0.5 0.6 0.7 0.8 0.9 1. 1.1 1.2 1.3 1.4]
[0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.4]
You can use numpy.round() with astype() method to round your float, getting something consistant upon tolist() conversion.
By the conversion via .tolist() you're not gaining or loosing any precision. You're just converting to another data type which chooses to represent itself differently. You seem to be thinking that by the conversion it turns the 0.8 into 0.7999999999999999, but the situation is, it has never been 0.8. This is due to limited floating point precision. You can verify yourself:
>>> import numpy
>>> a = numpy.arange(0.5, 1.5, 0.1, dtype=numpy.float64)
>>> a
array([0.5, 0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3, 1.4])
>>> a[3]
0.7999999999999999
The situation is that that every type can decide how its instances are represented via __repr__ or __str__. In this specific case np.ndarray decides to show a rounded version of its elements for more clarity. This can be controlled via numpy.set_printoptions. This function only affects how arrays are displayed:
>>> numpy.set_printoptions(floatmode='unique')
>>> a
array([0.5 , 0.6 , 0.7 ,
0.7999999999999999, 0.8999999999999999, 0.9999999999999999,
1.0999999999999999, 1.1999999999999997, 1.2999999999999998,
1.4 ])
A note on np.arange from the docs:
When using a non-integer step, such as 0.1, the results will often not be consistent. It is better to use
numpy.linspacefor these cases.
Here are the results for linspace:
>>> np.linspace(0.5, 1.5, 11)
array([0.5 , 0.6 , 0.7 ,
0.8 , 0.9 , 1. ,
1.1 , 1.2000000000000002, 1.3 ,
1.4 , 1.5 ])
python - changing numpy array to float - Stack Overflow
python - How to convert an array of strings to an array of floats in numpy? - Stack Overflow
Convert NumPy array to Python list - Stack Overflow
Convert list or numpy array of single element to float in python - Stack Overflow
Generally your idea of trying to apply astype to each column is fine.
In [590]: X[:,0].astype(int)
Out[590]: array([1, 2, 3, 4, 5])
But you have to collect the results in a separate list. You can't just put them back in X. That list can then be concatenated.
In [601]: numlist=[]; obj_ind=[]
In [602]: for ind in range(X.shape[1]):
.....: try:
.....: x = X[:,ind].astype(np.float32)
.....: numlist.append(x)
.....: except:
.....: obj_ind.append(ind)
In [603]: numlist
Out[603]: [array([ 3., 4., 5., 6., 7.], dtype=float32)]
In [604]: np.column_stack(numlist)
Out[604]:
array([[ 3.],
[ 4.],
[ 5.],
[ 6.],
[ 7.]], dtype=float32)
In [606]: obj_ind
Out[606]: [1]
X is a numpy array with dtype object:
In [582]: X
Out[582]:
array([[1, 'A'],
[2, 'A'],
[3, 'C'],
[4, 'D'],
[5, 'B']], dtype=object)
You could use the same conversion logic to create a structured array with a mix of int and object fields.
In [616]: ytype=[]
In [617]: for ind in range(X.shape[1]):
try:
x = X[:,ind].astype(np.float32)
ytype.append('i4')
except:
ytype.append('O')
In [618]: ytype
Out[618]: ['i4', 'O']
In [620]: Y=np.zeros(X.shape[0],dtype=','.join(ytype))
In [621]: for i in range(X.shape[1]):
Y[Y.dtype.names[i]] = X[:,i]
In [622]: Y
Out[622]:
array([(3, 'A'), (4, 'A'), (5, 'C'), (6, 'D'), (7, 'B')],
dtype=[('f0', '<i4'), ('f1', 'O')])
Y['f0'] gives the the numeric field.
I think this might help
def func(x):
a = None
try:
a = x.astype(float)
except:
# x.name represents the current index value
# which is column name in this case
obj.append(x.name)
a = x
return a
obj = []
new_df = df.apply(func, axis=0)
This will keep the object columns as such which you can use later.
Note: While using pandas.DataFrame avoid using iteration using loop as this much slower than performing the same operation using apply.
Well, if you're reading the data in as a list, just do np.array(map(float, list_of_strings)) (or equivalently, use a list comprehension). (In Python 3, you'll need to call list on the map return value if you use map, since map returns an iterator now.)
However, if it's already a numpy array of strings, there's a better way. Use astype().
import numpy as np
x = np.array(['1.1', '2.2', '3.3'])
y = x.astype(np.float)
Another option might be numpy.asarray:
import numpy as np
a = ["1.1", "2.2", "3.2"]
b = np.asarray(a, dtype=float)
print(a, type(a), type(a[0]))
print(b, type(b), type(b[0]))
resulting in:
['1.1', '2.2', '3.2'] <class 'list'> <class 'str'>
[1.1 2.2 3.2] <class 'numpy.ndarray'> <class 'numpy.float64'>
Use tolist():
>>> import numpy as np
>>> np.array([[1,2,3],[4,5,6]]).tolist()
[[1, 2, 3], [4, 5, 6]]
Note that this converts the values from whatever numpy type they may have (e.g. np.int32 or np.float32) to the "nearest compatible Python type" (in a list). If you want to preserve the numpy data types, you could call list() on your array instead, and you'll end up with a list of numpy scalars. (Thanks to Mr_and_Mrs_D for pointing that out in a comment.)
c = np.array([[1,2,3],[4,5,6]])
list(c.flatten())
You may want to use the ndarray.item method, as in a.item(). This is also equivalent to (the now deprecated) np.asscalar(a). This has the benefit of working in situations with views and superfluous axes, while the above solutions will currently break. For example,
>>> a = np.asarray(1).view()
>>> a.item() # correct
1
>>> a[0] # breaks
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: too many indices for array
>>> a = np.asarray([[2]])
>>> a.item() # correct
2
>>> a[0] # bad result
array([2])
This also has the benefit of throwing an exception if the array is not actually a scalar, while the a[0] approach will silently proceed (which may lead to bugs sneaking through undetected).
>>> a = np.asarray([1, 2])
>>> a[0] # silently proceeds
1
>>> a.item() # detects incorrect size
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: can only convert an array of size 1 to a Python scalar
Just access the first item of the list/array, using the index access and the index 0:
>>> list_ = [4]
>>> list_[0]
4
>>> array_ = np.array([4])
>>> array_[0]
4
This will be an int since that was what you inserted in the first place. If you need it to be a float for some reason, you can call float() on it then:
>>> float(list_[0])
4.0
You can use numpy's .flatten() method followed by a list and map function.
>>> arr = np.array([[0.],[1.],[2.],[3.],[4.]])
>>> arr = arr.flatten()
>>> arr
array([0., 1., 2., 3., 4.])
>>> arr = list(map(lambda x:f"{x:.0f}", arr))
>>> arr
['0', '1', '2', '3', '4']
First try to understand what the array is:
In [73]: arr = np.array([[ 0.], [ 1.], [ 2.], [ 3.], [ 4.]])
In [74]: arr
Out[74]:
array([[0.],
[1.],
[2.],
[3.],
[4.]])
In [75]: arr.shape, arr.dtype
Out[75]: ((5, 1), dtype('float64'))
There is a good, fast method to convert an array to list - that is close to the original in use of [] and type - floats.
In [76]: arr.tolist()
Out[76]: [[0.0], [1.0], [2.0], [3.0], [4.0]]
If you don't want those inner [], dimension, lets ravel/flatten array. That easier done with the array than the nested list:
In [77]: arr.ravel()
Out[77]: array([0., 1., 2., 3., 4.])
We could also convert the floats to strings:
In [78]: arr.ravel().astype(str)
Out[78]: array(['0.0', '1.0', '2.0', '3.0', '4.0'], dtype='<U32')
In [79]: arr.ravel().astype(str).tolist()
Out[79]: ['0.0', '1.0', '2.0', '3.0', '4.0']
But you want integers - so lets do that conversion first:
In [80]: arr.ravel().astype(int).astype(str).tolist()
Out[80]: ['0', '1', '2', '3', '4']
The other answer show how to do these conversion with lists.
First please do not use 'list'.Therefore, I changed it to my_list.
You may make an array using 'np.array()' and You can specify your data type using the optional 'dtype' flag.
>>> import numpy as np
>>> my_list = [1.5, 2.5, 3.5]
>>> my_array = np.array(my_list,dtype=" ")
And always you can check your data type by using:
>>> my_array.dtype
dtype('float64')
What about setting the dtype of your GaussMatrix to np.float64?
GaussMatrix = np.array(GaussMatrix, dtype=np.float64)
Let's say I have 5 lists, list1, list2, list3, list4, list5.
The first 3 are lists of strings e.g. while the last 2 are lists of floats e.g.
list1 = ['1','2','3'...] list2 = list3 = list1 list4 = [1.0,2.0,3.0...] list5 = list4
All lists are the same length.
I'd like to turn them into a numpy array, while preserving their original datatypes. I say
intermediateList = zip(list1,list2,list3,list4,list5)
which gives me a list (length=5) of lists, with the original data types preserved. Perfect.
However, when I try to convert to numpy.array
myArray=asarray(intermediateList)
numPy converts everything into numpy_string type, which is very irritating. I can now no longer perform numerical operations on list4 and list5 without re-converting the values to float.
(as a sidenote, if I were to try to fix this by saying
list5=list5.astype(float)
it will convert to float in the console, however when I call list5 again it is back to numpy_string! anyone know why?)
Any suggestions?
The tolist() method should do what you want. If you have a numpy array, just call tolist():
In [17]: a
Out[17]:
array([ 0. , 0.14285714, 0.28571429, 0.42857143, 0.57142857,
0.71428571, 0.85714286, 1. , 1.14285714, 1.28571429,
1.42857143, 1.57142857, 1.71428571, 1.85714286, 2. ])
In [18]: a.dtype
Out[18]: dtype('float64')
In [19]: b = a.tolist()
In [20]: b
Out[20]:
[0.0,
0.14285714285714285,
0.2857142857142857,
0.42857142857142855,
0.5714285714285714,
0.7142857142857142,
0.8571428571428571,
1.0,
1.1428571428571428,
1.2857142857142856,
1.4285714285714284,
1.5714285714285714,
1.7142857142857142,
1.857142857142857,
2.0]
In [21]: type(b)
Out[21]: list
In [22]: type(b[0])
Out[22]: float
If, in fact, you really have python list of numpy.float64 objects, then @Alexander's answer is great, or you could convert the list to an array and then use the tolist() method. E.g.
In [46]: c
Out[46]:
[0.0,
0.33333333333333331,
0.66666666666666663,
1.0,
1.3333333333333333,
1.6666666666666665,
2.0]
In [47]: type(c)
Out[47]: list
In [48]: type(c[0])
Out[48]: numpy.float64
@Alexander's suggestion, a list comprehension:
In [49]: [float(v) for v in c]
Out[49]:
[0.0,
0.3333333333333333,
0.6666666666666666,
1.0,
1.3333333333333333,
1.6666666666666665,
2.0]
Or, convert to an array and then use the tolist() method.
In [50]: np.array(c).tolist()
Out[50]:
[0.0,
0.3333333333333333,
0.6666666666666666,
1.0,
1.3333333333333333,
1.6666666666666665,
2.0]
If you are concerned with the speed, here's a comparison. The input, x, is a python list of numpy.float64 objects:
In [8]: type(x)
Out[8]: list
In [9]: len(x)
Out[9]: 1000
In [10]: type(x[0])
Out[10]: numpy.float64
Timing for the list comprehension:
In [11]: %timeit list1 = [float(v) for v in x]
10000 loops, best of 3: 109 ยตs per loop
Timing for conversion to numpy array and then tolist():
In [12]: %timeit list2 = np.array(x).tolist()
10000 loops, best of 3: 70.5 ยตs per loop
So it is faster to convert the list to an array and then call tolist().
You could use a list comprehension:
floats = [float(np_float) for np_float in np_float_list]