Obviously some of your lines don't have valid float data, specifically some line have text id which can't be converted to float.
When you try it in interactive prompt you are trying only first line, so best way is to print the line where you are getting this error and you will know the wrong line e.g.
#!/usr/bin/python
import os,sys
from scipy import stats
import numpy as np
f=open('data2.txt', 'r').readlines()
N=len(f)-1
for i in range(0,N):
w=f[i].split()
l1=w[1:8]
l2=w[8:15]
try:
list1=[float(x) for x in l1]
list2=[float(x) for x in l2]
except ValueError,e:
print "error",e,"on line",i
result=stats.ttest_ind(list1,list2)
print result[1]
Answer from Anurag Uniyal on Stack OverflowObviously some of your lines don't have valid float data, specifically some line have text id which can't be converted to float.
When you try it in interactive prompt you are trying only first line, so best way is to print the line where you are getting this error and you will know the wrong line e.g.
#!/usr/bin/python
import os,sys
from scipy import stats
import numpy as np
f=open('data2.txt', 'r').readlines()
N=len(f)-1
for i in range(0,N):
w=f[i].split()
l1=w[1:8]
l2=w[8:15]
try:
list1=[float(x) for x in l1]
list2=[float(x) for x in l2]
except ValueError,e:
print "error",e,"on line",i
result=stats.ttest_ind(list1,list2)
print result[1]
My error was very simple: the text file containing the data had some space (so not visible) character on the last line.
As an output of grep, I had 45 instead of just 45.
Videos
I want to do some math on a dataframe but (I think) can't get one column/series into the necessary format. The column contains strings; some are '.123' while others are '0'. When I attempt the math on the column of strings by converting everything to an integer like so:
dfteam1['cloff'] = dfteam1.cloff.astype(int)
I get the following error
ValueError: invalid literal for int() with base 10: '.123'
I think it's b/c .123 isn't an integer but a float, so I change the code like so:
dfteam1['cloff'] = dfteam1.cloff.astype(float)
now I get the following error
ValueError: could not convert string to float:
I think it's b/c 0 isn't a float but an integer? Do I need to change all the 0 values to 0.00 or am I completely off base? All feedback is welcome.
I have a list of strings and I would to get the average of the list. so I need to convert those strings in the list into float but I keep getting this:
ValueError: could not convert string to float: '"104.8"\n'
How do I solve it? Thanks
THIS IS MY TASK:
Read the number list from file and convert it into an array. 2. Create a counter and set it to zero. 3. Create a for loop that traverses the number array. 4. Inside the for loop, check if the current number is less than the next number. 5. If the current number is less than the next number, decrement the counter. 6. After the for loop finishes traversing, return the counter.
THIS IS MY CODE:
with open("numbers.txt") as file:
numbers = [
float(line)
for line in file
]
count = 0
for idx, number in enumerate(numbers[:-1]):
if number < numbers[idx+1]:
count += 1
print(count)
AND THIS IS MY ERROR:
Exception has occurred: ValueError
could not convert string to float: '\n'
i really have no idea how to fix it
Though not the best solution, I found some success by converting it into pandas dataframe and working along.
code snippet
# convert X into dataframe
X_pd = pd.DataFrame(data=X)
# replace all instances of URC with 0
X_replace = X_pd.replace('�',0, regex=True)
# convert it back to numpy array
X_np = X_replace.values
# set the object type as float
X_fa = X_np.astype(float)
input
array([['85', '0', '0', '1980', '0', '0'],
['233', '54', '27', '-1', '0', '0'],
['���', '�', '�����', '�', '��', '���']], dtype='<U5')
output
array([[ 8.50e+01, 0.00e+00, 0.00e+00, 1.98e+03, 0.00e+00, 0.00e+00],
[ 2.33e+02, 5.40e+01, 2.70e+01, -1.00e+00, 0.00e+00, 0.00e+00],
[ 0.00e+00, 0.00e+00, 0.00e+00, 0.00e+00, 0.00e+00, 0.00e+00]])
Let's try to use pandas dataframe and convert strings into numeric classes
from sklearn import preprocessing
def convert(data):
number = preprocessing.LabelEncoder()
data['column_name'] = number.fit_transform(data['column_name'])
data=data.fillna(-999) # fill holes with default value
return data
call the above convert() function like, test = convert(test)