The question has changed, so too has the answer:
Strings can't be tested using math.isnan as this expects a float argument. In your countries list, you have floats and strings.
In your case the following should suffice:
cleanedList = [x for x in countries if str(x) != 'nan']
Old answer
In your countries list, the literal 'nan' is a string not the Python float nan which is equivalent to:
float('NaN')
In your case the following should suffice:
cleanedList = [x for x in countries if x != 'nan']
Answer from user764357 on Stack OverflowThe question has changed, so too has the answer:
Strings can't be tested using math.isnan as this expects a float argument. In your countries list, you have floats and strings.
In your case the following should suffice:
cleanedList = [x for x in countries if str(x) != 'nan']
Old answer
In your countries list, the literal 'nan' is a string not the Python float nan which is equivalent to:
float('NaN')
In your case the following should suffice:
cleanedList = [x for x in countries if x != 'nan']
Using your example where...
countries= [nan, 'USA', 'UK', 'France']
Since nan is not equal to nan (nan != nan) and countries[0] = nan, you should observe the following:
countries[0] == countries[0]
False
However,
countries[1] == countries[1]
True
countries[2] == countries[2]
True
countries[3] == countries[3]
True
Therefore, the following should work:
cleanedList = [x for x in countries if x == x]
python - Removing a nan from a list - Stack Overflow
pandas - Remove NaN from lists in python - Stack Overflow
pandas - Removing nan from list - Python - Stack Overflow
python - delete all nan values from list in pandas dataframe - Stack Overflow
How do I remove NaN from a Python list?
How do I remove NaN from a NumPy array?
Should I use pandas for a list with missing values?
I think you need dropna for remove NaNs:
incoms=data['int_income'].dropna().unique().tolist()
print (incoms)
[75000.0, 50000.0, 0.0, 200000.0, 100000.0, 25000.0, 10000.0, 175000.0, 150000.0, 125000.0]
And if all values are integers only:
incoms=data['int_income'].dropna().astype(int).unique().tolist()
print (incoms)
[75000, 50000, 0, 200000, 100000, 25000, 10000, 175000, 150000, 125000]
Or remove NaNs by selecting all non NaN values by numpy.isnan:
a = data['int_income'].unique()
incoms= a[~np.isnan(a)].tolist()
print (incoms)
[75000.0, 50000.0, 0.0, 200000.0, 100000.0, 25000.0, 10000.0, 175000.0, 150000.0, 125000.0]
a = data['int_income'].unique()
incoms= a[~np.isnan(a)].astype(int).tolist()
print (incoms)
[75000, 50000, 0, 200000, 100000, 25000, 10000, 175000, 150000, 125000]
Pure python solution - slowier if big DataFrame:
incoms=[x for x in list(set(data['int_income'])) if pd.notnull(x)]
print (incoms)
[0.0, 100000.0, 200000.0, 25000.0, 125000.0, 50000.0, 10000.0, 150000.0, 175000.0, 75000.0]
incoms=[int(x) for x in list(set(data['int_income'])) if pd.notnull(x)]
print (incoms)
[0, 100000, 200000, 25000, 125000, 50000, 10000, 150000, 175000, 75000]
What you can do is simply get a cleaned list where you don't put the values that, once converted to strings, are 'nan'.
The code would be :
incoms = [incom for incom in incoms if str(incom) != 'nan']
It is not working because you are trying to compare it to string 'nan'.
If excel cell is empty it is returned as NaN value in pandas.
You can use numpy library, to compare it with NaN:
import numpy as np
for l in df_list:
newlist = [x for x in l if x != np.nan]
print(newlist)
EDIT:
If you want to get all values from the dataframe, which are not NaN, you can just do:
df.stack().tolist()
If you want to print values with the loop (as in your example), you can do:
for l in df.columns:
print(list(df[l][df[l].notna()]))
To create nested list with a loop:
main = []
for l in df.T.columns:
new_list = list(df.T[l][df.T[l].notna()])
main.append(new_list)
print(main)
You can always try the approach that is proposed here:
import numpy as np
newlist = [x for x in df_list if np.isnan(x) == False]
print(newlist)
I hope that this will help.
for i in z:
if not math.isnan(i):
ztt.append(i)
ztt = []
for z_i in z:
row = []
for z_ij in z_i:
if math.isnan(z_ij):
row.append(z_ij)
# If you want to replace with, for example, 0:
# else:
# row.append(0)
ztt.append(row)
Alternatively, with nested list comprehensions:
ztt = [[z_ij for z_ij in zi if math.isnan(z_ij)] for z_i in z]
Btw, if you are using NumPy, you probably can just do:
import numpy as np
ztt =[value[~np.isnan(value)] for value in z]
This is one approach using df.apply.
import pandas as pd
import numpy as np
df = pd.DataFrame({"a": [[np.nan, np.nan, np.nan, "a", np.nan], [np.nan, np.nan], ["a", "b"]]})
df["a"] = df["a"].apply(lambda x: [i for i in x if str(i) != "nan"])
print(df)
Output:
a
0 [a]
1 []
2 [a, b]
You can use the fact that np.nan == np.nan evaluates to False:
df = pd.DataFrame([[0, [np.nan, 'a', 'b', 'c']],
[1, [np.nan, np.nan, np.nan]],
[2, [np.nan, 'a', np.nan, np.nan]]],
columns=['index', 'values'])
df['values'] = df['values'].apply(lambda x: [i for i in x if i == i])
print(df)
index values
0 0 [a, b, c]
1 1 []
2 2 [a]
lambda is just an anonymous function. You could also use a named function:
def remove_nan(x):
return [i for i in x if i == i]
df['values'] = df['values'].apply(remove_nan)
Related: Why is NaN not equal to NaN?
The existing dataframe is made up from different length rows, as a result I'm getting nan values called in my function, trying to stop that from happening if anyone can please explain or help with dropping/skipping the nan values, would be much appreciated, cheers.
code:
import webbrowser
from pandas import *
import numpy as np
data= read_csv(r"C:\Users\win95\Desktop\shop-list.csv")
itm = data['meat'].tolist()
url = "https://shop.coles.com.au/a/whiteman-edge/everything/search"
for t in itm:
webbrowser.open(f"{url}/{t}?orderBy=UP&personaliseSort=false&pageNumber=1")Try iterating over the list and building a new list using numpy's isnan method.
import numpy as np
nan = np.nan
l = [[1,2,3,nan,nan,nan],[4,5,nan],[9,8,7,6,5,4,3, nan, nan, nan, nan, nan]]
[[x for x in y if not np.isnan(x)] for y in l]
Without numpy, using math module:
import math
nan = float('nan')
l = [[1,2,3,nan,nan,nan],[4,5,nan],[9,8,7,6,5,4,3,nan,nan,nan,nan,nan]]
l = [[j for j in i if not math.isnan(j)] for i in l]
print(l)
Prints:
[[1, 2, 3], [4, 5], [9, 8, 7, 6, 5, 4, 3]]
My excel spreadsheet:
| model 1 | model 2 | model 3 |
|---|---|---|
| 36325677 | 11508380 | 11508589 |
| 36325678 | 11508451 | 11508572 |
| 36325673 | 11508422 | 11508541 |
| 36105349 | 11508486 | |
| 36325679 | 11508492 | |
| 11508385 | ||
| 11508470 | ||
| 11508416 |
My code:
import pandas as pd
dictionary = pd.read_excel('dictionary.xlsx').to_dict('list')
model_name = input('model name ')
print(dictionary[model_name])Output when input = model 3:
[11508589.0, 11508572.0, 11508541.0, nan, nan, nan, nan, nan]
Desired output:
[11508589, 11508572, 11508541]