df = df.replace({np.nan: None})
Note: For pandas versions <1.4, this changes the dtype of all affected columns to object.
To avoid that, use this syntax instead:
df = df.replace(np.nan, None)
Note 2: If you don't want to import numpy, np.nan can be replaced with native float('nan'):
df = df.replace({float('nan'): None})
Credit goes to this guy here on this Github issue, Killian Huyghe's comment and Matt's answer.
Answer from EliadL on Stack Overflowdf = df.replace({np.nan: None})
Note: For pandas versions <1.4, this changes the dtype of all affected columns to object.
To avoid that, use this syntax instead:
df = df.replace(np.nan, None)
Note 2: If you don't want to import numpy, np.nan can be replaced with native float('nan'):
df = df.replace({float('nan'): None})
Credit goes to this guy here on this Github issue, Killian Huyghe's comment and Matt's answer.
For pandas > 1.3.0 see this answer.
@bogatron has it right, you can use where, it's worth noting that you can do this natively in pandas:
df1 = df.where(pd.notnull(df), None)
Note: this changes the dtype of all columns to object.
Example:
In [1]: df = pd.DataFrame([1, np.nan])
In [2]: df
Out[2]:
0
0 1
1 NaN
In [3]: df1 = df.where(pd.notnull(df), None)
In [4]: df1
Out[4]:
0
0 1
1 None
Note: what you cannot do recast the DataFrames dtype to allow all datatypes types, using astype, and then the DataFrame fillna method:
df1 = df.astype(object).replace(np.nan, 'None')
Unfortunately neither this, nor using replace, works with None see this (closed) issue.
As an aside, it's worth noting that for most use cases you don't need to replace NaN with None, see this question about the difference between NaN and None in pandas.
However, in this specific case it seems you do (at least at the time of this answer).
Replace None with NaN in pandas dataframe - Stack Overflow
python - Pandas Replace NaN with blank/empty string - Stack Overflow
python - How to replace NaN values in a dataframe column - Stack Overflow
BUG: Replacing NaN with None in Pandas 1.3 does not work
Videos
You can use DataFrame.fillna or Series.fillna which will replace the Python object None, not the string 'None'.
import pandas as pd
import numpy as np
For dataframe:
df = df.fillna(value=np.nan)
For column or series:
df.mycol.fillna(value=np.nan, inplace=True)
Here's another option:
df.replace(to_replace=[None], value=np.nan, inplace=True)
df = df.fillna('')
This will fill na's (e.g. NaN's) with ''.
inplace is possible but should be avoided as it makes a copy internally anyway, and it will be deprecated:
df.fillna('', inplace=True)
To fill only a single column:
df.column1 = df.column1.fillna('')
One can use df['column1'] instead of df.column1.
import numpy as np
df1 = df.replace(np.nan, '', regex=True)
This might help. It will replace all NaNs with an empty string.
DataFrame.fillna() or Series.fillna() will do this for you.
Example:
In [7]: df
Out[7]:
0 1
0 NaN NaN
1 -0.494375 0.570994
2 NaN NaN
3 1.876360 -0.229738
4 NaN NaN
In [8]: df.fillna(0)
Out[8]:
0 1
0 0.000000 0.000000
1 -0.494375 0.570994
2 0.000000 0.000000
3 1.876360 -0.229738
4 0.000000 0.000000
To fill the NaNs in only one column, select just that column.
In [12]: df[1] = df[1].fillna(0)
In [13]: df
Out[13]:
0 1
0 NaN 0.000000
1 -0.494375 0.570994
2 NaN 0.000000
3 1.876360 -0.229738
4 NaN 0.000000
Or you can use the built in column-specific functionality:
df = df.fillna({1: 0})
It is not guaranteed that the slicing returns a view or a copy. You can do
df['column'] = df['column'].fillna(value)