Given that this is the top Google result when searching for "Pandas replace is not working" I'd like to also mention that:
replace does full replacement searches, unless you turn on the regex switch. Use regex=True, and it should perform partial replacements as well.
This took me 30 minutes to find out, so hopefully I've saved the next person 30 minutes.
Answer from Reddspark on Stack OverflowThis is genuinely driving me crazy.
I have a data frame of unit prices in string format i'm trying to get them to a float
item_df['Unit Price'] = item_df['Unit Price'].replace('$','')and all the '$' are still there.
THEN when I do this:
item_df['Unit Price'][1] = item_df['Unit Price'][1].replace('$','')The '$' is gone from that index ಠ_ಠ. What the hell is going on?? Am I taking crazy pills or missing some fundamental concept?
Any help would be much appreciated.
Thanks,
Given that this is the top Google result when searching for "Pandas replace is not working" I'd like to also mention that:
replace does full replacement searches, unless you turn on the regex switch. Use regex=True, and it should perform partial replacements as well.
This took me 30 minutes to find out, so hopefully I've saved the next person 30 minutes.
You need to assign back
df = df.replace('white', np.nan)
or pass param inplace=True:
In [50]:
d = {'color' : pd.Series(['white', 'blue', 'orange']),
'second_color': pd.Series(['white', 'black', 'blue']),
'value' : pd.Series([1., 2., 3.])}
df = pd.DataFrame(d)
df.replace('white', np.nan, inplace=True)
df
Out[50]:
color second_color value
0 NaN NaN 1.0
1 blue black 2.0
2 orange blue 3.0
Most pandas ops return a copy and most have param inplace which is usually defaulted to False
Series.str.replace() is not actually the same as str.replace()
BUG: Pandas.DataFrame.str.replace function fails silently for mixed data mixing strings and float/int and replaces with NaN
python - Replace method in pandas not giving expected result - Data Science Stack Exchange
Problem with DataFrame.replace using None
Try this if you decide to use pandas:
readFile = pd.read_csv("C:/Users/siddhesh.kalgaonkar/Desktop/data01.txt",header=None)
readFile.columns = ['IP']
readFile['IP'] = readFile['IP'].replace(regex='((?<=[0-9])[0-9]|(?<=\.)[0-9])',value='X')
print(readFile)
and this without pandas:
readFile = open("C:/Users/siddhesh.kalgaonkar/Desktop/data01.txt","r")
for line in readFile:
lines = line.strip()
finalline = re.sub(pattern='((?<=[0-9])[0-9]|(?<=\.)[0-9])',repl='X',string=lines)
print(finalline)
(?<=[0-9])[0-9] this part matches if the current position in the string is a digit and is preceded by a digit.
| or
(?<=\.)[0-9]) this part matches if the current position in the string is is a digit and is preceded by a period.
This is pure(almost) python:
list(map(lambda x: x[0] + '.'.join(['X' * len(c) for c in x[1:].split('.')]), my_df['IP']))
Explanation:
- Use the map to iterate over each row in my_df['IP'] column.
- Per each IP value, split into first char and others using the x[0], x[1:]
notation.
- Get each part in x[1:] using the split method.
- For each part get its length and accordingly create a string made of X's.
- Rejoin this X's string into one string with '.' between them.
If using python2 you dont need the list.
I am trying to clean my twitter_handle column where some of the names have ?langen at the end of them. This is what I tried...
updated['twitter_handle'] = updated['twitter_handle'].str.replace('?langen', '', regex=True)re.error: nothing to repeat at position 0