You can perform this task by forming a |-separated string. This works because pd.Series.str.replace accepts regex:
Replace occurrences of pattern/regex in the Series/Index with some other string. Equivalent to str.replace() or re.sub().
This avoids the need to create a dictionary.
import pandas as pd
df = pd.DataFrame({'A': ['LOCAL TEST', 'TEST FOREIGN', 'ANOTHER HELLO', 'NOTHING']})
pattern = '|'.join(['LOCAL', 'FOREIGN', 'HELLO'])
df['A'] = df['A'].str.replace(pattern, 'CORP', regex=True)
# A
# 0 CORP TEST
# 1 TEST CORP
# 2 ANOTHER CORP
# 3 NOTHING
Answer from jpp on Stack OverflowYou can perform this task by forming a |-separated string. This works because pd.Series.str.replace accepts regex:
Replace occurrences of pattern/regex in the Series/Index with some other string. Equivalent to str.replace() or re.sub().
This avoids the need to create a dictionary.
import pandas as pd
df = pd.DataFrame({'A': ['LOCAL TEST', 'TEST FOREIGN', 'ANOTHER HELLO', 'NOTHING']})
pattern = '|'.join(['LOCAL', 'FOREIGN', 'HELLO'])
df['A'] = df['A'].str.replace(pattern, 'CORP', regex=True)
# A
# 0 CORP TEST
# 1 TEST CORP
# 2 ANOTHER CORP
# 3 NOTHING
The answer of @Rakesh is very neat but does not allow for substrings. With a small change however, it does.
- Use a replacement dictionary because it makes it much more generic
- Add the keyword argument
regex=TruetoSeries.replace()(notSeries.str.replace) This does two things actually: It changes your replacement to regex replacement, which is much more powerful but you will have to escape special characters. Beware for that. Secondly it will make the replace work on substrings instead of the entire string. Which is really cool!
replacement = {
"LOCAL": "CORP",
"FOREIGN": "CORP",
"HELLO": "CORP"
}
dataUS['sec_type'].replace(replacement, regex=True)
Full code example
dataUS = pd.DataFrame({'sec_type': ['LOCAL', 'Sample text LOCAL', 'Sample text LOCAL sample FOREIGN']})
replacement = {
"LOCAL": "CORP",
"FOREIGN": "CORP",
"HELLO": "CORP"
}
dataUS['sec_type'].replace(replacement, regex=True)
Output
0 CORP
1 CORP
2 Sample text CORP
3 Sample text CORP sample CORP
Name: sec_type, dtype: object
Vectorized .str.replace() for multiple characters in pandas
python - Pandas: replace substring in string - Stack Overflow
python - pandas dataframe replace multiple substring of column - Stack Overflow
python 2.7 - Pandas dataframe replace string in multiple columns by finding substring - Stack Overflow
I have a dataframe:
{'country': {0: 'Afghanistan?*', 1: 'Albania?*'},
'region': {0: 'Asia', 1: 'Europe'},
'subregion': {0: 'Southern Asia', 1: 'Southern Europe'},
'rate_per_1000': {0: 6.7, 1: 2.1},
'count': {0: '2,474', 1: '61'},
'year': {0: 2018, 1: 2020},
'source': {0: 'NSO', 1: 'NSO'}}
country region subregion rate_per_1000 count year source
0 Afghanistan?* Asia Southern Asia 6.7 2,474 2018 NSO
1 Albania?* Europe Southern Europe 2.1 61 2020 NSOThere are multiple bad characters here that I want to get rid of. I made a short function for .apply() to get rid of them, however I am looping over a defined list of bad characters. This gives a bad code smell to me, I think this operation could be more vectorized in some way. This is what I've tried:
bad_chars = ['?', '*', ',']
def string_cleaner(col):
if col.dtype == 'object':
for char in bad_chars:
col = col.str.replace(f'{char}', '')
return col
homicide_by_country = homicide_by_country.apply(string_cleaner)
homicide_by_country
country region subregion rate_per_1000 count year source
0 Afghanistan Asia Southern Asia None 2474 None NSO
1 Albania Europe Southern Europe None 61 None NSOMy desired outcome is a more pythonic/pandonic technique for accomplishing the same outcome.
You may notice for some reason my rate_per_1000 columns goes blank. I haven't troubleshot that problem yet but if you spot something obvious I'm all ears.
Use replace with dict for replacing and regex=True:
df['url'] = df['url'].replace({'icashier.alipay.com': 'aliexpress.com'}, regex=True)
print (df)
url
0 aliexpress.com/catalog/2758186/detail.aspx
1 aliexpress.com/catalog/2758186/detail.aspx
2 aliexpress.com/catalog/2758186/detail.aspx
3 vk.com
use str.replace to replace a substring, replace looks for exact matches unless you pass a regex pattern and param regex=True:
In [25]:
df['url'] = df['url'].str.replace('icashier.alipay.com', 'aliexpress.com')
df['url']
Out[25]:
0 aliexpress.com/catalog/2758186/detail.aspx
1 aliexpress.com/catalog/2758186/detail.aspx
2 aliexpress.com/catalog/2758186/detail.aspx
3 vk.com
Name: url, dtype: object
Use:
import pandas as pd
df = pd.DataFrame({'A': ['$5,756', '3434', '$45', '1,344']})
df['A'] = df['A'].str.replace('[$,]', '', regex=True)
print(df)
Output
A
0 5756
1 3434
2 45
3 1344
The problem is that the character $ has a special meaning in regular expressions. From the documentation (emphasis mine):
$
Matches the end of the string or just before the newline at the end of the string, and in MULTILINE mode also matches before a newline. foo matches both ‘foo’ and ‘foobar’, while the regular expression foo$ matches only ‘foo’. More interestingly, searching for foo.$ in 'foo1\nfoo2\n' matches ‘foo2’ normally, but ‘foo1’ in MULTILINE mode; searching for a single $ in 'foo\n' will find two (empty) matches: one just before the newline, and one at the end of the string.mode; searching for a single $ in 'foo\n' will find two (empty) matches: one just before the newline, and one at the end of the string.
So you need to escape the character or put it inside a character class.
As an alternative use:
df['A'].str.replace('\$|,', '', regex=True) # note the escaping \
If you only have integer-like numbers an easy option is to remove all but digits \D, then you don't have to deal with other special regex characters like $:
df['A'] = df['A'].str.replace(r'\D', '', regex=True)
output:
A
0 5756
1 3434
2 45
3 1344