A one liner without map is:
df['E'] = df['B'].str.replace('\W', '')
Answer from Amir Imani on Stack Overflowpython - How to remove special characers from a column of dataframe using module re? - Stack Overflow
python - Remove special characters in a pandas column using regex - Stack Overflow
python - Simple way to remove special characters and alpha numerical from dataframe - Stack Overflow
Code for removing brackets and their contents in Python. I'm sure someone might find a use for it.
Why not just use regex?
More on reddit.comA one liner without map is:
df['E'] = df['B'].str.replace('\W', '')
As this answer shows, you can use map() with a lambda function that will assemble and return any expression you like:
df['E'] = df['B'].map(lambda x: re.sub(r'\W+', '', x))
lambda simply defines anonymous functions. You can leave them anonymous, or assign them to a reference like any other object. my_function = lambda x: x.my_method(3) is equivalent to def my_function(x): return x.my_method(3).
The regex groups the digits on either side of the '.' ignoring all non-digits. The code uses these groups to create the required output. Regex101
import pandas as pd
def clean_input(m):
print(m.group(0))
if m:
val = m.group(1)
if m.group(2):
val = val + '.' +m.group(2)
return val
a = pd.DataFrame({'colA':
['7.8.',
'5..3',
'%3.2',
' ',
'3.*8',
'3.8*',
'140',
'5.5.',
'14.5 of HGB',
'>14.5',
'<14.5',
'14,5',
'14. 5']})
a['colA'].str.replace('[^\d]*(\d+)[^\d]*(?:\.)?[^\d]*(\d)*[^\d]*', clean_input)
Output:
0 7.8
1 5.3
2 3.2
3
4 3.8
5 3.8
6 140
7 5.5
8 14.5
9 14.5
10 14.5
11 14.5
12 14.5
Regex explanation:
\d- matches a digit[^<pattern>]- matches any character except the[^\d]- matches any character except for digits.[^\d]+- matches one or more of the above.(?:)- is non-capturing group where the matched characters are not captured.<pattern>?- zero or one occurance of the pattern.\.- since.is a meta character, it has to be escaped with\
Another take: split a string by periods, extract all digits from the first and second fragments, concatenate them with a period.
parts = df['colA'].str.split('\.')
part0 = parts.str[0].str.extract('(\d+)').fillna('0')
part1 = parts.str[1].str.extract('(\d+)').fillna('0')
part0 + "." + part1
Output:
#0 7.8
#1 5.0
#2 3.2
#3 0.0
#4 3.8
#5 3.8
#6 140.0
Is that what you want?
In [71]: df.nonhashtag.apply(' '.join).str.replace('[^A-Za-z\s]+', '') \
.str.split(expand=False)
Out[71]:
0 [want, better, than, Dhabi, United, Arab, Emir...
1 [Just, posted, photo, Rasim, Villa]
2 [Dhabi, International, Airport, AUH, Dhabi]
3 [just, shrug, off, Dubai, Mall, Burj, Khalifa]
4 [out, Cowboy, steppin, Notorious, going, sleep...
5 [Buona, notte, Viceroy, Hotel, Yas]
Name: nonhashtag, dtype: object
'[^A-Za-z\s]+' - is a RegEx meaning take all characters except those:
- with ASCII codes from
AtoZ - from
atoz - spaces and tabs
So .str.replace('[^A-Za-z\s]+', '') will remove all characters except letters belonging to english alphabet, spaces and tabs
I import lot of files and many a times columns names are dirty, they get Unwanted special characters and I don't know which all characters might come. I only want Underscores in column names and no spaces
df.columns = df.columns.str.strip()
df.columns = df.columns.str.replace(' ', '_')
df.columns = df.columns.str.replace(r"[^a-zA-Z\d\_]+", "")
df.columns = df.columns.str.replace(r"[^a-zA-Z\d\_]+", "")