You can use the vectorised str method to split on '@' character and then join the left side with the new domain name:
In [42]:
df = pd.DataFrame({'email':['[email protected]', '[email protected]', '[email protected]']})
df
Out[42]:
email
0 [email protected]
1 [email protected]
2 [email protected]
In [43]:
df['email'] = df.email.str.split('@').str[0] + '@newcompany.com'
df
Out[43]:
email
0 [email protected]
1 [email protected]
2 [email protected]
another method is to call the vectorised replace which accepts a regex as a pattern on the strings:
In [56]:
df['email'] = df['email'].str.replace(r'@.+', '@newcompany.com')
df
Out[56]:
email
0 [email protected]
1 [email protected]
2 [email protected]
Timings
In [58]:
%timeit df['email'] = df['email'].str.replace(r'@.+', '@newcompany.com')
1000 loops, best of 3: 632 µs per loop
In [60]:
%timeit df['email'] = df.email.str.split('@').str[0] + '@newcompany.com'
1000 loops, best of 3: 1.66 ms per loop
In [63]:
%timeit df['email'] = df['email'].replace(r'@.+', '@newcompany.com', regex=True)
1000 loops, best of 3: 738 µs per loop
Here we can see that the str.replace regex version is nearly 3x faster than the split method, interestingly the Series.replace method which would seem to be doing the same thing as the str.replace is slower.
You can use the vectorised str method to split on '@' character and then join the left side with the new domain name:
In [42]:
df = pd.DataFrame({'email':['[email protected]', '[email protected]', '[email protected]']})
df
Out[42]:
email
0 [email protected]
1 [email protected]
2 [email protected]
In [43]:
df['email'] = df.email.str.split('@').str[0] + '@newcompany.com'
df
Out[43]:
email
0 [email protected]
1 [email protected]
2 [email protected]
another method is to call the vectorised replace which accepts a regex as a pattern on the strings:
In [56]:
df['email'] = df['email'].str.replace(r'@.+', '@newcompany.com')
df
Out[56]:
email
0 [email protected]
1 [email protected]
2 [email protected]
Timings
In [58]:
%timeit df['email'] = df['email'].str.replace(r'@.+', '@newcompany.com')
1000 loops, best of 3: 632 µs per loop
In [60]:
%timeit df['email'] = df.email.str.split('@').str[0] + '@newcompany.com'
1000 loops, best of 3: 1.66 ms per loop
In [63]:
%timeit df['email'] = df['email'].replace(r'@.+', '@newcompany.com', regex=True)
1000 loops, best of 3: 738 µs per loop
Here we can see that the str.replace regex version is nearly 3x faster than the split method, interestingly the Series.replace method which would seem to be doing the same thing as the str.replace is slower.
This sounds like a job for regex! Pandas' replace will let you use regular expressions, you just have to set it to true. You're most of the way there, the following should work for you.
df_users['EMAIL'].replace('@.*$', '@newcompany.com', inplace=True, regex=True)
Python doesn't really deal with wildcards. You should read more on Regex, which isn't Python specific.
In any case, this should take care of it:
ex = "/Volumes/Obelix/5215.tif, /Volumes/Gemeinsam/25.tif, /Volumes/Obelix/5100.tif"
newex = re.sub(r'/Volumes/Gemeinsam/.+?,?(\s|$)', '', ex)
You'll have to do a bit of pre-parsing depending on how sure you are of the data, but you can use the in in operator to check if a string contains another string. Then, you can either use a loop or list comprehension depending on what is most readable.
ex = [x.strip() for x in "/Volumes/Obelix/5215.tif, /Volumes/Gemeinsam/25.tif, /Volumes/Obelix/5100.tif".split(',')]
check = "/Volumes/Gemeinsam"
new_ex = [x for x in ex if check not in x]
df_new = pd.DataFrame(np.ones(df.shape), columns=df.columns)
import numpy as np
import pandas as pd
d = [
[1,1,1,1,1],
[2,2,2,2,2],
[3,3,3,3,3],
[4,4,4,4,4],
[5,5,5,5,5],
]
cols = ["A","B","C","D","E"]
%timeit df1 = pd.DataFrame(np.ones(df.shape), columns=df.columns)
10000 loops, best of 3: 94.6 µs per loop
%timeit df2 = df.copy(); df2.loc[:, :] = 1
1000 loops, best of 3: 245 µs per loop
%timeit df3 = df * 0 + 1
1000 loops, best of 3: 200 µs per loop
It's actually pretty easy.
import pandas as pd
d = [
[1,1,1,1,1],
[2,2,2,2,2],
[3,3,3,3,3],
[4,4,4,4,4],
[5,5,5,5,5],
]
cols = ["A","B","C","D","E"]
df = pd.DataFrame(d, columns=cols)
print df
print "------------------------"
df.loc[:,:] = 1
print df
Result:
A B C D E
0 1 1 1 1 1
1 2 2 2 2 2
2 3 3 3 3 3
3 4 4 4 4 4
4 5 5 5 5 5
------------------------
A B C D E
0 1 1 1 1 1
1 1 1 1 1 1
2 1 1 1 1 1
3 1 1 1 1 1
4 1 1 1 1 1
Obviously, df.loc[:,:] means you target all rows across all columns. Just use df2 = df.copy() or something if you want a new dataframe.
You can perform this task by forming a |-separated string. This works because pd.Series.str.replace accepts regex:
Replace occurrences of pattern/regex in the Series/Index with some other string. Equivalent to str.replace() or re.sub().
This avoids the need to create a dictionary.
import pandas as pd
df = pd.DataFrame({'A': ['LOCAL TEST', 'TEST FOREIGN', 'ANOTHER HELLO', 'NOTHING']})
pattern = '|'.join(['LOCAL', 'FOREIGN', 'HELLO'])
df['A'] = df['A'].str.replace(pattern, 'CORP', regex=True)
# A
# 0 CORP TEST
# 1 TEST CORP
# 2 ANOTHER CORP
# 3 NOTHING
The answer of @Rakesh is very neat but does not allow for substrings. With a small change however, it does.
- Use a replacement dictionary because it makes it much more generic
- Add the keyword argument
regex=TruetoSeries.replace()(notSeries.str.replace) This does two things actually: It changes your replacement to regex replacement, which is much more powerful but you will have to escape special characters. Beware for that. Secondly it will make the replace work on substrings instead of the entire string. Which is really cool!
replacement = {
"LOCAL": "CORP",
"FOREIGN": "CORP",
"HELLO": "CORP"
}
dataUS['sec_type'].replace(replacement, regex=True)
Full code example
dataUS = pd.DataFrame({'sec_type': ['LOCAL', 'Sample text LOCAL', 'Sample text LOCAL sample FOREIGN']})
replacement = {
"LOCAL": "CORP",
"FOREIGN": "CORP",
"HELLO": "CORP"
}
dataUS['sec_type'].replace(replacement, regex=True)
Output
0 CORP
1 CORP
2 Sample text CORP
3 Sample text CORP sample CORP
Name: sec_type, dtype: object
Try with split
df['col4'] = df.col4.str.split('.').str[2]
Or for a more precise match try with .str.extract since you’re looking into regular expressions:
>>> df['col4'] = df['col4'].str.extract('\d+\.UNITED STATES\.(.*)\.NBA\.csv')[0]
>>> df
col0 col1 col2 col3 col4
0 128544 20210831 2200 882.2 LAKERS
1 128545 20210831 2300 918.9 LAKERS
2 128546 20210901 0 NaN CELTICS
3 128547 20210901 100 NaN CELTICS