You need mask:
sample['PR'] = sample['PR'].mask(sample['PR'] < 90, np.nan)
Another solution with loc and boolean indexing:
sample.loc[sample['PR'] < 90, 'PR'] = np.nan
Sample:
import pandas as pd
import numpy as np
sample = pd.DataFrame({'PR':[10,100,40] })
print (sample)
PR
0 10
1 100
2 40
sample['PR'] = sample['PR'].mask(sample['PR'] < 90, np.nan)
print (sample)
PR
0 NaN
1 100.0
2 NaN
sample.loc[sample['PR'] < 90, 'PR'] = np.nan
print (sample)
PR
0 NaN
1 100.0
2 NaN
EDIT:
Solution with apply:
sample['PR'] = sample['PR'].apply(lambda x: np.nan if x < 90 else x)
Timings len(df)=300k:
sample = pd.concat([sample]*100000).reset_index(drop=True)
In [853]: %timeit sample['PR'].apply(lambda x: np.nan if x < 90 else x)
10 loops, best of 3: 102 ms per loop
In [854]: %timeit sample['PR'].mask(sample['PR'] < 90, np.nan)
The slowest run took 4.28 times longer than the fastest. This could mean that an intermediate result is being cached.
100 loops, best of 3: 3.71 ms per loop
Answer from jezrael on Stack OverflowHow to properly apply a lambda function into a pandas data frame column - Stack Overflow
python - How can I use the apply() function for a single column? - Stack Overflow
python - Creating a new pandas column by using lambda function on two existing columns - Stack Overflow
How to apply a lambda on pandas DataFrame column headers
You need mask:
sample['PR'] = sample['PR'].mask(sample['PR'] < 90, np.nan)
Another solution with loc and boolean indexing:
sample.loc[sample['PR'] < 90, 'PR'] = np.nan
Sample:
import pandas as pd
import numpy as np
sample = pd.DataFrame({'PR':[10,100,40] })
print (sample)
PR
0 10
1 100
2 40
sample['PR'] = sample['PR'].mask(sample['PR'] < 90, np.nan)
print (sample)
PR
0 NaN
1 100.0
2 NaN
sample.loc[sample['PR'] < 90, 'PR'] = np.nan
print (sample)
PR
0 NaN
1 100.0
2 NaN
EDIT:
Solution with apply:
sample['PR'] = sample['PR'].apply(lambda x: np.nan if x < 90 else x)
Timings len(df)=300k:
sample = pd.concat([sample]*100000).reset_index(drop=True)
In [853]: %timeit sample['PR'].apply(lambda x: np.nan if x < 90 else x)
10 loops, best of 3: 102 ms per loop
In [854]: %timeit sample['PR'].mask(sample['PR'] < 90, np.nan)
The slowest run took 4.28 times longer than the fastest. This could mean that an intermediate result is being cached.
100 loops, best of 3: 3.71 ms per loop
You need to add else in your lambda function. Because you are telling what to do in case your condition(here x < 90) is met, but you are not telling what to do in case the condition is not met.
sample['PR'] = sample['PR'].apply(lambda x: 'NaN' if x < 90 else x)
Given a sample dataframe df as:
a b
0 1 2
1 2 3
2 3 4
3 4 5
what you want is:
df['a'] = df['a'].apply(lambda x: x + 1)
that returns:
a b
0 2 2
1 3 3
2 4 4
3 5 5
For a single column better to use map(), like this:
df = pd.DataFrame([{'a': 15, 'b': 15, 'c': 5}, {'a': 20, 'b': 10, 'c': 7}, {'a': 25, 'b': 30, 'c': 9}])
a b c
0 15 15 5
1 20 10 7
2 25 30 9
df['a'] = df['a'].map(lambda a: a / 2.)
a b c
0 7.5 15 5
1 10.0 10 7
2 12.5 30 9
You can use function map and select by function np.where more info
print df
# a b
#0 aaa rrrr
#1 bb k
#2 ccc e
#condition if condition is True then len column a else column b
df['c'] = np.where(df['a'].map(len) > df['b'].map(len), df['a'].map(len), df['b'].map(len))
print df
# a b c
#0 aaa rrrr 4
#1 bb k 2
#2 ccc e 3
Next solution is with function apply with parameter axis=1:
axis = 1 or ‘columns’: apply function to each row
df['c'] = df.apply(lambda x: max(len(x['a']), len(x['b'])), axis=1)
Working on strings is a bit of a special case because string operations in pandas are not optimized so, a Python loop may actually perform better than vectorized pandas methods. So a list comprehension is a viable method; it's readable and very fast:
df['c'] = [max(len(a), len(b)) for a, b in zip(df['a'], df['b'])]
For a little shorter code, you can try map() (or applymap() for pandas<2.1.0):
df['c'] = df.map(len).max(axis=1)
If you're applying a lambda using if-condition, make sure to also supply the else.
df['c'] = df.apply(lambda row: len(row['a']) if len(row['a']) > len(row['b']) else len(row['b']), axis=1)
In general, you should avoid using a lambda wherever possible, because pandas has a whole host of optimized operations you can use to operate directly on the columns. For example, if you need to find the maximum value of each row, you can simply call max(axis=1) like: df[['a', 'b']].max(1).