For multiple conditions ie. (df['employrate'] <=55) & (df['employrate'] > 50)
use this:
df['employrate'] = np.where(
(df['employrate'] <=55) & (df['employrate'] > 50) , 11, df['employrate']
)
or you can do it this way as well,
gm.loc[(gm['employrate'] <55) & (gm['employrate'] > 50),'employrate']=11
here informal syntax can be:
<dataset>.loc[<filter1> & (<filter2>),'<variable>']='<value>'
out[108]:
country employrate alcconsumption
0 Afghanistan 55.700001 .03
1 Albania 11.000000 7.29
2 Algeria 11.000000 .69
3 Andorra nan 10.17
4 Angola 75.699997 5.57
therefore syntax we used here is:
df['<column_name>'] = np.where((<filter 1> ) & (<filter 2>) , <new value>, df['column_name'])
for single condition, ie. ( 'employrate'] > 70 )
country employrate alcconsumption
0 Afghanistan 55.7000007629394 .03
1 Albania 51.4000015258789 7.29
2 Algeria 50.5 .69
3 Andorra 10.17
4 Angola 75.6999969482422 5.57
use this:
df.loc[df['employrate'] > 70, 'employrate'] = 7
country employrate alcconsumption
0 Afghanistan 55.700001 .03
1 Albania 51.400002 7.29
2 Algeria 50.500000 .69
3 Andorra nan 10.17
4 Angola 7.000000 5.57
therefore syntax here is:
df.loc[<mask>(here mask is generating the labels to index) , <optional column(s)> ]
Answer from Harshit Jain on Stack Overflow
python - Replacing values in a pandas dataframe based on multiple conditions - Stack Overflow
data mining - Pandas change value of a column based another column condition - Data Science Stack Exchange
Python Pandas replace value based on multiple column conditions - Stack Overflow
python - Pandas DataFrame: replace all values in a column, based on condition - Stack Overflow
For multiple conditions ie. (df['employrate'] <=55) & (df['employrate'] > 50)
use this:
df['employrate'] = np.where(
(df['employrate'] <=55) & (df['employrate'] > 50) , 11, df['employrate']
)
or you can do it this way as well,
gm.loc[(gm['employrate'] <55) & (gm['employrate'] > 50),'employrate']=11
here informal syntax can be:
<dataset>.loc[<filter1> & (<filter2>),'<variable>']='<value>'
out[108]:
country employrate alcconsumption
0 Afghanistan 55.700001 .03
1 Albania 11.000000 7.29
2 Algeria 11.000000 .69
3 Andorra nan 10.17
4 Angola 75.699997 5.57
therefore syntax we used here is:
df['<column_name>'] = np.where((<filter 1> ) & (<filter 2>) , <new value>, df['column_name'])
for single condition, ie. ( 'employrate'] > 70 )
country employrate alcconsumption
0 Afghanistan 55.7000007629394 .03
1 Albania 51.4000015258789 7.29
2 Algeria 50.5 .69
3 Andorra 10.17
4 Angola 75.6999969482422 5.57
use this:
df.loc[df['employrate'] > 70, 'employrate'] = 7
country employrate alcconsumption
0 Afghanistan 55.700001 .03
1 Albania 51.400002 7.29
2 Algeria 50.500000 .69
3 Andorra nan 10.17
4 Angola 7.000000 5.57
therefore syntax here is:
df.loc[<mask>(here mask is generating the labels to index) , <optional column(s)> ]
df1.apply(np.sign).replace({-1: 'down', 1: 'up', 0: 'zero'})
Output:
0 1 2
0 down up up
1 up down down
2 up down down
3 down down up
4 down down up
5 down up up
6 down up down
7 up down down
8 up up down
9 down up up
P.S. Getting exactly zero with randn is pretty unlikely, of course
What I want to achieve: Condition: where column2 == 2 leave to be 2 if column1 < 30 elsif change to 3 if column1 > 90
This can be simplified into where (column2 == 2 and column1 > 90) set column2 to 3. The column1 < 30 part is redundant, since the value of column2 is only going to change from 2 to 3 if column1 > 90.
In the code that you provide, you are using pandas function replace, which operates on the entire Series, as stated in the reference:
Values of the Series are replaced with other values dynamically. This differs from updating with .loc or .iloc, which require you to specify a location to update with some value.
This means that for each iteration of for x in filter1 your code performs global replacement, which is not what you want to do - you want to update the specific row of column2 that corresponds to x from column1 (which you are iterating over).
the problem is 2 does not change to 3 where column1 > 90
This is truly strange. I would expect the code you provided to have changed every instance of 2 in column2 to 3 as soon as it encountered an x >= 30, as dictated by your code conditional statement (the execution of the else branch). This discrepancy may stem from the fact that you are assigning to column2 the result of global replacement performed on the column Output (the contents of which are unknown). In any case, if you want your program to do something under a specific condition, such as x > 90, it should be explicitly stated in the code. You should also note that the statement data['column2'] = data['column2'].replace([2], [2]) achieves nothing, since 2 is being replaced with 2 and the same column is both the source and the destination.
What you could use to solve this particular task is a boolean mask (or the query method). Both are explained in an excellent manner in this question.
Using a boolean mask would be the easiest approach in your case:
mask = (data['column2'] == 2) & (data['column1'] > 90)
data['column2'][mask] = 3
The first line builds a Series of booleans (True/False) that indicate whether the supplied condition is satisfied.
The second line assigns the value 3 to those rows of column2 where the mask is True.
I've had success approaching this in a slightly different way.
import numpy as np
data['column2'] = np.where((data['column1'] < 30)
& (data['column2'] ==2), #Identifies the case to apply to
data['column2'], #This is the value that is inserted
data['column2']) #This is the column that is affected
data['column2'] = np.where((data['column1'] > 90)
& (data['column2'] ==2), #For rows with column1 > 90
data['column3'], #We place column3 values
data['column2']) #In column two
This is a little wordier than a loop, but I've found it to be the most intuitive way to do this sort of data manipulation with pandas.
You are very close, but you have the arguments switched in np.where, the syntax is np.where(cond, if_cond_True, if_cond_False). The columns A and B should have the value of column if the condition is satisfied (if_cond_True), otherwise they keep their original values (if_cond_False).
import pandas as pd
import numpy as np
data_in = {'A':['A1', '', '', 'A4',''],
'B':['', 'B2', 'B3', '',''],
'C':['C1','C2','','','C5']}
df_in = pd.DataFrame(data_in)
maskA = df_in['A'] != '' # A not empty
maskB = df_in['B'] != '' # B not empty
maskC = df_in['C'] != '' # C not empty
# If the column havs NaNs instead of '' then use :
#
# maskA = df_in['A'].notnull() # A not empty
# maskB = df_in['B'].notnull() # B not empty
# maskC = df_in['C'].notnull() # C not empty
# If A and C are not empty, A = C, else A keep its value
df_in['A'] = np.where(maskA & maskC, df_in['C'], df_in['A'])
# If B and C are not empty, B = C, else B keep its value
df_in['B'] = np.where(maskB & maskC, df_in['C'], df_in['B'])
# If (A and C are not empty) or (B and C are not empty),
# C should be empty, else C keep its value
df_in['C'] = np.where((maskA & maskC) | (maskB & maskC), "", df_in['C'])
Output
>>> df_in
A B C
0 C1
1 C2
2 B3
3 A4
4 C5
I'm not sure if there is an issue setting a columns value that is also in the where condition off hand but you could always create a temp column and rename/drop other outputs based on that.
An alternative is to use the apply function.
def update_data(row):
a = row['A']
b = row['B']
c = row['C']
if not c.isna():
if a.isna():
row['A'] = c
if b.isna():
row['B'] = c
return row
df_new = df.apply(update_data, axis=1)
Apply will definitely get you the correct result, however, I'm not certain as to what your desired outcome is so you may need to adjust the logic. The above will set columns A and/or B = C if A is a na type object ("" is a na type) and C is not a na type object. Otherwise it will not update anything.
I'm not sure what you want by "clear column C". You can just drop the column if that's what you want. If you want to change the value you can do so in the update_data function or do a string replace.
You need to select that column:
In [41]:
df.loc[df['First Season'] > 1990, 'First Season'] = 1
df
Out[41]:
Team First Season Total Games
0 Dallas Cowboys 1960 894
1 Chicago Bears 1920 1357
2 Green Bay Packers 1921 1339
3 Miami Dolphins 1966 792
4 Baltimore Ravens 1 326
5 San Franciso 49ers 1950 1003
So the syntax here is:
df.loc[<mask>(here mask is generating the labels to index) , <optional column(s)> ]
You can check the docs and also the 10 minutes to pandas which shows the semantics
EDIT
If you want to generate a boolean indicator then you can just use the boolean condition to generate a boolean Series and cast the dtype to int this will convert True and False to 1 and 0 respectively:
In [43]:
df['First Season'] = (df['First Season'] > 1990).astype(int)
df
Out[43]:
Team First Season Total Games
0 Dallas Cowboys 0 894
1 Chicago Bears 0 1357
2 Green Bay Packers 0 1339
3 Miami Dolphins 0 792
4 Baltimore Ravens 1 326
5 San Franciso 49ers 0 1003
A bit late to the party but still - I prefer using numpy where:
import numpy as np
df['First Season'] = np.where(df['First Season'] > 1990, 1, df['First Season'])