Sure! Setup:
>>> import pandas as pd
>>> from random import randint
>>> df = pd.DataFrame({'A': [randint(1, 9) for x in range(10)],
'B': [randint(1, 9)*10 for x in range(10)],
'C': [randint(1, 9)*100 for x in range(10)]})
>>> df
A B C
0 9 40 300
1 9 70 700
2 5 70 900
3 8 80 900
4 7 50 200
5 9 30 900
6 2 80 700
7 2 80 400
8 5 80 300
9 7 70 800
We can apply column operations and get boolean Series objects:
>>> df["B"] > 50
0 False
1 True
2 True
3 True
4 False
5 False
6 True
7 True
8 True
9 True
Name: B
>>> (df["B"] > 50) & (df["C"] != 900)
or
>>> (df["B"] > 50) & ~(df["C"] == 900)
0 False
1 False
2 True
3 True
4 False
5 False
6 False
7 False
8 False
9 False
[Update, to switch to new-style .loc]:
And then we can use these to index into the object. For read access, you can chain indices:
>>> df["A"][(df["B"] > 50) & (df["C"] != 900)]
2 5
3 8
Name: A, dtype: int64
but you can get yourself into trouble because of the difference between a view and a copy doing this for write access. You can use .loc instead:
>>> df.loc[(df["B"] > 50) & (df["C"] != 900), "A"]
2 5
3 8
Name: A, dtype: int64
>>> df.loc[(df["B"] > 50) & (df["C"] != 900), "A"].values
array([5, 8], dtype=int64)
>>> df.loc[(df["B"] > 50) & (df["C"] != 900), "A"] *= 1000
>>> df
A B C
0 9 40 300
1 9 70 700
2 5000 70 900
3 8000 80 900
4 7 50 200
5 9 30 900
6 2 80 700
7 2 80 400
8 5 80 300
9 7 70 800
Answer from DSM on Stack Overflowpython - Selecting with complex criteria from pandas.DataFrame - Stack Overflow
python - Selecting specific rows from a pandas dataframe - Stack Overflow
Selecting rows in Pandas given condition a or condition b
Good idea to provide sample input - > output. Otherwise we may refer you to the same answers that are already not working for you.
Here's a snippet:
(df["B"] > 30) | (df["C"] == 700)
This will return rows where the B column is bigger than 30 or (|) C column equals 700.
python - How to select a range of values in a pandas dataframe column? - Stack Overflow
Sure! Setup:
>>> import pandas as pd
>>> from random import randint
>>> df = pd.DataFrame({'A': [randint(1, 9) for x in range(10)],
'B': [randint(1, 9)*10 for x in range(10)],
'C': [randint(1, 9)*100 for x in range(10)]})
>>> df
A B C
0 9 40 300
1 9 70 700
2 5 70 900
3 8 80 900
4 7 50 200
5 9 30 900
6 2 80 700
7 2 80 400
8 5 80 300
9 7 70 800
We can apply column operations and get boolean Series objects:
>>> df["B"] > 50
0 False
1 True
2 True
3 True
4 False
5 False
6 True
7 True
8 True
9 True
Name: B
>>> (df["B"] > 50) & (df["C"] != 900)
or
>>> (df["B"] > 50) & ~(df["C"] == 900)
0 False
1 False
2 True
3 True
4 False
5 False
6 False
7 False
8 False
9 False
[Update, to switch to new-style .loc]:
And then we can use these to index into the object. For read access, you can chain indices:
>>> df["A"][(df["B"] > 50) & (df["C"] != 900)]
2 5
3 8
Name: A, dtype: int64
but you can get yourself into trouble because of the difference between a view and a copy doing this for write access. You can use .loc instead:
>>> df.loc[(df["B"] > 50) & (df["C"] != 900), "A"]
2 5
3 8
Name: A, dtype: int64
>>> df.loc[(df["B"] > 50) & (df["C"] != 900), "A"].values
array([5, 8], dtype=int64)
>>> df.loc[(df["B"] > 50) & (df["C"] != 900), "A"] *= 1000
>>> df
A B C
0 9 40 300
1 9 70 700
2 5000 70 900
3 8000 80 900
4 7 50 200
5 9 30 900
6 2 80 700
7 2 80 400
8 5 80 300
9 7 70 800
Another solution is to use the query method:
import pandas as pd
from random import randint
df = pd.DataFrame({'A': [randint(1, 9) for x in xrange(10)],
'B': [randint(1, 9) * 10 for x in xrange(10)],
'C': [randint(1, 9) * 100 for x in xrange(10)]})
print df
A B C
0 7 20 300
1 7 80 700
2 4 90 100
3 4 30 900
4 7 80 200
5 7 60 800
6 3 80 900
7 9 40 100
8 6 40 100
9 3 10 600
print df.query('B > 50 and C != 900')
A B C
1 7 80 700
2 4 90 100
4 7 80 200
5 7 60 800
Now if you want to change the returned values in column A you can save their index:
my_query_index = df.query('B > 50 & C != 900').index
....and use .iloc to change them i.e:
df.iloc[my_query_index, 0] = 5000
print df
A B C
0 7 20 300
1 5000 80 700
2 5000 90 100
3 4 30 900
4 5000 80 200
5 5000 60 800
6 3 80 900
7 9 40 100
8 6 40 100
9 3 10 600
You could use either pandas.DataFrame.loc or pandas.DataFrame.iloc. See examples below.
import pandas as pd
d = [{'a': 1, 'b': 2, 'c': 3, 'd': 4},
{'a': 100, 'b': 200, 'c': 300, 'd': 400},
{'a': 1000, 'b': 2000, 'c': 3000, 'd': 4000 },
{'a': 1500, 'b': 2500, 'c': 3500, 'd': 4500}]
df = pd.DataFrame(d)
print(df) # Print original dataframe
print(df.loc[1:2]) # Print rows with index 1 and 2, (method 1)
print(df.iloc[1:3]) # Print rows with index 1 and 2, (method 2)
Original dataframe: print(df) will print:
a b c d
0 1 2 3 4
1 100 200 300 400
2 1000 2000 3000 4000
3 1500 2500 3500 4500
And print(df.loc[1:2]) for index selection by label:
a b c d
1 100 200 300 400
2 1000 2000 3000 4000
And print(df.iloc[1:3]) for row selection by integer. As mentioned by ALollz, rows are treated as numbers from 0 to len(df):
a b c d
1 100 200 300 400
2 1000 2000 3000 4000
A rule of thumb could be:
Use
.locwhen you want to refer to the actual value of the index, being a string or integer.Use
.ilocwhen you want to refer to the underlying row number which always ranges from 0 tolen(df).
Note that the end value of the slice in .loc is included. This is not the case for .iloc, and for Python slices in general.
Pandas in general
Pandas has 'easy' ways of doing all sorts of stuff like this. If you have a problem that you think is common for manipulation of tabular data, try searching for pandas ways of getting it done before inventing it yourself. Pandas will almost always have a syntactically concise and computationally faster way of doing things than what we can write ourselves.
Use this:
rowData = your_df.loc[ 'index' , : ]
How do I select rows of a dataframe that fulfill condition a or condition b? I've spent the past hour Googling this, and somehow the few answers I've found give an error when I try them. I somehow can do and fine, just not or.
Use between with inclusive=False for strict inequalities:
df['two'].between(-0.5, 0.5, inclusive=False)
The inclusive parameter determines if the endpoints are included or not (True: <=, False: <). This applies to both signs. If you want mixed inequalities, you'll need to code them explicitly:
(df['two'] >= -0.5) & (df['two'] < 0.5)
.between is a good solution, but if you want finer control use this:
(0.5 <= df['two']) & (df['two'] < 0.5)
The operator & is different from and. The other operators are | for or, ~ for not. See this discussion for more info.
Your statement was the same as this:
(0.5 <= df['two']) and (df['two'] < 0.5)
Hence it raised the error.