You can use a bool index array that you can produce using np.in1d.
You can index a np.ndarray along any axis you want using for example an array of bools indicating whether an element should be included. Since you want to index along axis=0, meaning you want to choose from the outest index, you need to have 1D np.array whose length is the number of rows. Each of its elements will indicate whether the row should be included.
A fast way to get this is to use np.in1d on the second column of a. You get all elements of that column by a[:, 1]. Now you have a 1D np.array whose elements should be checked against your filter. Thats what np.in1d is for.
So the complete code would look like:
import numpy as np
a = np.asarray([[2,'a'],[3,'b'],[4,'c'],[5,'d']])
filter = np.asarray(['a','c'])
a[np.in1d(a[:, 1], filter)]
or in a longer form:
import numpy as np
a = np.asarray([[2,'a'],[3,'b'],[4,'c'],[5,'d']])
filter = np.asarray(['a','c'])
mask = np.in1d(a[:, 1], filter)
a[mask]
Answer from jotasi on Stack OverflowYou can use a bool index array that you can produce using np.in1d.
You can index a np.ndarray along any axis you want using for example an array of bools indicating whether an element should be included. Since you want to index along axis=0, meaning you want to choose from the outest index, you need to have 1D np.array whose length is the number of rows. Each of its elements will indicate whether the row should be included.
A fast way to get this is to use np.in1d on the second column of a. You get all elements of that column by a[:, 1]. Now you have a 1D np.array whose elements should be checked against your filter. Thats what np.in1d is for.
So the complete code would look like:
import numpy as np
a = np.asarray([[2,'a'],[3,'b'],[4,'c'],[5,'d']])
filter = np.asarray(['a','c'])
a[np.in1d(a[:, 1], filter)]
or in a longer form:
import numpy as np
a = np.asarray([[2,'a'],[3,'b'],[4,'c'],[5,'d']])
filter = np.asarray(['a','c'])
mask = np.in1d(a[:, 1], filter)
a[mask]
A somewhat elaborate pure numpy vectorized solution:
>>> import numpy
>>> a = numpy.asarray([[2,'a'],[3,'b'],[4,'c'],[5,'d']])
>>> filter = numpy.array(['a','c'])
>>> a[(a[:,1,None] == filter[None,:]).any(axis=1)]
array([['2', 'a'],
['4', 'c']],
dtype='|S21')
None in the index creates a singleton dimension, therefore we can compare the column of a and the row of filter, and then reduce the resulting boolean array
>>> a[:,1,None] == filter[None,:]
array([[ True, False],
[False, False],
[False, True],
[False, False]], dtype=bool)
over the second dimension with any.
>>> a[ (-6<a[:,1]) & (a[:,1]<3) ]
array([[ 1, 2],
[ 3, -5]])
The np.ma.masked_inside(a, -6, 3) will create a MaskedArray object, where the values between -6 and 3 are masked (that is, flagged as invalid). In other terms, you're filtering out the values between -6 and 3.
Instead, you should use np.ma.masked_outside(a, -6, 3):
>>> a = np.array([[1,2],[3,-5],[6,-15],[10,2]])
>>> np.ma.masked_outside(a,-6,3)
>>> masked_array(data =
[[1 2]
[3 -5]
[-- --]
[-- 2]],
mask =
[[False False]
[False False]
[ True True]
[ True False]],
fill_value = 999999)
Note that with this function, you are filtering out the whole array, element by element, which is not what you want.
The indexing approach given in another solution is by far the most straightforward and understandable.
you can use multiple filters in a slice, something like this:
x = np.arange(90.).reshape(30, 3)
#set the first 10 rows of cols 1,2 to be zero
x[0:10, 0:2] = 0.0
x[(x[:,0] == 0.) & (x[:,1] == 0.) & (x[:,2] > 10)]
#should give only a few rows
array([[ 0., 0., 11.],
[ 0., 0., 14.],
[ 0., 0., 17.],
[ 0., 0., 20.],
[ 0., 0., 23.],
[ 0., 0., 26.],
[ 0., 0., 29.]])
How about this -
meta[meta[:,2]<X * np.all(meta[:,0:2]==0,1),:]
Sample run -
In [89]: meta
Out[89]:
array([[ 1, 2, 3, 4],
[ 0, 0, 2, 0],
[ 9, 0, 11, 12]])
In [90]: X
Out[90]: 4
In [91]: meta[meta[:,2]<X * np.all(meta[:,0:2]==0,1),:]
Out[91]: array([[0, 0, 2, 0]])
Using np.array instead of np.matrix allows you to do a simple mask indexing like:
a = a[a[:, 2] != 0]
to convert from np.matrix to np.array you can do:
a = np.asarray(a)
I've just learnt about the np.where function and applied on this task. This solution preserves the type of the original data.
a = np.matrix([[5, 3, 1],
[4, 4, 1],
[6, 4, 1],
[8, 2, 1],
[3, 5, 1],
[1, 7, 1],
[5, 4, 1],
[0, 1, 0],
[2, 0, 0]])
row_ndx,_ = np.where(a[:,2]>0)
a[row_ndx]
or a oneliner:
a[np.where(a[:,2]>0)[0]]
You can use the NumPy-based library, Pandas, which has a more generally useful implementation of ndarrays:
>>> # import the library
>>> import pandas as PD
Create some sample data as python dictionary, whose keys are the column names and whose values are the column values as a python list; one key/value pair per column
>>> data = {'category_code': ['D', 'A', 'B', 'C', 'D', 'A', 'C', 'A'],
'value':[4, 2, 6, 3, 8, 4, 3, 9]}
>>> # convert to a Pandas 'DataFrame'
>>> D = PD.DataFrame(data)
To return just the rows in which the category_code is either B or C, two steps conceptually, but can easily be done in a single line:
>>> # step 1: create the index
>>> idx = (D.category_code== 'B') | (D.category_code == 'C')
>>> # then filter the data against that index:
>>> D.ix[idx]
category_code value
2 B 6
3 C 3
6 C 3
Note the difference between indexing in Pandas versus NumPy, the library upon which Pandas is built. In NumPy, you would just place the index inside the brackets, indicating which dimension you are indexing with a ",", and using ":" to indicate that you want all of the values (columns) in the other dimension:
>>> D[idx,:]
In Pandas, you call the the data frame's ix method, and place only the index inside the brackets:
>>> D.loc[idx]
If you can choose, I strongly recommend pandas: it has "column indexing" built-in plus a lot of other features. It is built on numpy.