Note that I have implemented new cut and qcut functions for discretizing continuous data:
http://pandas-docs.github.io/pandas-docs-travis/basics.html#discretization-and-quantiling
For labeled columns like the a and c column in your example you can use the pandas build-in method get_dummies().
Ex.:
import pandas as pd
s1 = ['a', 'b', np.nan]
pd.get_dummies(s1)
a b
0 1 0
1 0 1
2 0 0
OK I just hacked this quickly, so this uses np.array_split so that for non-equal sized bins it doesn't barf, this sorts the data first and then performs the calculations to split and return the cutoffs:
import random
import numpy as np
dat = np.arange(1,13)/2.0
def discretize(data, bins):
split = np.array_split(np.sort(data), bins)
cutoffs = [x[-1] for x in split]
cutoffs = cutoffs[:-1]
discrete = np.digitize(data, cutoffs, right=True)
return discrete, cutoffs
discrete_dat, cutoff = discretize(dat, 3)
print "dat: {}".format(dat)
print "discrete_dat: {}".format(discrete_dat)
print "cutoff: {}".format(cutoff)
>> dat: [ 0.5 1. 1.5 2. 2.5 3. 3.5 4. 4.5 5. 5.5 6. ]
>> discrete_dat: [0 0 0 0 1 1 1 1 2 2 2 2]
>> cutoff: [2.0, 4.0]
pandas.qcut does exactly that.
>>>pd.qcut(range(5), 4, labels=False)
array([0, 0, 1, 2, 3]) 3])
Update (Sep 2018): As of version 0.20.0, there is a function, sklearn.preprocessing.KBinsDiscretizer, which provides discretization of continuous features using a few different strategies:
- Uniformly-sized bins
- Bins with "equal" numbers of samples inside (as much as possible)
- Bins based on K-means clustering
Unfortunately, at the moment, the function does not accept custom intervals (which is a bummer for me as that is what I wanted and the reason I ended up here). If you want to achieve the same, you can use Pandas function cut:
import numpy as np
import pandas as pd
n_samples = 10
a = np.random.randint(0, 10, n_samples)
# say you want to split at 1 and 3
boundaries = [1, 3]
# add min and max values of your data
boundaries = sorted({a.min(), a.max() + 1} | set(boundaries))
a_discretized_1 = pd.cut(a, bins=boundaries, right=False)
a_discretized_2 = pd.cut(a, bins=boundaries, labels=range(len(boundaries) - 1), right=False)
a_discretized_3 = pd.cut(a, bins=boundaries, labels=range(len(boundaries) - 1), right=False).astype(float)
print(a, '\n')
print(a_discretized_1, '\n', a_discretized_1.dtype, '\n')
print(a_discretized_2, '\n', a_discretized_2.dtype, '\n')
print(a_discretized_3, '\n', a_discretized_3.dtype, '\n')
which produces:
[2 2 9 7 2 9 3 0 4 0]
[[1, 3), [1, 3), [3, 10), [3, 10), [1, 3), [3, 10), [3, 10), [0, 1), [3, 10), [0, 1)]
Categories (3, interval[int64]): [[0, 1) < [1, 3) < [3, 10)]
category
[1, 1, 2, 2, 1, 2, 2, 0, 2, 0]
Categories (3, int64): [0 < 1 < 2]
category
[1. 1. 2. 2. 1. 2. 2. 0. 2. 0.]
float64
Note that, by default, pd.cut returns a pd.Series object of dtype Category with elements of type interval[int64]. If you specify your own labels, the dtype of the output will still be a Category, but the elements will be of type int64. If you want the series to have a numeric dtype, you can use .astype(np.int64).
My example uses integer data, but it should work just as fine with floats.
The answer is no. There is no binning in scikit-learn. As eickenberg said, you might want to use np.histogram. Features in scikit-learn are assumed to be continuous, not discrete. The main reason why there is no binning is probably that most of sklearn is developed on text, image featuers or dataset from the scientific community. In these settings, binning is rarely helpful. Do you know of a freely available dataset where binning is really beneficial?