Imagine your have five different classes e.g. ['cat', 'dog', 'fish', 'bird', 'ant']. If you would use one-hot-encoding you would represent the presence of 'dog' in a five-dimensional binary vector like [0,1,0,0,0]. If you would use multi-hot-encoding you would first label-encode your classes, thus having only a single number which represents the presence of a class (e.g. 1 for 'dog') and then convert the numerical labels to binary vectors of size 
.
Examples:
'cat' = [0,0,0]
'dog' = [0,0,1]
'fish' = [0,1,0]
'bird' = [0,1,1]
'ant' = [1,0,0]
This representation is basically the middle way between label-encoding, where you introduce false class relationships (0 < 1 < 2 < ... < 4, thus 'cat' < 'dog' < ... < 'ant') but only need a single value to represent class presence and one-hot-encoding, where you need a vector of size 
(which can be huge!) to represent all classes but have no false relationships.
Note: multi-hot-encoding introduces false additive relationships, e.g. [0,0,1] + [0,1,0] = [0,1,1] that is 'dog' + 'fish' = 'bird'. That is the price you pay for the reduced representation.
Imagine your have five different classes e.g. ['cat', 'dog', 'fish', 'bird', 'ant']. If you would use one-hot-encoding you would represent the presence of 'dog' in a five-dimensional binary vector like [0,1,0,0,0]. If you would use multi-hot-encoding you would first label-encode your classes, thus having only a single number which represents the presence of a class (e.g. 1 for 'dog') and then convert the numerical labels to binary vectors of size .
Examples:
'cat' = [0,0,0]
'dog' = [0,0,1]
'fish' = [0,1,0]
'bird' = [0,1,1]
'ant' = [1,0,0]
This representation is basically the middle way between label-encoding, where you introduce false class relationships (0 < 1 < 2 < ... < 4, thus 'cat' < 'dog' < ... < 'ant') but only need a single value to represent class presence and one-hot-encoding, where you need a vector of size (which can be huge!) to represent all classes but have no false relationships.
Note: multi-hot-encoding introduces false additive relationships, e.g. [0,0,1] + [0,1,0] = [0,1,1] that is 'dog' + 'fish' = 'bird'. That is the price you pay for the reduced representation.
The accepted answer seems rather eccentric to me. I think that is rarely done, if ever, and will usually yield bad results.
There's a much more common, sensible use case for this. "Multi-hot encoding" doesn't seem to be a standard term, but I'm not sure there's any standard term. scikit-learn refers to a multi label binarizer.
This is simply used for multi label problems. That is, problems where more than one label can be associated with each example.
For example, say you are trying to detect whether certain types of animal are in a photo. Note that multiple types of animal can be in a single photo. Say the possible types of animal are ['cat', 'dog', 'fish', 'bird', 'ant']. A photo containing cats and dogs would be represented as [1, 1, 0, 0, 0].
I’m training a sentiment classification model of sorts that takes as input a sentence and then outputs however many categories it thinks that sentence belongs to. Often times it will only belong to one category, maybe “happy” or “sad,” but sometimes it could belong to several, like “happy, excited, nervous, etc.”
Right now I have the data in a pandas dataframe with the sentences in one column and then the categories that those sentences belong to in another column separated by commas:
SENTENCES SENTIMENTS
sentence 1 || sentiment 1, sentiment 2
sentence 2 || sentiment 1
sentence 3 || sentiment 2, sentiment 3, sentiment 6, sentiment 7
sentence 4 || sentiment 4, sentiment 5
I was able to find info on one-hot encoding, but I cannot for the life of me figure out how to multi-hot encode the sentiments column.
Does anyone know of a way to multi-hot encode a column in a Pandas dataframe?
Surely this is a common thing?
(Sorry for the bad representations of the data. I’m on my phone and haven’t yet figured out how to insert code into a post using the app.)
Edit: formatting
I have a pandas data series of strings that each have a bunch of text symbols in them (I'll call them words for discussion's sake, but in my use case they aren't actually words). The series of strings is already parsed to give me a vocabulary of all the words found anywhere in the series.
I've taken that vocabulary (vocab1, a list of all the words in the vocabulary) and made a dict (vocab) to assign an index to each word:
vocab = {c:i for i,c in enumerate(vocab1)}I need to change these strings into a multi-hot encoded numpy array (for machine learning blah blah). The arrays are the size of the vocabulary. 1 at a given position in the array indicates the absence of the corresponding word in the string, while 0 indicates its absence. The strings typically only have a few words in them (which sometimes are redundantly repeated in the original data) but there are 30k different words in the vocabulary, and in the broader use case this goes up to 260k or so.
Anyway, vocablength is the number of words in the vocabulary. I'm using pandas.Series.transform to apply my function to the entire series, which is over 200k strings (and will be millions in the broader use case).
def multihot_codes(codes):
o = np.zeros((vocablength,),dtype=np.int32)
for k in codes.split():
o[vocab[k]] = 1
return o
df["codesmh"] = df["codess"].transform(multihot_codes)This takes close to forever and has significant disk usage. There's probably a faster way to do this. For example, Tensorflow generates binary one-hot vectors for each word and then does a row reduce on them, but if you dig into the code, they have a compiled C++ module that does the heavy lifting. Not exactly what I'm prepared to do here myself....
Any ideas on how to improve this, or if there's something already in numpy/scipy or another package that's suited for use here? Thanks in advance!
Use get_dummies with DataFrame.set_index and aggregate max or sum:
#always 0,1 in output
df1 = pd.get_dummies(df.set_index('Name')['Class']).max(level=0).reset_index()
#if need count values
#df1 = pd.get_dummies(df.set_index('Name')['Class']).sum(level=0).reset_index()
print (df1)
Name FB GRS TWT
0 Aci 1 1 0
1 Dan 1 0 1
2 Ann 0 1 0
The other solutions were failing for me due to memory overflows when I have 315 samples and 1908 labels. Here are some more performant methods.
Using pandas pivot:
def multihotencode(data, samples_col, labels_col):
data = data.copy()
data['present'] = 1
multihot = data.pivot(index=samples_col, columns=labels_col, values='present')
return multihot.fillna(0).astype(int)
Using sklearn:
import pandas as pd
from sklearn.preprocessing import MultiLabelBinarizer
def multihotencode(data, samples_col, labels_col):
## Aggregate the labels into a tuple for each sample
tupled = data.groupby(samples_col).apply(lambda x: tuple(x[labels_col].to_numpy()))
## Get multi-hot encoded matrix.
mlb = MultiLabelBinarizer()
multihot = mlb.fit_transform(samplepgs)
## Reapply the index and column headers
return pd.DataFrame(multihot, index=tupled.index, columns=mlb.classes_)
Both methods give the same result:
df = pd.DataFrame([
['Aci', 'FB'],
['Dan', 'TWT'],
['Ann', 'GRS'],
['Aci', 'GRS'],
['Dan', 'FB']
], columns=['Name', 'Class'])
multihotencode(df, 'Name', 'Class')
>>>
FB GRS TWT
Name
Aci 1 1 0
Ann 0 1 0
Dan 1 0 1
If I understand Multi Hot encoding correctly basically we take arrays of different lengths, and turn them into arrays/lists of all the same length with each index of the new array/list being on(0) or off(1) for the various entries in the original array.
so
[1,3] would become [0, 1, 0, 1]
My question is, how do we account for if a number appears twice in the original array/list,
for example [1,3,3]
Your original approach, without one-hot encoding, was doing what you wanted.
One-hot encoding is meant for inputs to many models, but outputs for only a few (e.g. training a neural network with cross-entropy loss). So these are only needed for some algorithm implementations, while others can do fine without it.
For output labels, a classifier like RandomForest is just fine with strings and multiple classes.
You don't have to make one hot encoding when using random forest in sklearn.
What you need is "label encoder", and your Y should looks like
from sklearn.preprocessing import LabelEncoder
y = ["A","B","D","A","C"]
le = LabelEncoder()
le.fit_transform(y)
# array([0, 1, 3, 0, 2], dtype=int64)
I tried to modified the sample code sklearn provided :
from sklearn.ensemble import RandomForestClassifier
import numpy as np
from sklearn.datasets import make_classification
>>> X, y = make_classification(n_samples=1000, n_features=4,
... n_informative=2, n_redundant=0,
... random_state=0, shuffle=False)
y = np.random.choice(["A","B","C","D"],1000)
print(y.shape)
>>> clf = RandomForestClassifier(max_depth=2, random_state=0)
>>> clf.fit(X, y)
>>> clf.classes_
# array(['A', 'B', 'C', 'D'], dtype='<U1')
Either process the y with label encoding or without, it both worked with RandomForestClassifier.