Imagine your have five different classes e.g. ['cat', 'dog', 'fish', 'bird', 'ant']. If you would use one-hot-encoding you would represent the presence of 'dog' in a five-dimensional binary vector like [0,1,0,0,0]. If you would use multi-hot-encoding you would first label-encode your classes, thus having only a single number which represents the presence of a class (e.g. 1 for 'dog') and then convert the numerical labels to binary vectors of size .

Examples:

'cat'  = [0,0,0]  
'dog'  = [0,0,1]  
'fish' = [0,1,0]  
'bird' = [0,1,1]  
'ant'  = [1,0,0]   

This representation is basically the middle way between label-encoding, where you introduce false class relationships (0 < 1 < 2 < ... < 4, thus 'cat' < 'dog' < ... < 'ant') but only need a single value to represent class presence and one-hot-encoding, where you need a vector of size (which can be huge!) to represent all classes but have no false relationships.

Note: multi-hot-encoding introduces false additive relationships, e.g. [0,0,1] + [0,1,0] = [0,1,1] that is 'dog' + 'fish' = 'bird'. That is the price you pay for the reduced representation.

You can think of binary encoding as a compromise between label encoding and one-hot encoding. For distinct categories, label encoding introduces a false linear order that brings a lot of noise into the model (category 1 < category 2 < category 3....) . Binary encoding introduces false additive relationships between the categories (e.g. category 4 + category 1 = category 5 or 100 + 001 = 101) but fewer of them.

Therefore, binary will usually work better than label encoding, however only one-hot encoding will usually preserve the full information in the data.

Unless your algorithm (or computing power) is limited in the number of categories it can handle, one-hot encoding will be preferred over other encoding schemes. If you are limited, mean encoding is a powerful alternative because it transform a categorical feature into a numeric one (giving you the minimal number of inputs) while preserving the most important information in the data.

[Mean encoding replaces every category with its target mean. The mean encodings need to be constructed carefully on a separate dataset to avoid data leakage. If you want to reduce your input dimensions as much as possible, this can be helpful. Mean encoding can also be helpful as an additional feature and is very popular on Kaggle to squeeze out some extra performance. This is also known as target encoding or likelihood encoding.]

Without having read the paper, from the name I would say a multi-hot encoding is a matrix , with .

You obtain the normal hot-encoding by setting .

Without using MultiLabelBinarizer

import numpy as np
classes=['comedy', 'xyz','thriller', 'sci-fi']
id_dict = {c:i for i,c in enumerate(classes)}
lists_of_movieIds = [{'sci-fi', 'thriller'}, {'comedy'}]
list_of_ratings = [[4,3],[5]]

data = np.zeros((len(lists_of_movieIds), len(classes)))
for i, (m_ids,rs) in enumerate(zip(lists_of_movieIds, list_of_ratings)):
  for m_id,r in zip(m_ids,rs):
    data[i, id_dict[m_id]] = r

print (data)

Output:

[[0. 0. 3. 4.]
 [5. 0. 0. 0.]]

Here’s a data.table solution in case anyone is interested:

library(data.table)
bf <- data.table(crops = c(1, 3, 345, 9562))

bf[, crops := strsplit(as.character(crops), "")]
cols <- sort(unique(unlist(bf$crops)))
bf[, (cols) := lapply(cols, \(col) sapply(crops, \(row) col %in% row))]
bf

##      crops     1     2     3     4     5     6     9
## 1:       1  TRUE FALSE FALSE FALSE FALSE FALSE FALSE
## 2:       3 FALSE FALSE  TRUE FALSE FALSE FALSE FALSE
## 3:   3,4,5 FALSE FALSE  TRUE  TRUE  TRUE FALSE FALSE
## 4: 9,5,6,2 FALSE  TRUE FALSE FALSE  TRUE  TRUE  TRUE

And a Base R solution:

bf <- data.frame(crops = c(1, 3, 345, 9562))
bf$crops <- strsplit(as.character(bf$crops), "")
cols <- sort(unique(unlist(bf$crops)))
for (col in cols) {
    bf[[col]] <- sapply(bf$crops, \(row) col %in% row)
}
bf

##        crops     1     2     3     4     5     6     9
## 1          1  TRUE FALSE FALSE FALSE FALSE FALSE FALSE
## 2          3 FALSE FALSE  TRUE FALSE FALSE FALSE FALSE
## 3    3, 4, 5 FALSE FALSE  TRUE  TRUE  TRUE FALSE FALSE
## 4 9, 5, 6, 2 FALSE  TRUE FALSE FALSE  TRUE  TRUE  TRUE