Your original approach, without one-hot encoding, was doing what you wanted.

One-hot encoding is meant for inputs to many models, but outputs for only a few (e.g. training a neural network with cross-entropy loss). So these are only needed for some algorithm implementations, while others can do fine without it.

For output labels, a classifier like RandomForest is just fine with strings and multiple classes.

Your code processes symbols instead of words.

Fixes

# classes = np.unique(list(itertools.chain.from_iterable(y)))
classes = np.unique(y)

# for class_ in item:
#     y_one_hot[i][self.class_to_index[class_]] = 1
y_one_hot[i][self.class_to_index[item]] = 1

Also, take a look at sklearn.preprocessing.OneHotEncoder

from sklearn.preprocessing import OneHotEncoder

label_encoder = OneHotEncoder(sparse=False)
label_encoder.fit(y.to_frame())
label_encoder.transform(y.to_frame())

So now I am totally confused how people use one hot encoding for target variable in practice? please help me out.

This mostly comes down to the tool you are using. Sklearn, which I assume you are using, does not use one hot encoded target variables. So your y should be of dimension (1600, 1) where the classes are 0, 1, 2 and 3. Instead of applying one hot encoding you can use a LabelEncoder to get it on the correct format.

I suspect the reason for your confusion comes from having seen deep learning frameworks such as Tensorflow and Keras. With them you always one hot encode your target variable.

Short answer:

• Using sklearn: Label encode
• Using deep learning: one hot encode

I think you might be confusing a multiclass (your case) with a multioutput classification.

In multiclass classification problems, your output should only be a single target column, and you'll be training the model to classify among the classes in that column. You'd have to split into separate target columns, in the case you had to predict n different classes per sample, which is not the case, you only want one of the targets per sample.

So for multiclass classification, there's no need to OneHotEncode the target, since you only want a single target column (which can also be categorical in SVC). What you do have to encode, either using OneHotEncoder or with some other encoders, is the categorical input features, which have to be numeric.

Also, SVC can deal with categorical targets, since it LabelEncode's them internally:

from sklearn.datasets import load_iris
from sklearn.svm import SVC
from sklearn.model_selection import train_test_split

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y)
y_train_categorical = load_iris()['target_names'][y_train]
# array(['setosa', 'setosa', 'versicolor',...

sv = SVC()
sv.fit(X_train, y_train_categorical)
sv.classes_
# array(['setosa', 'versicolor', 'virginica'], dtype='<U10')

Imagine your have five different classes e.g. ['cat', 'dog', 'fish', 'bird', 'ant']. If you would use one-hot-encoding you would represent the presence of 'dog' in a five-dimensional binary vector like [0,1,0,0,0]. If you would use multi-hot-encoding you would first label-encode your classes, thus having only a single number which represents the presence of a class (e.g. 1 for 'dog') and then convert the numerical labels to binary vectors of size .

Examples:

'cat'  = [0,0,0]  
'dog'  = [0,0,1]  
'fish' = [0,1,0]  
'bird' = [0,1,1]  
'ant'  = [1,0,0]   

This representation is basically the middle way between label-encoding, where you introduce false class relationships (0 < 1 < 2 < ... < 4, thus 'cat' < 'dog' < ... < 'ant') but only need a single value to represent class presence and one-hot-encoding, where you need a vector of size (which can be huge!) to represent all classes but have no false relationships.

Note: multi-hot-encoding introduces false additive relationships, e.g. [0,0,1] + [0,1,0] = [0,1,1] that is 'dog' + 'fish' = 'bird'. That is the price you pay for the reduced representation.

You may need to try cat2vec which converts categorical features into vector representation using Word2Vec approach. Check also this link for multi-feature inputs into LSTM.

For the target y, one-hot is a better technique for NN-based models.

The MultiLabelBinarizer class allow to do one-hot encoding on multilabel sets, like you have in column b:

print (df)
                 a                        b
0  2007_000027.jpg               {'person'}
1  2007_000032.jpg  {'aeroplane', 'person'}
2  2007_000033.jpg            {'aeroplane'}
3  2007_000039.jpg            {'tvmonitor'}
4  2007_000042.jpg                {'train'}

from sklearn.preprocessing import MultiLabelBinarizer

mlb = MultiLabelBinarizer()
df = pd.DataFrame(mlb.fit_transform(df['b']),columns=mlb.classes_)
print (df)
   aeroplane  person  train  tvmonitor
0          0       1      0          0
1          1       1      0          0
2          1       0      0          0
3          0       0      0          1
4          0       0      1          0

Or Series.str.join with Series.str.get_dummies, but it should be slower in large DataFrame:

df = df['b'].str.join('|').str.get_dummies()
print (df)

   aeroplane  person  train  tvmonitor
0          0       1      0          0
1          1       1      0          0
2          1       0      0          0
3          0       0      0          1
4          0       0      1          0