You could use this code snippet to transform your class indices into a one-hot encoded target: target = torch.randint(0, 10, (10,)) one_hot = torch.nn.functional.one_hot(target) Answer from ptrblck on discuss.pytorch.org
🌐
GitHub
gist.github.com › NegatioN › acbd8bb6be866ce1831b2d073fd7c450
PyTorch Multi-dimensional One hot encoding · GitHub
PyTorch Multi-dimensional One hot encoding. GitHub Gist: instantly share code, notes, and snippets.
🌐
PyTorch Forums
discuss.pytorch.org › nlp
PyTocrh way for one-hot-encoding multiclass target variable - nlp - PyTorch Forums
February 1, 2020 - Hey, Sorry for maybe super basic question but could not find it. What is a correct Pytorch way to encode multi-class target variable? I have > 30 target classes for target variable - like AA, AB, BB, BA, BC .... Should I use ScikitLearn tools and then convert numpy arrays into torch tensors?
Discussions

One hot encoding for multi label classification using BCEWithLogitsLoss() loss
I am using resnet18 with BCEWithLogitsLoss() and i am encoding my labels using y_onehot = nn.functional.one_hot(labels, num_classes=3) y_onehot = y_onehot.float() Which is I think not true for multi label data. How should I encode my labels to get multi labels More on discuss.pytorch.org
🌐 discuss.pytorch.org
3
1
May 1, 2020
How to reverse a multi-hot encoding
I have my data encoded as multi-hot vectors for a multi-label classification task (4-classes in this example): multihot_batch = torch.tensor([[0,1,0,1], [0,0,0,1], [0,0,1,1]]) How can I undo this encoding and have each entry be a list of the classes present, like this: tensor([[1, 3], [3], ... More on discuss.pytorch.org
🌐 discuss.pytorch.org
1
0
February 18, 2021
Batched, Sequential, Multi-hot Vectors
Hi all, I am looking for an idiomatic way to make batched multi-hot vectors (multi-hot being like a one-hot but several can be hot). For example, we might have [[1. 0. 1. 0. 0. 0.] [0. 1. 0. 0. 1. 0.] [0. 1. 0. 0. 0. 1.]] as a sequence of three 2-hot vectors generated from the sequence [a b ... More on discuss.pytorch.org
🌐 discuss.pytorch.org
2
0
May 23, 2020
How to cover a label list under the multi-label classification context into one-hot encoding with pytorch? - Stack Overflow
I have a list of one batch data with multi-label for every sample. So how to covert it into torch.Tensor in one-hot encoding? More on stackoverflow.com
🌐 stackoverflow.com
🌐
PyTorch
docs.pytorch.org › api reference › torchrl.data package › tensorspec system › multionehot
MultiOneHot — torchrl main documentation
January 1, 2023 - This class can be used when a single tensor must carry information about multiple one-hot encoded values.
🌐
PyTorch Forums
discuss.pytorch.org › t › one-hot-encoding-for-multi-label-classification-using-bcewithlogitsloss-loss › 79262
One hot encoding for multi label classification using BCEWithLogitsLoss() loss - PyTorch Forums
May 1, 2020 - I am using resnet18 with BCEWithLogitsLoss() and i am encoding my labels using y_onehot = nn.functional.one_hot(labels, num_classes=3) y_onehot = y_onehot.float() Which is I think not true for multi label da…
🌐
PyTorch Forums
discuss.pytorch.org › t › how-to-reverse-a-multi-hot-encoding › 112221
How to reverse a multi-hot encoding - PyTorch Forums
February 18, 2021 - I have my data encoded as multi-hot vectors for a multi-label classification task (4-classes in this example): multihot_batch = torch.tensor([[0,1,0,1], [0,0,0,1], [0,0,1,1]]) How can I undo this encoding and have each entry be a list of the classes present, like this: tensor([[1, 3], [3], [2, 3]]) torch.argmax only returns the index of the first max: torch.argmax(multihot_batch, dim=1, keepdim=True) tensor([[1], [3], [2]]) This approach gives them all, but...
🌐
PyTorch Forums
discuss.pytorch.org › t › batched-sequential-multi-hot-vectors › 82574
Batched, Sequential, Multi-hot Vectors - PyTorch Forums
May 23, 2020 - Hi all, I am looking for an idiomatic way to make batched multi-hot vectors (multi-hot being like a one-hot but several can be hot). For example, we might have [[1. 0. 1. 0. 0. 0.] [0. 1. 0. 0. 1. 0.] [0. 1. 0. 0. 0. 1.]] as a sequence of three 2-hot vectors generated from the sequence [a b ...
Find elsewhere
Top answer
1 of 2
3

If the batch size can be derived from len(labels):

def to_onehot(labels, n_categories, dtype=torch.float32):
    batch_size = len(labels)
    one_hot_labels = torch.zeros(size=(batch_size, n_categories), dtype=dtype)
    for i, label in enumerate(labels):
        # Subtract 1 from each LongTensor because your
        # indexing starts at 1 and tensor indexing starts at 0
        label = torch.LongTensor(label) - 1
        one_hot_labels[i] = one_hot_labels[i].scatter_(dim=0, index=label, value=1.)
    return one_hot_labels

and you have 6 categories and want the output to be a tensor of integers:

to_onehot(labels, n_categories=6, dtype=torch.int64)
tensor([[1, 1, 1, 0, 0, 0],
        [0, 0, 0, 1, 0, 1],
        [1, 0, 0, 0, 0, 0],
        [1, 0, 0, 1, 1, 0],
        [0, 0, 0, 1, 0, 0]])

I would stick to torch.float32 in case you want to use label smoothing, mix-up or something along those lines later.

2 of 2
0

To handle any situation (include string labels) I've extended @karniol's answer:

def multihot_encoder(labels, dtype=torch.float32):
    """ Convert list of label lists into a 2-D multihot Tensor """
    label_set = set()
    for label_list in labels:
        label_set = label_set.union(set(label_list))
    label_set = sorted(label_set)

    multihot_vectors = []
    for label_list in labels:
        multihot_vectors.append([1 if x in label_list else 0 for x in label_set])

    # To keep track of which columns are which, set dtype to None and...
    # import pandas as pd
    if dtype is None:
        return pd.DataFrame(multihot_vectors, columns=label_set)
    return torch.Tensor(multihot_vectors).to(dtype)

Your use case:

label_lists = [[1,2,3], [4,6], [1], [1,4,5], [4]]
>>> multihot_encoder(label_lists)
tensor([[1., 1., 1., 0., 0., 0.],
        [0., 0., 0., 1., 0., 1.],
        [1., 0., 0., 0., 0., 0.],
        [1., 0., 0., 1., 1., 0.],
        [0., 0., 0., 1., 0., 0.]])

If you want to keep track of your labels (feature names) before converting your dataset to a Tensor, just set dtype to None:

label_lists = [
    ['happy', 'kind'], ['sad', 'mean'],
    ['loud', 'happy'], ['quiet', 'kind']
    ]
multihot_encoder(label_lists, dtype=None)
   happy  kind  loud  mean  quiet  sad
0      1     1     0     0      0    0
1      0     0     0     1      0    1
2      1     0     1     0      0    0
3      0     1     0     0      1    0

and you have 6 categories and want the output to be a tensor of integers:

to_onehot(labels, n_categories=6, dtype=torch.int64)
tensor([[1, 1, 1, 0, 0, 0],
        [0, 0, 0, 1, 0, 1],
        [1, 0, 0, 0, 0, 0],
        [1, 0, 0, 1, 1, 0],
        [0, 0, 0, 1, 0, 0]])
🌐
PyTorch Forums
discuss.pytorch.org › t › one-hot-encoding-of-multi-class-mask › 120170
One-hot encoding of multi-class mask - PyTorch Forums
May 4, 2021 - Hello, I am creating one-hot encoded labels from 3D Mask (with HxWxD shape) by the following code: seg_3d = nib.load(seg_file) seg = seg_3d.get_data() # fixme labels =np.unique(seg) # [ 0 1 2 3 4 8 10 11 56] num_labels = len(labels) # 9 segD = np.zeros((num_labels, seg.shape[0], seg.shape[1], seg.shape[2])) for i in range(1, num_labels): # this loop starts from label 1 to ignore background 0 segD[i, :, :, :] = seg == labels[i] Are there better and efficient ways to do that?
🌐
PyTorch Forums
discuss.pytorch.org › vision
What kind of loss to use with multi-hot encoded multiclass ordinal features? - vision - PyTorch Forums
June 17, 2024 - I have some ordinal classes with a range of 1-5 each and some implicit relationships, and decided to multi-hot encode those such that each level gets an activation like { 1: [0, 0, 0, 0] 2: [1, 0, 0, 0] ... 5: [1, 1, 1, 1] } And then concatenate ...
🌐
PyTorch
docs.pytorch.org › rl › 0.8 › reference › generated › torchrl.data.MultiOneHot.html
MultiOneHot — torchrl 0.8 documentation
This class can be used when a single tensor must carry information about multiple one-hot encoded values.
🌐
PyTorch Forums
discuss.pytorch.org › t › how-can-i-use-multi-hot-embedding-efficiently › 96361
How can I use multi-hot embedding efficiently? - PyTorch Forums
September 15, 2020 - I want use embedding trick to encode my multi-hot vector ,such as [1,1,0,1,0,0,0,0],[1,0,0,0,0,1,0]. However, I find there is not multi-hot embedding in Pytorch.I writed a function to achieve multi-hot embedding,but it is used in for-loop,so it is so slow and time-consuming.
🌐
PyTorch Forums
discuss.pytorch.org › vision
One-hot encoding multi-label predicting all zeros - vision - PyTorch Forums
August 13, 2021 - Hi, I’m trying to implement a multi-label classification model starting from resnet18, In my dataset I have 188 classes and each element can belong to one, two or three classes, I’m using one hot encoding. So about 99…
🌐
PyTorch Forums
discuss.pytorch.org › vision
One hot encode label for multi-label classification - vision - PyTorch Forums
May 18, 2021 - Dear all, im try to prepare dataset for multi-label classification with pytorch, there is an example with pytorch (dataloader) for multi-label classification? thanks 🙂
🌐
PyTorch Forums
discuss.pytorch.org › t › one-hot-encoding-of-multi-class-mask › 120170 › 2
One-hot encoding of multi-class mask - #2 by tom - PyTorch Forums
May 4, 2021 - These days there is a torch.nn.functional.one_hot that does what you need except that the label dimension is last, but one_hot + permute will get you what you need. You can (almost) always use torch.from_numpy(array) and tensor.numpy() to get from and to numpy efficiently. Best regards Thomas
🌐
Reddit
reddit.com › r/learnmachinelearning › pytorch sanity check: do i need a special activation function for multiclass targets?
r/learnmachinelearning on Reddit: PyTorch sanity check: do I need a special activation function for multiclass targets?
March 9, 2024 -

It's been a while since I've worked with deep learning and I can't tell why this seems so unintuitive. I have a simple NN where the final layer has 16 nodes passed through a sigmoid incoming and should output an integer target 0-4.

If I let the output dimension be 5, i.e. probabilities of being each of the 5 categories, then the provided boilerplate code won't work because it uses CrossEntropyLoss which accepts only a 0D, 1D Tensor.

If I let the output dimension be 1, i.e. a number representing the class it most likely is, I'd need some obscure activation function to map the output to the integers 0 to 4, wouldn't I? And while possible, it doesn't seem like it would be part of the ask considering how simple this is meant to be.

Top answer
1 of 3
4
The standard treatment of this problem is to use 5 output values, which are Softmax activated. Because you're doing a multi-class problem, presumably each sample belongs to exactly 1-of-5 classes. This is not just a Crossentropy problem, but a CategoricalCrossentropy problem. Using simple CrossEntorpy would imply that these are 5 independent True/False problems, where any of those 5 values can be True simultaneously. It would usually be expected that your target data is the same shape as your output. However, with one-hot encoding this can be wasteful of bandwidth and memory, so it is common to leave your target data as it is currently encoded. However, to have your target data understood correctly by the loss function, you must use SparseCategoricalCrossentropy. As a note, the 0D and 1D restriction you seemed to get from Crossentropy was actually due to the shape of your target data, not the Crossentropy loss itself. This is all tensorflow terminology so there may be some gaps or translation necessary for pytorch.
2 of 3
1
For multi-class classification, the model predicts a probability in range [0, 1] for each class. There are a few cases, depending on the problem. Only one correct class per example: apply a softmax function to ensure that probabilities over classes sum to 1. Possibly many correct classes per example: apply sigmoid activation and set the activation threshold to 0.5 for each class. Cross-entropy is a notion of difference between two probability distributions. Model outputs are compared to a one-hot vector. If the class at the third index is correct, the label vector is [0, 0, 1, 0, 0]. After applying softmax, the predictions might look like [0.05, 0.1, 0.55, 0.05, 0.25]. PyTorch's CrossEntropyLoss takes the unnormalized predictions (logits) and applies softmax for you. BCEWithLogitsLoss takes the unnormalized predictions and applies sigmoid for you. It uses binary cross-entropy because each class has its own (binary) distribution.
🌐
PyTorch Forums
discuss.pytorch.org › t › activation-and-loss-function-for-multi-dimensional-one-hot-encoded-output › 97987
Activation and loss function for multi dimensional one hot encoded output - PyTorch Forums
October 2, 2020 - I have a multi dimensional output model with the shape of (B,C,T) before the softmax layer. Its target is a row wise one hot encoded matrix with the same shape of model prediction ie (B,C,T) . The trouble is PyTorch sof…
🌐
Towards Data Science
towardsdatascience.com › home › latest › building autoencoders on sparse, one hot encoded data
Building Autoencoders on Sparse, One Hot Encoded Data | Towards Data Science
January 17, 2025 - We start with two functions: The Encoder Model, and the Decoder Model. Both 'models' are wrapped into a class called Network, which will encompass the entire system for our training and evaluation. Lastly, we define a function Forward, which is what PyTorch uses as the entryway into the Network that wraps both the encoding and the decoding of the data.