Hi, Backprop-wise, they will give the exact same result. The difference is that if the original dimension you want to expand is of size 1, you can use torch.expand() to do it without using extra memory. If the dimension you want to expand is of size more than 1, then you actually want to repeat w… Answer from albanD on discuss.pytorch.org
🌐
PyTorch Forums
discuss.pytorch.org › autograd
Torch.repeat and torch.expand which to use? - autograd - PyTorch Forums
October 24, 2018 - I have one vector and one 2-D vector. Say embed1= [1, 2, 3] shape is (3) and embed2=[[1, 2, 3], [4,5,6], [7,8,9]] shape is (3, 3). Now I want to expand embed1 to [[1, 2, 3], [1, 2, 3], [1, 2, 3]] so that I can do torch.c…
🌐
PyTorch Forums
discuss.pytorch.org › uncategorized
Expand vs Repeat: Semantic Difference? - Uncategorized - PyTorch Forums
November 1, 2019 - Hi, How do the below two ways of extending a batch_size x feature_size tensor differ? visual_feature = visual_feat.expand([batch_size,batch_size,self.feature_size]) text_feature = sentence_embed.repeat(1,1,batch_size).view(batch_size,batch_size,self.feature_size)
🌐
GeeksforGeeks
geeksforgeeks.org › deep learning › repeating-tensors-in-a-specific-new-dimension-in-pytorch
Repeating Tensors in a Specific New Dimension in PyTorch - GeeksforGeeks
July 23, 2025 - Tensor repetition involves duplicating the elements of a tensor along one or more dimensions. This operation is analogous to the concept of tiling in NumPy and can be used to expand the size of a tensor by repeating its elements. PyTorch provides several functions for tensor repetition, including torch.repeat, torch.repeat_interleave, and torch.tile
🌐
Medium
medium.com › data-scientists-diary › how-to-perform-an-expand-operation-in-pytorch-bf1c07532dc1
How to Perform an Expand Operation in PyTorch? | by Amit Yadav | Data Scientist’s Diary | Medium
January 19, 2025 - Here’s the deal: repeat actually duplicates data, leading to increased memory usage, while expand creates a view that merely appears as expanded. Let’s break down the difference with an example using Python’s sys.getsizeof() to measure ...
🌐
Medium
medium.com › biased-algorithms › how-to-repeat-a-tensor-in-a-specific-new-dimension-in-pytorch-7519f4172065
How to Repeat a Tensor in a Specific New Dimension in PyTorch | by Amit Yadav | Biased-Algorithms | Medium
January 19, 2025 - expand vs. repeat: When memory efficiency matters, expand can save resources since it doesn’t duplicate data. tile for Readability: In some cases, torch.tile provides a cleaner alternative to repeat, especially for larger repetitions.
🌐
Stack Overflow
stackoverflow.com › questions › 67799518 › why-use-torch-repeat-in-place-of-for-loop-while-increasing-the-dimension-in-torc
pytorch - Why use torch.repeat in place of for loop while increasing the dimension in torch tensor? - Stack Overflow
And it's cleaner : using repeat makes it a self-explanatory 1-line code, whereas using the for-loop makes the code lose clarity. Final side note : if you can, use expand instead of repeat, the former is much more memory efficient since there ...
🌐
PyTorch Forums
discuss.pytorch.org › t › expand-vs-repeat-used-within-backward-raise-error › 87303
Expand vs repeat used within backward raise Error - PyTorch Forums
June 29, 2020 - Hi. I am implementing a Gaussian observation model for a multioutput task. The class I have implemented has the following init method. Basically the observation noise can be shared by the different outputs. class Gauss…
🌐
GitHub
gist.github.com › goddoe › 381a1cd1bdd66bc541aeda2d89f51bc2
repeat vs expand pytorch · GitHub
repeat vs expand pytorch. GitHub Gist: instantly share code, notes, and snippets.
Find elsewhere
Top answer
1 of 3
16

Note, that you need to decide whether you would like to allocate new memory for your expanded array or whether you simply require a new view of the existing memory of the original array.

In PyTorch, this distinction gives rise to the two methods expand() and repeat(). The former only creates a new view on the existing tensor where a dimension of size one is expanded to a larger size by setting the stride to 0. Any dimension of size 1 can be expanded to an arbitrary value without allocating new memory. In contrast, the latter copies the original data and allocates new memory.

In PyTorch, you can use expand() and repeat() as follows for your purposes:

import torch

L = 10
N = 20
A = torch.randn(L,L)
A.expand(N, L, L) # specifies new size
A.repeat(N,1,1) # specifies number of copies

In Numpy, there are a multitude of ways to achieve what you did above in a more elegant and efficient manner. For your particular purpose, I would recommend np.tile() over np.repeat(), since np.repeat() is designed to operate on the particular elements of an array, while np.tile() is designed to operate on the entire array. Hence,

import numpy as np

L = 10
N = 20
A = np.random.rand(L,L)
np.tile(A,(N, 1, 1))
2 of 3
4

If you don't mind creating new memory:

  • In numpy, you can use np.repeat() or np.tile(). With efficiency in mind, you should choose the one which organises the memory for your purposes, rather than re-arranging after the fact:
    • np.repeat([1, 2], 2) == [1, 1, 2, 2]
    • np.tile([1, 2], 2) == [1, 2, 1, 2]
  • In pytorch, you can use tensor.repeat(). Note: This matches np.tile, not np.repeat.

If you don't want to create new memory:

  • In numpy, you can use np.broadcast_to(). This creates a readonly view of the memory.
  • In pytorch, you can use tensor.expand(). This creates an editable view of the memory, so operations like += will have weird effects.
🌐
CodeGenes
codegenes.net › blog › pytorch-expand-vs-repeat
PyTorch Expand vs Repeat: A Comprehensive Guide — codegenes.net
In this example, we expand tensor a to shape (2, 3) using the expand method so that it can be added to tensor b. The repeat method is often used for data augmentation in deep learning.
🌐
Medium
medium.com › dejunhuang › learning-day-6-pytorch-unsqueeze-squeeze-transpose-and-broadcasting-88e75dc8facb
Learning Day 6: Pytorch unsqueeze, squeeze, transpose and broadcasting | by De Jun Huang | dejunhuang | Medium
April 24, 2021 - # repeat (memory copy) not recommended as it changes the memory print("\nrepeat (memory copy)") print(b.shape) # OUTPUT:torch.Size([1, 32, 1, 1]) print(b.repeat(4, 32, 1, 1).shape) # not the same as .expand # OUTPUT: torch.Size([4, 1024, 1, 1]) # 1024 = 32*32 print(b.repeat(4, 1, 1, 1).shape) # parameter means how many times to copy print(b.repeat(4, 1, 32, 32).shape)
🌐
Stack Overflow
stackoverflow.com › questions › 68528008 › pytorch-dimension-change
stack - Pytorch dimension change - Stack Overflow
expand will make it so the expanded dimensions refer (references not copies) to the original dimensions, which means the tensor takes the same amount of memory. repeat on the other hand will copy the data; In your case, going from (1, 512, 1, ...
🌐
DEV Community
dev.to › hyperkai › expand-in-pytorch-4e7m
expand in PyTorch - DEV Community
November 5, 2024 - My post explains repeat_interleave(). expand() can get the 0D or more D view tensor of zero or more expanded elements from the 0D or more D tensor of zero or more elements as shown below: *Memos: expand() can be used with a tensor but not with torch.
🌐
PyTorch
docs.pytorch.org › reference api › torch.tensor › torch.tensor.repeat
torch.Tensor.repeat — PyTorch 2.13 documentation
January 1, 2023 - Unlike expand(), this function copies the tensor’s data. Warning · repeat() behaves differently from numpy.repeat, but is more similar to numpy.tile. For the operator similar to numpy.repeat, see torch.repeat_interleave(). Parameters: repeat (torch.Size, int..., tuple of int or list of int) – The number of times to repeat this tensor along each dimension ·
🌐
Medium
medium.com › swlh › torch-playing-with-the-dimensions-and-shape-of-the-tensor-850e65bde7b3
Torch — Playing with the dimensions and shape of the tensor
May 30, 2020 - Some of the others include · flatten — this reduces the tensor to a single dimension. expand_as — this is the same as the expand function, but takes a tensor as an argument for the reference shape. repeat_as — this is the same as the ...
🌐
GitHub
github.com › arogozhnikov › einops › issues › 183
A question about repeat · Issue #183 · arogozhnikov/einops
April 9, 2022 - will the output_tensor just like torch.expand(), which only create a new view of the input_tensor or like torch.repeat() which just assign new memory ?
Author: arogozhnikov