Imagine have n tensors. If we stay in 3D, those correspond to volumes, namely rectangular cuboids. Stacking corresponds to combining those n volumes on an additional dimension: here a 4th dimension is added to host the n 3D volumes. This operation is in clear contrast with concatenation, where the volumes would be combined on one of the existing dimensions. So concatenation of three-dimensional tensors would result in a 3D tensor.
Here is a possible representation of the stacking operations for limited dimensions sizes (up to three-dimensional inputs):
Where you chose to perform the stacking defines along which new dimension the stack will take place. In the above examples, the newly created dimension is last, hence the idea of "added dimension" makes more sense.
In the following visualization, we observe how tensors can be stacked on different axes. This in turn affects the resulting tensor shape
For the 1D case, for instance, it can also happen on the first axis, see below:
With code:
>>> x_1d = list(torch.empty(3, 2)) # 3 lines >>> torch.stack(x_1d, 0).shape # axis=0 stacking torch.Size([3, 2]) >>> torch.stack(x_1d, 1).shape # axis=1 stacking torch.Size([2, 3])Similarly for two-dimensional inputs:
With code:
>>> x_2d = list(torch.empty(3, 2, 2)) # 3 2x2-squares >>> torch.stack(x_2d, 0).shape # axis=0 stacking torch.Size([3, 2, 2]) >>> torch.stack(x_2d, 1).shape # axis=1 stacking torch.Size([2, 3, 2]) >>> torch.stack(x_2d, 2).shape # axis=2 stacking torch.Size([2, 2, 3])
With this state of mind, you can intuitively extend the operation to n-dimensional tensors.
Answer from Ivan on Stack OverflowImagine have n tensors. If we stay in 3D, those correspond to volumes, namely rectangular cuboids. Stacking corresponds to combining those n volumes on an additional dimension: here a 4th dimension is added to host the n 3D volumes. This operation is in clear contrast with concatenation, where the volumes would be combined on one of the existing dimensions. So concatenation of three-dimensional tensors would result in a 3D tensor.
Here is a possible representation of the stacking operations for limited dimensions sizes (up to three-dimensional inputs):
Where you chose to perform the stacking defines along which new dimension the stack will take place. In the above examples, the newly created dimension is last, hence the idea of "added dimension" makes more sense.
In the following visualization, we observe how tensors can be stacked on different axes. This in turn affects the resulting tensor shape
For the 1D case, for instance, it can also happen on the first axis, see below:
With code:
>>> x_1d = list(torch.empty(3, 2)) # 3 lines >>> torch.stack(x_1d, 0).shape # axis=0 stacking torch.Size([3, 2]) >>> torch.stack(x_1d, 1).shape # axis=1 stacking torch.Size([2, 3])Similarly for two-dimensional inputs:
With code:
>>> x_2d = list(torch.empty(3, 2, 2)) # 3 2x2-squares >>> torch.stack(x_2d, 0).shape # axis=0 stacking torch.Size([3, 2, 2]) >>> torch.stack(x_2d, 1).shape # axis=1 stacking torch.Size([2, 3, 2]) >>> torch.stack(x_2d, 2).shape # axis=2 stacking torch.Size([2, 2, 3])
With this state of mind, you can intuitively extend the operation to n-dimensional tensors.
Very simple! I will use 4 varibles for this example. The function torch.stack is very similar to numpy (vstack and hstack). Example :
t1 = torch.tensor([1,2,3])
t2 = torch.tensor([4,5,6])
t3 = torch.tensor([7,8,9])
t4 = torch.tensor([10,11,12])
if you try this cmd
>> torch.stack((t1,t2,t3,t4),dim=1).size()
>> torch.Size([3, 4])
if you change dim=1 with dim=0
>> torch.stack((t1,t2,t3,t4),dim=0).size()
>> torch.Size([4, 3])
In first case you have a tensor with 3x4 dimensional but in last case you have a 4x3 tensor. Test this code without using .size() !
>> torch.stack((t1,t2,t3,t4),dim=1)
>> tensor([[ 1, 4, 7, 10],
[ 2, 5, 8, 11],
[ 3, 6, 9, 12]])
>> torch.stack((t1,t2,t3,t4),dim=0)
>> tensor([[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12]])
Happy Coding!
stack
Concatenates sequence of tensors along a new dimension.
cat
Concatenates the given sequence of seq tensors in the given dimension.
So if A and B are of shape (3, 4):
torch.cat([A, B], dim=0)will be of shape (6, 4)torch.stack([A, B], dim=0)will be of shape (2, 3, 4)
t1 = torch.tensor([[1, 2],
[3, 4]])
t2 = torch.tensor([[5, 6],
[7, 8]])
torch.stack |
torch.cat |
|---|---|
'Stacks' a sequence of tensors along a new dimension:  |
'Concatenates' a sequence of tensors along an existing dimension:  |
These functions are analogous to numpy.stack and numpy.concatenate.