The answer is in two parts. From the documentation of sys.getsizeof, firstly

All built-in objects will return correct results, but this does not have to hold true for third-party extensions as it is implementation specific.

so it could be that for tensors __sizeof__ is undefined or defined differently than you would expect - this function is not something you can rely on. Secondly

Only the memory consumption directly attributed to the object is accounted for, not the memory consumption of objects it refers to.

which means that if the torch.Tensor object merely holds a reference to the actual memory, this won't show in sys.getsizeof. This is indeed the case, if you check the size of the underlying storage instead, you will see the expected number

import torch, sys
b = torch.randn(1, 1, 128, 256, dtype=torch.float64)
sys.getsizeof(b)
>> 72
sys.getsizeof(b.storage())
>> 262208

Note: I am setting dtype to float64 explicitly, because that is the default dtype in numpy, whereas torch uses float32 by default.

Step 1 :
Get the dtype of the tensor.
This will tell you about the number of bytes e.g.float64 is 64 bits = 8 Bytes.

Step 2
Get the shape of the Tensor.
This will give you the number of place-holders of the dtype.
lets's assume shape = m x n x p
Count of the placeholders is C = m * n * p
Memory = 8 * C => Memory = 8 *m * n * p Bytes.

See this excerpt from Deep Learning with Python, by Francois Chollet

A 60-second, 144 × 256 YouTube video clip sampled at 4 frames per second would have 240 frames. A batch of four such video clips would be stored in a tensor of shape (4, 240, 144, 256, 3). That’s a total of 106,168,320 values! If the dtype of the tensor was float32, then each value would be stored in 32 bits, so the tensor would represent 405 MB

Per the PyTorch discussion forum:

sys.getsizeof() will return the size of the python object. It will the same for all tensors as all tensors are a python object containing a tensor. For each tensor, you have a method element_size() that will give you the size of one element in byte. And a function nelement() that returns the number of elements. So the size of a tensor a in memory (cpu memory for a cpu tensor and gpu memory for a gpu tensor) is a.element_size() * a.nelement().

Therefore, the (simple) answer appears to be as follows, and holds true when calculating expected memory usage and release in various scenarios:

a.element_size() * a.nelement()

This is supported by experimenting with the following:

torch.tensor(1, dtype=torch.int8).element_size()   # 1 (byte)
torch.tensor(1, dtype=torch.int16).element_size()  # 2 (bytes)
torch.tensor(1, dtype=torch.int32).element_size()  # 4 (bytes)

Additionally, this post offers a more detailed way to calculate the memory usage.

The reason the tensor takes up so much memory is because by default the tensor will store the values with the type torch.float32. This data type will use 4kb for each value in the tensor (check using .element_size()), which will give a total of ~48GB after multiplying with the number of zero values in your tensor (4 * 2000 * 2000 * 3200 = 47.68GB). What you could try to do is change the datatype from torch.float32 to something like torch.int8, which only uses 1kb for each value, reducing the memory needed by 75% to 12GB. But given that you only seem to have 8GB available this will also not solve your problem. The only solution therefore would simply be to use a tensor will fewer values.