To get real-time insight on used resources, do:

nvidia-smi -l 1

This will loop and call the view at every second.

If you do not want to keep past traces of the looped call in the console history, you can also do:

watch -n0.1 nvidia-smi

Where 0.1 is the time interval, in seconds.

For Nvidia GPUs there is a tool nvidia-smi that can show memory usage, GPU utilization and temperature of GPU. There also is a list of compute processes and few more options but my graphic card (GeForce 9600 GT) is not fully supported.

Sun May 13 20:02:49 2012       
+------------------------------------------------------+                       
| NVIDIA-SMI 3.295.40   Driver Version: 295.40         |                       
|-------------------------------+----------------------+----------------------+
| Nb.  Name                     | Bus Id        Disp.  | Volatile ECC SB / DB |
| Fan   Temp   Power Usage /Cap | Memory Usage         | GPU Util. Compute M. |
|===============================+======================+======================|
| 0.  GeForce 9600 GT           | 0000:01:00.0  N/A    |       N/A        N/A |
|   0%   51 C  N/A   N/A /  N/A |  90%  459MB /  511MB |  N/A      Default    |
|-------------------------------+----------------------+----------------------|
| Compute processes:                                               GPU Memory |
|  GPU  PID     Process name                                       Usage      |
|=============================================================================|
|  0.           Not Supported                                                 |
+-----------------------------------------------------------------------------+

• For Nvidia GPUs there is a tool nvidia-smi that can show memory usage, GPU utilization and temperature of GPU.

• For Intel GPU's you can use the intel-gpu-tools.

• AMD has two options

  1. fglrx (closed source drivers):

      aticonfig --odgc --odgt
     

  2. And for mesa (open source drivers), you can use RadeonTop

     sudo apt install radeontop
     radeontop
     

^{Source:GPU usage monitoring}