From the docs:

To uninstall Anaconda open a terminal window and remove the entire anaconda install directory: rm -rf ~/anaconda. You may also edit ~/.bash_profile and remove the anaconda directory from your PATH environment variable, and remove the hidden .condarc file and .conda and .continuum directories which may have been created in the home directory with rm -rf ~/.condarc ~/.conda ~/.continuum.

Further notes:

• Python3 installs may use a ~/anaconda3 dir instead of ~/anaconda.
• You might also have a ~/.anaconda hidden directory that may be removed.
• Depending on how you installed, it is possible that the PATH is modified in one of your runcom files, and not in your shell profile. So, for example if you are using bash, be sure to check your ~/.bashrc if you don't find the PATH modified in ~/.bash_profile.

Yes, indeed there were residual files on my server. I had run the following commands to completely remove any references to anaconda2.

$ rm -rf ~/anaconda2 
$ rm -rf ~/anaconda_backup 
$ rm -rf ~/.condarc ~/.conda ~/.continuum

Downloading and Installing Anaconda (Linux)

1. Download Anaconda from https://www.continuum.io/downloads.

2. Run installer script:

   $ bash AnacondaX-x.x.x-Linux-x86[_64].sh

3. Accept the license (enter shows the license). After reading it, scroll to the end with space. Accept it by typing yes and enter when required.

4. Specify the install location. (Default is ~/anacondaX, where X is the major version of Python release, e.g. X=3 for Python 3.5)

5. Add this location to PATH if your installer asks you to. Otherwise, you should add this location on your own to you PATH in ~/.bashrc:

   export PATH="/home/$USER/anacondaX/bin:$PATH"

Uninstalling

Anaconda is installed in a single directory. So, for uninstalling it, just remove the entire directory. Default install location is ~/anacondaX. Removing is easy:

rm -rf ~/anacondaX