Run in a terminal:

echo $PATH

or

printf "%s\n" "$PATH"

what you see is a list of directories, looking like:

/home/jacob/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

If you put an executable in either one of these directories, you do not need to set the path to the executable / script, but you can run it by its name as a command.

Executables in $PATH should not have a language extension by convention (although they would work)

Editing your $PATH variable

You can (permanently) add a directory to $PATH by adding the following line to your ~/.profile file (invisible by default, press Ctrl+H in the file manager to make it visible):

export PATH=$PATH:/path/to/dir

More usefull information on environment variables

(such as $PATH) can be found here (thanks for the suggestions @Letizia)

/ is equivalent to //

From the POSIX specification (emphasis added):

3.267 Pathname

A string that is used to identify a file. In the context of POSIX.1-2008, a pathname may be limited to {PATH_MAX} bytes, including the terminating null byte. It has optional beginning <slash> characters, followed by zero or more filenames separated by <slash> characters. A pathname can optionally contain one or more trailing <slash> characters. Multiple successive <slash> characters are considered to be the same as one <slash>, except for the case of exactly two leading <slash> characters.

And:

4.12 Pathname Resolution

...
A pathname consisting of a single <slash> shall resolve to the root directory of the process. A null pathname shall not be successfully resolved. If a pathname begins with two successive <slash> characters, the first component following the leading <slash> characters may be interpreted in an implementation-defined manner, although more than two leading <slash> characters shall be treated as a single <slash> character.

For a nice discussion of exceptions (cygwin and directories) see: How does linux handle multiple consecutive path separators (/home////username///file)?

The :+ is a form of parameter expansion:

${parameter:+[word]} : Use Alternative Value.

If parameter is unset or null, null shall be substituted; otherwise, the expansion of word (or an empty string if word is omitted) shall be substituted.

In other words, if the variable $var is defined, echo ${var:+foo} will print foo and, if it is not, it will print the empty string.

The second : is nothing special. It is the character used as a separator in the list of directories in $PATH. So, PATH="/usr/local/bin:/usr/bin${PATH:+:${PATH}}" is a shorthand way of writing:

if [ -z "$PATH" ]; then
    PATH=/usr/local/bin:/usr/bin
else
    PATH=/usr/local/bin:/usr/bin:$PATH
fi

It's just a clever trick to avoid adding an extra : when $PATH is not set. For example:

$ PATH="/usr/bin"
$ PATH="/new/dir:$PATH" ## Add a directory
$ echo "$PATH"
/new/dir:/usr/bin

But if PATH is unset:

$ unset PATH
$ PATH="/new/dir:$PATH"
$ echo "$PATH"
/new/dir:

A : by itself adds the current directory to the $PATH. Using PATH="/new/dir${PATH:+:$PATH}" avoids this. So sure, you can use PATH="${PATH:+${PATH}:}/usr/local/bin:/usr/bin" if you want to, or you can use PATH="$PATH:/usr/local/bin:/usr/bin" if you prefer. The only difference is that the former might add an extra :, thereby adding your current directory to your $PATH.

They usually mean to

• put your script or executable in one of the directories listed in the PATH environment variable, as shown by echo $PATH.

• or modify said variable to contain the directory where you script/program/application is

The PATH variable contains a list of colon (:) separated directories to be searched for programs to execute. For example:

$ echo $PATH
/usr/bin:/bin:/usr/local/bin:/usr/X11R6/bin/:/usr/games

You could put your program in e.g. /usr/local/bin, so that it will be detected by other programs without you having to explicitly tell them to look at e.g. /home/user/apps/MyApp.

Alternatively, you could modify that variable to contain /home/user/apps/MyApp. For a single bash shell session, this would do:

$ export PATH="$PATH":/home/user/apps/MyApp

To do it permanently for bash you have to enter this line (without the $ shell prompt) in ~/.bashrc or ~/.bash_profile (or both).

If you have another shell (bash is the default for most Linux distributions) the commands above should be changed accordingly.

./ or just . is unix shorthand for the current directory.

You need to specify it when the current directory is not in your PATH. The PATH is the list of folders searched when you run a program. (You can determine it by writing echo $PATH.) If an executable file is not in your PATH, to run it you need to specify the folder it's in. You could do this by giving the full path to the file, but if the file is in the current directory, you can use ./ as shorthand.

Similarly, ../ or just .. is shorthand for the directory above the current one.

Root directory, current working directory, and parent directory, respectively.

Assume root looks like:

/a/b
/c

Let's break it down to componenets:
/ -> root
/a -> in (a)
. -> THIS dir path
/a/./ -> still in /a
/a/./b -> in /a/b
.. -> go "up" one level
/a/./b/.. -> /a/b/.. -> /a
/a/./b/../.. -> /a/.. -> /
/a/./b/../../c -> /c

The basic concept to grasp here is that PATH can be defined in many places. As @demure explains in his answer, PATH=$PATH:/new/dir means add /new_dir to $PATH, it will not clear the original $PATH.

Now, one reason there are many files is intimately connected with the concept of login and non-login shells. See here for a nice summary. The following is from the bash man page (emphasis mine):

When bash is invoked as an interactive login shell, or as a non-interactive shell with the --login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable. The --noprofile option may be used when the shell is started to inhibit this behavior.

When you first log into your system, you start a login shell so bash will read the files listed above. Most distributions set a system-wide $PATH (which applies to all users) at /etc/profile and this is where you should make any changes that you want applied to all users. This is what I have on my Debian:

PATH="/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games"

Once you have logged in, when you open a terminal you start an interactive, non-login shell. This is what man bash has to say about those:

   When  an  interactive shell that is not a login shell
   is started, bash reads  and  executes  commands  from
   /etc/bash.bashrc and ~/.bashrc, if these files exist.

So, those files are read every time you open a new terminal. Your filnal $PATH is the combination of the values in all files. In a typical situation, you log in using a graphical log in manager and start a new session. At this pòint your $PATH is whatever was defined in the various profile files. If you open a terminal, then you are in an interactive shell and the different bashrc files are read which may append things to the $PATH.

To summarize, all you really need to know is that you can make changes to your user's $PATH by editing $HOME/.profile.