Run in a terminal:

echo $PATH

or

printf "%s\n" "$PATH"

what you see is a list of directories, looking like:

/home/jacob/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

If you put an executable in either one of these directories, you do not need to set the path to the executable / script, but you can run it by its name as a command.

Executables in $PATH should not have a language extension by convention (although they would work)

Editing your $PATH variable

You can (permanently) add a directory to $PATH by adding the following line to your ~/.profile file (invisible by default, press Ctrl+H in the file manager to make it visible):

export PATH=$PATH:/path/to/dir

More usefull information on environment variables

(such as $PATH) can be found here (thanks for the suggestions @Letizia)

/ is equivalent to //

From the POSIX specification (emphasis added):

3.267 Pathname

A string that is used to identify a file. In the context of POSIX.1-2008, a pathname may be limited to {PATH_MAX} bytes, including the terminating null byte. It has optional beginning <slash> characters, followed by zero or more filenames separated by <slash> characters. A pathname can optionally contain one or more trailing <slash> characters. Multiple successive <slash> characters are considered to be the same as one <slash>, except for the case of exactly two leading <slash> characters.

And:

4.12 Pathname Resolution

...
A pathname consisting of a single <slash> shall resolve to the root directory of the process. A null pathname shall not be successfully resolved. If a pathname begins with two successive <slash> characters, the first component following the leading <slash> characters may be interpreted in an implementation-defined manner, although more than two leading <slash> characters shall be treated as a single <slash> character.

For a nice discussion of exceptions (cygwin and directories) see: How does linux handle multiple consecutive path separators (/home////username///file)?

The :+ is a form of parameter expansion:

${parameter:+[word]} : Use Alternative Value.

If parameter is unset or null, null shall be substituted; otherwise, the expansion of word (or an empty string if word is omitted) shall be substituted.

In other words, if the variable $var is defined, echo ${var:+foo} will print foo and, if it is not, it will print the empty string.

The second : is nothing special. It is the character used as a separator in the list of directories in $PATH. So, PATH="/usr/local/bin:/usr/bin${PATH:+:${PATH}}" is a shorthand way of writing:

if [ -z "$PATH" ]; then
    PATH=/usr/local/bin:/usr/bin
else
    PATH=/usr/local/bin:/usr/bin:$PATH
fi

It's just a clever trick to avoid adding an extra : when $PATH is not set. For example:

$ PATH="/usr/bin"
$ PATH="/new/dir:$PATH" ## Add a directory
$ echo "$PATH"
/new/dir:/usr/bin

But if PATH is unset:

$ unset PATH
$ PATH="/new/dir:$PATH"
$ echo "$PATH"
/new/dir:

A : by itself adds the current directory to the $PATH. Using PATH="/new/dir${PATH:+:$PATH}" avoids this. So sure, you can use PATH="${PATH:+${PATH}:}/usr/local/bin:/usr/bin" if you want to, or you can use PATH="$PATH:/usr/local/bin:/usr/bin" if you prefer. The only difference is that the former might add an extra :, thereby adding your current directory to your $PATH.

They usually mean to

• put your script or executable in one of the directories listed in the PATH environment variable, as shown by echo $PATH.

• or modify said variable to contain the directory where you script/program/application is

The PATH variable contains a list of colon (:) separated directories to be searched for programs to execute. For example:

$ echo $PATH
/usr/bin:/bin:/usr/local/bin:/usr/X11R6/bin/:/usr/games

You could put your program in e.g. /usr/local/bin, so that it will be detected by other programs without you having to explicitly tell them to look at e.g. /home/user/apps/MyApp.

Alternatively, you could modify that variable to contain /home/user/apps/MyApp. For a single bash shell session, this would do:

$ export PATH="$PATH":/home/user/apps/MyApp

To do it permanently for bash you have to enter this line (without the $ shell prompt) in ~/.bashrc or ~/.bash_profile (or both).

If you have another shell (bash is the default for most Linux distributions) the commands above should be changed accordingly.

./ or just . is unix shorthand for the current directory.

You need to specify it when the current directory is not in your PATH. The PATH is the list of folders searched when you run a program. (You can determine it by writing echo $PATH.) If an executable file is not in your PATH, to run it you need to specify the folder it's in. You could do this by giving the full path to the file, but if the file is in the current directory, you can use ./ as shorthand.

Similarly, ../ or just .. is shorthand for the directory above the current one.

Root directory, current working directory, and parent directory, respectively.

Assume root looks like:

/a/b
/c

Let's break it down to componenets:
/ -> root
/a -> in (a)
. -> THIS dir path
/a/./ -> still in /a
/a/./b -> in /a/b
.. -> go "up" one level
/a/./b/.. -> /a/b/.. -> /a
/a/./b/../.. -> /a/.. -> /
/a/./b/../../c -> /c

The basic concept to grasp here is that PATH can be defined in many places. As @demure explains in his answer, PATH=$PATH:/new/dir means add /new_dir to $PATH, it will not clear the original $PATH.

Now, one reason there are many files is intimately connected with the concept of login and non-login shells. See here for a nice summary. The following is from the bash man page (emphasis mine):

When bash is invoked as an interactive login shell, or as a non-interactive shell with the --login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable. The --noprofile option may be used when the shell is started to inhibit this behavior.

When you first log into your system, you start a login shell so bash will read the files listed above. Most distributions set a system-wide $PATH (which applies to all users) at /etc/profile and this is where you should make any changes that you want applied to all users. This is what I have on my Debian:

PATH="/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games"

Once you have logged in, when you open a terminal you start an interactive, non-login shell. This is what man bash has to say about those:

   When  an  interactive shell that is not a login shell
   is started, bash reads  and  executes  commands  from
   /etc/bash.bashrc and ~/.bashrc, if these files exist.

So, those files are read every time you open a new terminal. Your filnal $PATH is the combination of the values in all files. In a typical situation, you log in using a graphical log in manager and start a new session. At this pòint your $PATH is whatever was defined in the various profile files. If you open a terminal, then you are in an interactive shell and the different bashrc files are read which may append things to the $PATH.

To summarize, all you really need to know is that you can make changes to your user's $PATH by editing $HOME/.profile.

For the sake of completeness ...

• Just path is a file or directory named path in the current directory.
• ./path is a file or directory named path in the current directory, with the directory spelled out. The dot directory . represents the current directory, and path is the name of the file or directory within this directory.
• ~/path is a shorthand for $HOME/path where $HOME is a variable which refers to your home directory. Typically your home directory will be somewhere like /home/you or /Users/you where you is your account name. (The command echo "$HOME" will display your home directory.) The expanded value is an absolute path (unless you have messed up the value of $HOME thoroughly), as indicated by the initial slash.
• /path is an absolute path which refers to a file or directory named path which is in the root directory /. Every file on Unix is ultimately somewhere in the directory tree which starts with the root directory.

A file name which begins with $ includes the value of a shell variable in its name (like for example $HOME above); you have to know the value of that variable to determine whether it ends up containing a relative or an absolute path. Similarly, ~ at the beginning of a file name gets replaced ("expanded") by the shell to a different string, as outlined above. (Technically, it's possible for a file name to begin with a literal dollar sign or tilde, too; you would then have to quote or backslash-escape that character to avoid having the shell expand it to something else. This is rather inconvenient, so these file names tend to be rare in practice.)

In the following exposition, we refer to the result of any such replacements, and ignore the complication of possible quoting.

Every file name which begins with / is an absolute path (aka full path) which explains how to reach a particular node starting from the root directory. For example, /var/tmp/you/reminder.txt refers to a file or directory reminder.txt (probably a file, judging from the name; but Unix doesn't care what you call your files or directories) which is in the directory you which is in the directory tmp which is in the directory var which is in the root directory.

Every file name which doesn't begin with / is a relative path which indicates how to reach a particular file or directory starting from the current directory. The special directory .. is the parent directory (that is, the directory which contains this directory) and the special directory . is the current directory. So path/there refers to the file or directory there inside the directory path in the current directory; and (hover the mouse over the gray area to display the spoiler)

there/.././and/back/.. is a (wicked complicated) way to refer to the directory and in the current directory, where we traverse the there directory and then move back to the current directory; then stay in the current directory; then refer to the directory back inside the directory and, but then move back to the parent directory of that, ending up with ./and.

In addition to ~/ for the current user's home directory, some shells and applications allow the notation ~them/ to refer to the home directory of the user account them. Also, some web server configurations allow each user to have a public web site in their directory ~/public_html and the URL notation http://server/~them/ would serve up the site of the user account them for outside visitors.

The current directory is a convenience which the shell provides so you don't have to type long paths all the time. You can, if you want to.

/bin/ls /home/you/Documents/unix-101/directories.txt

is a longwinded but perfectly valid way to say (assuming you are in your home directory),

ls Documents/unix-101/directories.txt

You could also say

cd Documents/unix-101
ls directories.txt

and until you cd again, all your commands will run in this directory.

See What exactly is current working directory? for a longer exposition of this related concept.

A "directory" is sometimes called a "folder" by people who are not yet old enough to prefer the former.

Tangentially, don't confuse the directory name . with the Bourne shell command which comprises a single dot (also known by its Bash alias source). The command

. ./scriptname

runs the commands from the file ./scriptname in the context of the current shell instance, as opposed to in a separate subshell (which is what just ./scriptname does). In other words, this command line invokes the dot command on a file scriptname in the dot directory.

The Bourne shell (and derivatives like Bash, Zsh, etc) use single quotes to prevent variable expansion and wildcard expansion, and double quotes to permit variable expansion, but inhibit wildcard expansion in a string. The quoting rules on Windows are different, and generally use double quotes to keep whitespace-separated values as a single string (and % instead of $ for variable substitutions).