Java doesn't throw an error if you increase a number after its maximum value. If you wish to have this behaviour, you could use the Math.addExact(long x, long y) method from Java 8. This method will throw an ArithmeticException if you pass the Long.MAX_VALUE.

The reason why Java doesn't throw an exception and you receive negative numbers has to do with the way numbers are stored. For a long primitive the first byte is used for indicating the sign of the number (0 -> positive, 1 -> negative), while the rest are used for the numeric value. This means that Long.MAX_VALUE which is the biggest positive value will be stored as 01111...111 (0 followed by 63 bits of 1). Since you add a number to Long.MAX_VALUE you will start receiving negative integers since the sign byte changes to 1. This means you have an numeric overflow, but this error isn't thrown by Java.

Numeric overflow

That's how Java and similar languages handle numeric overflow. In order to illustrate it, you can write a much simpler code:

public static void main(String[] args)
{
    int intV = Integer.MAX_VALUE;
    System.out.println(intV + 1);
}

which would produce -2147483648 as a result.

An answer on StackOverflow explains what happens under the hood, “binary-speaking”.

Note that doing this:

public static void main(String[] args)
{
    int intV = Integer.MAX_VALUE;
    long result = intV + 1;
    System.out.println(result);
}

since the intV + 1 part will be evaluated first, and only after that, the resulting value will be cast to a long. However, this would give a positive result, because the addition is now performed on a long value:

public static void main(String[] args)
{
    int intV = Integer.MAX_VALUE;
    long result = (long)intV + 1;
    System.out.println(result);
}

Specific handling of short

If you play with the first piece of code from my answer, you may end up writing code like this:

public static void main(String[] args)
{
    short shortV = Short.MAX_VALUE;
    System.out.println(shortV + 1);
}

You may be surprised to see that the result is 32768, which looks like nonsense since short can only store values from -32 768 to 32 767.

This is a particularity of Java (and similar languages such as C#) in a way the + operator works with short values. You can have a hint about what's going on by writing this piece of code:

public static void main(String[] args)
{
    short a = 5;
    short b = 4;
    short c = a + b;
    System.out.println(c);
}

This piece of code won't compile:

error: incompatible types: possible lossy conversion from int to short

What happens is that + operator first converts the short values to integers, and then produces the result. This is why in the piece of code above, you don't encounter a numeric overflow, since the addition is performed on an actual integer, and it's the value of an integer which is passed to println. On the other hand, this code:

public static void main(String[] args)
{
    short shortV = Short.MAX_VALUE;
    short result = (short)(shortV + 1);
    System.out.println(result);
}

will display -32768, because even if the addition itself resulted in 32768, the cast of 32768 to short produced a numeric overflow.

Yes, your understanding is correct (at least if I understood your understanding correctly).

The reason this happens is because (most) computers use 2's complement to store signed values. So when assigning a smaller datatype to a larger one, the value is sign extended meaning that the excess part of the datatype is filled with 0 or 1 bits depending on whether the original value was positive or negative.

Also related is the difference between >> and >>> operators in Java. The first one performs sign extending (keeping negative values negative) the second one does not (shifting a negative value makes it positive).

This image shows what you're looking for. In your case it's obviously larger numbers, but the principle stays the same.

Examples of limits in java are:
int: −2,147,483,648 to 2,147,483,647.
long: -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807

In the image 0000, 0001 etc, shows the binary representation of the numbers.

EDIT: In project euler you often have to think of a way to work around the lagre numbers. The problems are designed with numbers that big so that you can't use the ordinary way of problem solving. However, if you find that you really need to use them, i suggest studying BigInteger anyway. You will find it useful in the long run, and it's not all that complicated. Here is a link with lots of understandable examples: BigInteger Example