The 2 main differences are that:

1. equals will take any Object as a parameter, but compareTo will only take Strings.
2. equals only tells you whether they're equal or not, but compareTo gives information on how the Strings compare lexicographically.

I took a look at the String class code, and the algorithm within compareTo and equals looks basically the same. I believe his opinion was just a matter of taste, and I agree with you -- if all you need to know is the equality of the Strings and not which one comes first lexicographically, then I would use equals.

This confusion would occur in situations where there are conflicting understandings of equals.

compareTo is 'easy' in that it asks a simple question 'which one is bigger?' If you were given a bunch of Things, how do you sort them?

equals on the other hand wants to ask 'are these the same thing?'

BigDecimal in Java is one such place where there are conflicting understandings of what it means to have two things being equal.

BigDecimal foo = new BigDecimal("1.00");
BigDecimal bar = new BigDecimal("1.000");

These two entires will have the same meaning for sorting. They, however, are not the same when it comes to equality. They have a different underlying state (the precision of the number) and that means they are not equal but foo.compareTo(bar) will return 0.

Consider this, if you had a Map qux = HashMap<BigDecimal, Object>() for some reason, do you want qux.put(foo,foo) to take the same spot as qux.put(bar,bar) and thus evict the earlier insertion?

So, while they are math equals (which is how compareTo sorts them), they are not inner state equals, and thus the necessity of the inconsistency here.

Yes, this inconsistency comes at the price of a higher cognitive load for dealing with BigDecimal. It means maps may not behave like you want them to... and the question is "which map do you want to behave 'right'?"

import java.math.BigDecimal;
import java.util.HashMap;
import java.util.TreeMap;

class Main {
    public static void main (String[] args) {
        BigDecimal foo = new BigDecimal("1.00");
        BigDecimal bar = new BigDecimal("1.000");

        HashMap<BigDecimal, String> hash = new HashMap();
        TreeMap<BigDecimal, String> tree = new TreeMap();

        hash.put(foo, "foo");
        hash.put(bar, "bar");

        tree.put(foo, "foo");
        tree.put(bar, "bar");

        System.out.println("hash foo: " + hash.get(foo));
        System.out.println("hash bar: " + hash.get(bar));

        System.out.println("tree foo: " + tree.get(foo));
        System.out.println("tree bar: " + tree.get(bar));
    }
}

^ideone

Output:

hash foo: foo
hash bar: bar
tree foo: bar
tree bar: bar

Because compareTo returned 0, in the TreeMap, bar evicted foo when bar was inserted. However, because these are different objects with different internal state and thus different hash codes, they were both able to exist within a HashMap.

From the docs:

Note: care should be exercised if BigDecimal objects are used as keys in a SortedMap or elements in a SortedSet since BigDecimal's natural ordering is inconsistent with equals. See Comparable, SortedMap or SortedSet for more information.

And so, thats the problem, the inconsistency and the "there's no good answer to this" dilemma.

One could also imagine this with a Rational class where one wants to keep 2/4 having the internal state of 2/4 so that:

Rational twoFourths = new Rational(2,4);
Rational oneHalf = new Rational(1,2);

System.out.println(twoFourths); // prints 2/4
System.out.println(oneHalf);    // prints 1/2
System.out.println(twoFourths.compareTo(oneHalf)); // prints 0
System.out.println(twoFourths.equals(oneHalf));    // ???

And you are once again up against that same question. Are these equal with the mathematical sense of equality (which means the hashCode needs to also return the same value?) or are they not equal using the object oriented state of the object sense of equality despite being the 'same'.

Did you try it?

String a = new String("foo");
String b = new String("foo");

System.out.println(a == b); // false
System.out.println(a.equals(b)); // true
System.out.println(a.compareTo(b)); // 0

The JLS §4.3.2 defines the class object in the following way:

4.3.2. The Class Object

The class Object is a superclass (§8.1.4) of all other classes.

All class and array types inherit (§8.4.8) the methods of class Object, which are summarized as follows:

• The method clone is used to make a duplicate of an object.

• The method equals defines a notion of object equality, which is based on value, not reference, comparison.

• The method finalize is run just before an object is destroyed (§12.6).

• The method getClass returns the Class object that represents the class of the object.

• A Class object exists for each reference type. It can be used, for example, to discover the fully qualified name of a class, its members, its immediate superclass, and any interfaces that it implements.

  The type of a method invocation expression of getClass is Class<? extends |T|> where T is the class or interface searched (§15.12.1) for getClass.

  A class method that is declared synchronized (§8.4.3.6) synchronizes on the monitor associated with the Class object of the class.

• The method hashCode is very useful, together with the method equals, in hashtables such as java.util.Hashmap.

• The methods wait, notify, and notifyAll are used in concurrent programming using threads (§17.2).

• The method toString returns a String representation of the object.

So, that's why equals is in Object but compareTo is in a separate interface. I would speculate that they wanted to keep Object as minimal as possible. They probably figured that nearly all Objects would need equals and hashCode (which is really just a form of equality testing) but not all objects would need to have a concept of ordering, which is what compareTo is used for.

From Javadoc of java.lang.Comparable:

It is strongly recommended (though not required) that natural orderings be consistent with equals.