The compareTo method is described as follows:

Compares this object with the specified object for order. Returns a negative integer, zero, or a positive integer as this object is less than, equal to, or greater than the specified object.

Let's say we would like to compare Jedis by their age:

class Jedi implements Comparable<Jedi> {

    private final String name;
    private final int age;
        //...
}

Then if our Jedi is older than the provided one, you must return a positive, if they are the same age, you return 0, and if our Jedi is younger you return a negative.

public int compareTo(Jedi jedi){
    return this.age > jedi.age ? 1 : this.age < jedi.age ? -1 : 0;
}

By implementing the compareTo method (coming from the Comparable interface) your are defining what is called a natural order. All sorting methods in JDK will use this ordering by default.

There are ocassions in which you may want to base your comparision in other objects, and not on a primitive type. For instance, copare Jedis based on their names. In this case, if the objects being compared already implement Comparable then you can do the comparison using its compareTo method.

public int compareTo(Jedi jedi){
    return this.name.compareTo(jedi.getName());
}

It would be simpler in this case.

Now, if you inted to use both name and age as the comparison criteria then you have to decide your oder of comparison, what has precedence. For instance, if two Jedis are named the same, then you can use their age to decide which goes first and which goes second.

public int compareTo(Jedi jedi){
    int result = this.name.compareTo(jedi.getName());
    if(result == 0){
        result = this.age > jedi.age ? 1 : this.age < jedi.age ? -1 : 0;
    }
    return result;
}

If you had an array of Jedis

Jedi[] jediAcademy = {new Jedi("Obiwan",80), new Jedi("Anakin", 30), ..}

All you have to do is to ask to the class java.util.Arrays to use its sort method.

Arrays.sort(jediAcademy);

This Arrays.sort method will use your compareTo method to sort the objects one by one.

I think what you want is sort of multiple dispatch which is ususally implemented in Java using the Visitor-Pattern but you would have to change the code when adding new classes.

Since you don't really need full-blown multiple dispatch I think the following might be a good solution. Write your compareTo methods like this: (This is the one in TestB)

@Override
public int compareTo(TestA o) {
    if(o instanceof TestB) {
        // add your code instead
        System.out.println("Compare Bs");
        return 0;
    } else {
        return super.compareTo(o);
    }
}

Since the call to x.compareTo is resolved at run-time you always run the compareTo function of the most specific class. By checking if the other argument is instanceof the current class you can detect when you have to move up the hierarchy which you can do calling super.compareTo.

Let me know if you have any further questions. :)

Edit: I assumed what you wanted to do was compare as the most specific common ancestor as per your comment on the original question.