Null hypothesis

The purpose of the null is to convert a problem from one of inductive reasoning to one of deductive reasoning. The alternative, and the method that preceded it was the method of inverse probability. That method is now generally called Bayesian statistics.

Imagine that you had three scientific hypotheses, denoted a, b, and c. Imagine that the true model is d, but no one has yet to discover this. The world is still flat, white is still a color, and Mercury follows Newton's laws.

A Bayesian test would create three hypotheses, $H_a$, $H_b$, and $H_c$. For a data set that is large enough, you would end up with the hypothesis or the combination of hypotheses that are most likely true. However, since $H_d$ wasn't tested, you may continue to be fooled by the idea yet to be thought of.

The Frequentist hypothesis testing regime would assume that the alternative hypothesis is $H_A\to{H}_x$, and the null is $H_0\to\neg{H}_x$. The null contains every hypothesis that is not the alternative.

The first example in the academic literature, but not the first null hypothesis, is where R.A. Fisher assumed that Mendel's laws were false as the null. If you discredit the null, then you exclude every explanation, including those not yet considered. The first null hypothesis was that Muriel Bristol (Fisher's boss) could not correctly detect the difference between tea poured into mild from milk poured into tea. That was the very first statistical test.

There is a slight difference between R.A. Fisher's idea of a null and Pearson and Neyman's idea of a null. Fisher felt there was a null, but no alternative hypothesis. If you rejected the null, it told you what was wrong, but was not directive as to what was correct automatically. Pearson and Neyman championed acceptance and rejection regions based on frequencies, and they felt the method directed behavior.

The logic was that the method created a probabilistic version of modus tollens. Modus tollens is "if A then B; and, not B; therefore, not A." Or, if the null is true, then the test will appear in the acceptance region if it does not, then you can reject the null.

The weakness of the methodology was proposed by an author that I cannot currently locate in this somewhat tongue-in-cheek way. There are 535 elected members representing the states in the U.S. Congress. There are 360 million Americans. Therefore since 535/360000000 is less than .05, if you randomly sample a group of Americans and pick a member of Congress, they cannot be an American (p<.05).

While Fisher's no effect hypothesis is the most common version, because of its implication would be that something has an effect in the alternative, it is not a requirement that a parameter equals zero, or a set of parameters all equal zero.

What matters is that the null is the opposite of what you are wanting to assert before seeing the data.

That makes the null hypothesis method a potent tool. Think about this as a rhetorical device. Your opponent opposes $H_d$ that you recently believe you have discovered.

You do not assert $H_d$ is true. You assert your opponent's position of $\neg{H}_d$ is true and build your probabilities on the assumption that you are the only person that is wrong. Everyone is right, and you are wrong.

It is a powerful rhetorical tool to concede the argument from the beginning, but then ask, "what would the world look like if I am the one that is wrong?" That is the null. If you reject the null, then what you are really saying is that "nature rejects all other ideas except mine."

Now as to your question, you want to show that college algebra matters, therefore your null hypothesis is that college algebra does not matter. We will ignore all the other methodological issues that would really be present since people without college algebra may have other self-selection issues as will the people with college algebra.

Your null is that algebra does not matter. The alternative is that it does. If the p-value is less than your $\alpha$ cutoff, chosen before collecting the data, then you can reject the null. If it is not, then you should behave as if it is true until you either do more research or find another way to come to a conclusion.

It would be dubious, ignoring the methodology issues, to assert that college algebra matters as you only have one sample. The method is intended for repetition. Nonetheless, you would only be made a fool of no more than $\alpha$ percent of the time, ignoring the methodological issues by following the results of the test.

Your question starts out as if the statistical null and alternative hypotheses are what you are interested in, but the penultimate sentence makes me think that you might be more interested in the difference between scientific and statistical hypotheses.

Statistical hypotheses can only be those that are expressible within a statistical model. They typically concern values of parameters within the statistical model. Scientific hypotheses almost invariably concern the real world, and they often do not directly translate into the much more limited universe of the chosen statistical model. Few introductory stats books spend any real time considering what constitutes a statistical model (it can be very complicated) and the trivial examples used have scientific hypotheses so simple that the distinction between model and real-world hypotheses is blurry.

I have written an extensive account of hypothesis and significance testing that includes several sections dealing with the distinction between scientific and statistical hypotheses, as well as the dangers that might come from assuming a match between the statistical model and the real-world scientific concerns: A Reckless Guide to P-values

So, to answer your explicit questions:

• No, statisticians do not always use null and alternative hypotheses. Many statistical methods do not require them.

• It is common practice in some disciplines (and maybe some schools of statistics) to specify the null and alternative hypothesis when a hypothesis test is being used. However, you should note that a hypotheses test requires an explicit alternative for the planning stage (e.g. for sample size determination) but once the data are in hand that alternative is no longer relevant. Many times the post-data alternative can be no more than 'not the null'.

• I'm not sure of the mental heuristic thing, but it does seem possible to me that the beginner courses omit so much detail in the service of simplicity that the word 'hypothesis' loses its already vague meaning.

The rule for the proper formulation of a hypothesis test is that the alternative or research hypothesis is the statement that, if true, is strongly supported by the evidence furnished by the data.

The null hypothesis is generally the complement of the alternative hypothesis. Frequently, it is (or contains) the assumption that you are making about how the data are distributed in order to calculate the test statistic.

Here are a few examples to help you understand how these are properly chosen.

1. Suppose I am an epidemiologist in public health, and I'm investigating whether the incidence of smoking among a certain ethnic group is greater than the population as a whole, and therefore there is a need to target anti-smoking campaigns for this sub-population through greater community outreach and education. From previous studies that have been published in the literature, I find that the incidence among the general population is $p_0$. I can then go about collecting sample data (that's actually the hard part!) to test $$H_0 : p = p_0 \quad \mathrm{vs.} \quad H_a : p > p_0.$$ This is a one-sided binomial proportion test. $H_a$ is the statement that, if it were true, would need to be strongly supported by the data we collected. It is the statement that carries the burden of proof. This is because any conclusion we draw from the test is conditional upon assuming that the null is true: either $H_a$ is accepted, or the test is inconclusive and there is insufficient evidence from the data to suggest $H_a$ is true. The choice of $H_0$ reflects the underlying assumption that there is no difference in the smoking rates of the sub-population compared to the whole.

2. Now suppose I am a researcher investigating a new drug that I believe to be equally effective to an existing standard of treatment, but with fewer side effects and therefore a more desirable safety profile. I would like to demonstrate the equal efficacy by conducting a bioequivalence test. If $\mu_0$ is the mean existing standard treatment effect, then my hypothesis might look like this: $$H_0 : |\mu - \mu_0| \ge \Delta \quad \mathrm{vs.} \quad H_a : |\mu - \mu_0| < \Delta,$$ for some choice of margin $\Delta$ that I consider to be clinically significant. For example, a clinician might say that two treatments are sufficiently bioequivalent if there is less than a $\Delta = 10\%$ difference in treatment effect. Note again that $H_a$ is the statement that carries the burden of proof: the data we collect must strongly support it, in order for us to accept it; otherwise, it could still be true but we don't have the evidence to support the claim.

3. Now suppose I am doing an analysis for a small business owner who sells three products $A$, $B$, $C$. They suspect that there is a statistically significant preference for these three products. Then my hypothesis is $$H_0 : \mu_A = \mu_B = \mu_C \quad \mathrm{vs.} \quad H_a : \exists i \ne j \text{ such that } \mu_i \ne \mu_j.$$ Really, all that $H_a$ is saying is that there are two means that are not equal to each other, which would then suggest that some difference in preference exists.