The purpose of the null is to convert a problem from one of inductive reasoning to one of deductive reasoning. The alternative, and the method that preceded it was the method of inverse probability. That method is now generally called Bayesian statistics.

Imagine that you had three scientific hypotheses, denoted a, b, and c. Imagine that the true model is d, but no one has yet to discover this. The world is still flat, white is still a color, and Mercury follows Newton's laws.

A Bayesian test would create three hypotheses, , , and . For a data set that is large enough, you would end up with the hypothesis or the combination of hypotheses that are most likely true. However, since wasn't tested, you may continue to be fooled by the idea yet to be thought of.

The Frequentist hypothesis testing regime would assume that the alternative hypothesis is , and the null is . The null contains every hypothesis that is not the alternative.

The first example in the academic literature, but not the first null hypothesis, is where R.A. Fisher assumed that Mendel's laws were false as the null. If you discredit the null, then you exclude every explanation, including those not yet considered. The first null hypothesis was that Muriel Bristol (Fisher's boss) could not correctly detect the difference between tea poured into mild from milk poured into tea. That was the very first statistical test.

There is a slight difference between R.A. Fisher's idea of a null and Pearson and Neyman's idea of a null. Fisher felt there was a null, but no alternative hypothesis. If you rejected the null, it told you what was wrong, but was not directive as to what was correct automatically. Pearson and Neyman championed acceptance and rejection regions based on frequencies, and they felt the method directed behavior.

The logic was that the method created a probabilistic version of modus tollens. Modus tollens is "if A then B; and, not B; therefore, not A." Or, if the null is true, then the test will appear in the acceptance region if it does not, then you can reject the null.

The weakness of the methodology was proposed by an author that I cannot currently locate in this somewhat tongue-in-cheek way. There are 535 elected members representing the states in the U.S. Congress. There are 360 million Americans. Therefore since 535/360000000 is less than .05, if you randomly sample a group of Americans and pick a member of Congress, they cannot be an American (p<.05).

While Fisher's no effect hypothesis is the most common version, because of its implication would be that something has an effect in the alternative, it is not a requirement that a parameter equals zero, or a set of parameters all equal zero.

What matters is that the null is the opposite of what you are wanting to assert before seeing the data.

That makes the null hypothesis method a potent tool. Think about this as a rhetorical device. Your opponent opposes that you recently believe you have discovered.

You do not assert is true. You assert your opponent's position of is true and build your probabilities on the assumption that you are the only person that is wrong. Everyone is right, and you are wrong.

It is a powerful rhetorical tool to concede the argument from the beginning, but then ask, "what would the world look like if I am the one that is wrong?" That is the null. If you reject the null, then what you are really saying is that "nature rejects all other ideas except mine."

Now as to your question, you want to show that college algebra matters, therefore your null hypothesis is that college algebra does not matter. We will ignore all the other methodological issues that would really be present since people without college algebra may have other self-selection issues as will the people with college algebra.

Your null is that algebra does not matter. The alternative is that it does. If the p-value is less than your cutoff, chosen before collecting the data, then you can reject the null. If it is not, then you should behave as if it is true until you either do more research or find another way to come to a conclusion.

It would be dubious, ignoring the methodology issues, to assert that college algebra matters as you only have one sample. The method is intended for repetition. Nonetheless, you would only be made a fool of no more than percent of the time, ignoring the methodological issues by following the results of the test.

I'll start with a quote for context and to point to a helpful resource that might have an answer for the OP. It's from V. Amrhein, S. Greenland, and B. McShane. Scientists rise up against statistical significance. Nature, 567:305–307, 2019. https://doi.org/10.1038/d41586-019-00857-9

We must learn to embrace uncertainty.

I understand it to mean that there is no need to state that we reject a hypothesis, accept a hypothesis, or don't reject a hypothesis to explain what we've learned from a statistical analysis. The accept/reject language implies certainty; statistics is better at quantifying uncertainty.

Note: I assume the question refers to making a binary reject/accept choice dictated by the significance (P ≤ 0.05) or non-significance (P > 0.05) of a p-value P.

The simplest way to understand hypothesis testing (NHST) — at least for me — is to keep in mind that p-values are probabilities about the data (not about the null and alternative hypotheses): Large p-value means that the data is consistent with the null hypothesis, small p-value means that the data is inconsistent with the null hypothesis. NHST doesn't tell us what hypothesis to reject and/or accept so that we have 100% certainty in our decision: hypothesis testing doesn't prove anything^٭. The reason is that a p-value is computed by assuming the null hypothesis is true [3].

So rather than wondering if, on calculating P ≤ 0.05, it's correct to declare that you "reject the null hypothesis" (technically correct) or "accept the alternative hypothesis" (technically incorrect), don't make a reject/don't reject determination but report what you've learned from the data: report the p-value or, better yet, your estimate of the quantity of interest and its standard error or confidence interval.

٭ Probability ≠ proof. For illustration, see this story about a small p-value at CERN leading scientists to announce they might have discovered a brand new force of nature: New physics at the Large Hadron Collider? Scientists are excited, but it’s too soon to be sure. Includes a bonus explanation of p-values.

References

[1] S. Goodman. A dirty dozen: Twelve p-value misconceptions. Seminars in Hematology, 45(3):135–140, 2008. https://doi.org/10.1053/j.seminhematol.2008.04.003

All twelve misconceptions are important to study, understand and avoid. But Misconception #12 is particularly relevant to this question: It's not the case that A scientific conclusion or treatment policy should be based on whether or not the P value is significant.

Steven Goodman explains: "This misconception (...) is equivalent to saying that the magnitude of effect is not relevant, that only evidence relevant to a scientific conclusion is in the experiment at hand, and that both beliefs and actions flow directly from the statistical results."

[2] Using p-values to test a hypothesis in Improving Your Statistical Inferences by Daniël Lakens.

This is my favorite explanation of p-values, their history, theory and misapplications. Has lots of examples from the social sciences.

[3] What is the meaning of p values and t values in statistical tests?