What’s the difference between a research hypothesis and a statistical hypothesis?
What is hypothesis testing?
What are null and alternative hypotheses?
I understand that we either "reject" or "fail to reject" the null hypothesis. But in either case, what about the alternative hypothesis?
I.e. if we reject the null hypothesis, do we accept the alternative hypothesis?
Similarly, if we fail to reject the null hypothesis, do we reject the alternative hypothesis?
I'll start with a quote for context and to point to a helpful resource that might have an answer for the OP. It's from V. Amrhein, S. Greenland, and B. McShane. Scientists rise up against statistical significance. Nature, 567:305–307, 2019. https://doi.org/10.1038/d41586-019-00857-9
We must learn to embrace uncertainty.
I understand it to mean that there is no need to state that we reject a hypothesis, accept a hypothesis, or don't reject a hypothesis to explain what we've learned from a statistical analysis. The accept/reject language implies certainty; statistics is better at quantifying uncertainty.
Note: I assume the question refers to making a binary reject/accept choice dictated by the significance (P ≤ 0.05) or non-significance (P > 0.05) of a p-value P.
The simplest way to understand hypothesis testing (NHST) — at least for me — is to keep in mind that p-values are probabilities about the data (not about the null and alternative hypotheses): Large p-value means that the data is consistent with the null hypothesis, small p-value means that the data is inconsistent with the null hypothesis. NHST doesn't tell us what hypothesis to reject and/or accept so that we have 100% certainty in our decision: hypothesis testing doesn't prove anything٭. The reason is that a p-value is computed by assuming the null hypothesis is true [3].
So rather than wondering if, on calculating P ≤ 0.05, it's correct to declare that you "reject the null hypothesis" (technically correct) or "accept the alternative hypothesis" (technically incorrect), don't make a reject/don't reject determination but report what you've learned from the data: report the p-value or, better yet, your estimate of the quantity of interest and its standard error or confidence interval.
٭ Probability ≠ proof. For illustration, see this story about a small p-value at CERN leading scientists to announce they might have discovered a brand new force of nature: New physics at the Large Hadron Collider? Scientists are excited, but it’s too soon to be sure. Includes a bonus explanation of p-values.
References
[1] S. Goodman. A dirty dozen: Twelve p-value misconceptions. Seminars in Hematology, 45(3):135–140, 2008. https://doi.org/10.1053/j.seminhematol.2008.04.003
All twelve misconceptions are important to study, understand and avoid. But Misconception #12 is particularly relevant to this question: It's not the case that A scientific conclusion or treatment policy should be based on whether or not the P value is significant.
Steven Goodman explains: "This misconception (...) is equivalent to saying that the magnitude of effect is not relevant, that only evidence relevant to a scientific conclusion is in the experiment at hand, and that both beliefs and actions flow directly from the statistical results."
[2] Using p-values to test a hypothesis in Improving Your Statistical Inferences by Daniël Lakens.
This is my favorite explanation of p-values, their history, theory and misapplications. Has lots of examples from the social sciences.
[3] What is the meaning of p values and t values in statistical tests?
Say you have the hypothesis
"on stackexchange there is not yet an answer to my question"
When you randomly sample 1000 questions then you might find zero answers. Based on this, can you 'accept' the null hypothesis?
You can read about this among many older questions and answers, for instance:
- Why do statisticians say a non-significant result means "you can't reject the null" as opposed to accepting the null hypothesis?
- Why do we need alternative hypothesis?
- Is it possible to accept the alternative hypothesis?
Also check out the questions about two one-sided tests (TOST) which is about formulating the statement behind a null hypothesis in a way such that it can be a statement that you can potentially 'accept'.
More seriously, a problem with the question is that it is unclear. What does 'accept' actually mean?
And also, it is a loaded question. It asks for something that is not true. Like 'why is it that the earth is flat, but the moon is round?'.
There is no 'acceptance' of an alternative theory. Or at least, when we 'accept' some alternative hypothesis then either:
- Hypothesis testing: the alternative theory is extremely broad and reads as 'something else than the null hypothesis is true'. Whatever this 'something else' means, that is left open. There is no 'acceptance' of a particular theory. See also: https://en.m.wikipedia.org/wiki/Falsifiability
- Expression of significance: or 'acceptance' means that we observed an effect, and consider it as a 'significant' effect. There is no literal 'acceptance' of some theory/hypothesis here. There is just the consideration that we found that the data shows there is some effect and it is significantly different from a case when to there would be zero effect. Whether this means that the alternative theory should be accepted, that is not explicitly stated and should also not be assumed implicitly. The alternative hypothesis (related to the effect) works for the present data, but that is different from being accepted, (it just has not been rejected yet).
Im just always bugged by this question on our homeworks and Im starting to see the pattern. I just dont know if my assumptions are going to lead me to the right track. Thanks in advance!