I'll start with a quote for context and to point to a helpful resource that might have an answer for the OP. It's from V. Amrhein, S. Greenland, and B. McShane. Scientists rise up against statistical significance. Nature, 567:305–307, 2019. https://doi.org/10.1038/d41586-019-00857-9
We must learn to embrace uncertainty.
I understand it to mean that there is no need to state that we reject a hypothesis, accept a hypothesis, or don't reject a hypothesis to explain what we've learned from a statistical analysis. The accept/reject language implies certainty; statistics is better at quantifying uncertainty.
Note: I assume the question refers to making a binary reject/accept choice dictated by the significance (P ≤ 0.05) or non-significance (P > 0.05) of a p-value P.
The simplest way to understand hypothesis testing (NHST) — at least for me — is to keep in mind that p-values are probabilities about the data (not about the null and alternative hypotheses): Large p-value means that the data is consistent with the null hypothesis, small p-value means that the data is inconsistent with the null hypothesis. NHST doesn't tell us what hypothesis to reject and/or accept so that we have 100% certainty in our decision: hypothesis testing doesn't prove anything٭. The reason is that a p-value is computed by assuming the null hypothesis is true [3].
So rather than wondering if, on calculating P ≤ 0.05, it's correct to declare that you "reject the null hypothesis" (technically correct) or "accept the alternative hypothesis" (technically incorrect), don't make a reject/don't reject determination but report what you've learned from the data: report the p-value or, better yet, your estimate of the quantity of interest and its standard error or confidence interval.
٭ Probability ≠ proof. For illustration, see this story about a small p-value at CERN leading scientists to announce they might have discovered a brand new force of nature: New physics at the Large Hadron Collider? Scientists are excited, but it’s too soon to be sure. Includes a bonus explanation of p-values.
References
[1] S. Goodman. A dirty dozen: Twelve p-value misconceptions. Seminars in Hematology, 45(3):135–140, 2008. https://doi.org/10.1053/j.seminhematol.2008.04.003
All twelve misconceptions are important to study, understand and avoid. But Misconception #12 is particularly relevant to this question: It's not the case that A scientific conclusion or treatment policy should be based on whether or not the P value is significant.
Steven Goodman explains: "This misconception (...) is equivalent to saying that the magnitude of effect is not relevant, that only evidence relevant to a scientific conclusion is in the experiment at hand, and that both beliefs and actions flow directly from the statistical results."
[2] Using p-values to test a hypothesis in Improving Your Statistical Inferences by Daniël Lakens.
This is my favorite explanation of p-values, their history, theory and misapplications. Has lots of examples from the social sciences.
[3] What is the meaning of p values and t values in statistical tests?
Answer from dipetkov on Stack ExchangeI have been thinking about how we do not accept a null hypothesis if we reject it, and I am not sure if i do not understand it well enough, what I think is that we do not accept the null hypothesis because when we fail to reject the null hypothesis we are only saying that the alternative hypothesis is incorrect but that does not make it impossible to another alternative hypothesis to appear and this one be correct. Please let me know if this is correct
In case that the last paragraph is correct then I do not know why we say that we do not accept the null hypothesis if this is based in how we think things are, would it not be more appropiate to say that the null hypothesis is correct when we compare it to the the alternative that we just reject, because we do not know which alternative hypothesis might make us reject the null
Thank you
I'll start with a quote for context and to point to a helpful resource that might have an answer for the OP. It's from V. Amrhein, S. Greenland, and B. McShane. Scientists rise up against statistical significance. Nature, 567:305–307, 2019. https://doi.org/10.1038/d41586-019-00857-9
We must learn to embrace uncertainty.
I understand it to mean that there is no need to state that we reject a hypothesis, accept a hypothesis, or don't reject a hypothesis to explain what we've learned from a statistical analysis. The accept/reject language implies certainty; statistics is better at quantifying uncertainty.
Note: I assume the question refers to making a binary reject/accept choice dictated by the significance (P ≤ 0.05) or non-significance (P > 0.05) of a p-value P.
The simplest way to understand hypothesis testing (NHST) — at least for me — is to keep in mind that p-values are probabilities about the data (not about the null and alternative hypotheses): Large p-value means that the data is consistent with the null hypothesis, small p-value means that the data is inconsistent with the null hypothesis. NHST doesn't tell us what hypothesis to reject and/or accept so that we have 100% certainty in our decision: hypothesis testing doesn't prove anything٭. The reason is that a p-value is computed by assuming the null hypothesis is true [3].
So rather than wondering if, on calculating P ≤ 0.05, it's correct to declare that you "reject the null hypothesis" (technically correct) or "accept the alternative hypothesis" (technically incorrect), don't make a reject/don't reject determination but report what you've learned from the data: report the p-value or, better yet, your estimate of the quantity of interest and its standard error or confidence interval.
٭ Probability ≠ proof. For illustration, see this story about a small p-value at CERN leading scientists to announce they might have discovered a brand new force of nature: New physics at the Large Hadron Collider? Scientists are excited, but it’s too soon to be sure. Includes a bonus explanation of p-values.
References
[1] S. Goodman. A dirty dozen: Twelve p-value misconceptions. Seminars in Hematology, 45(3):135–140, 2008. https://doi.org/10.1053/j.seminhematol.2008.04.003
All twelve misconceptions are important to study, understand and avoid. But Misconception #12 is particularly relevant to this question: It's not the case that A scientific conclusion or treatment policy should be based on whether or not the P value is significant.
Steven Goodman explains: "This misconception (...) is equivalent to saying that the magnitude of effect is not relevant, that only evidence relevant to a scientific conclusion is in the experiment at hand, and that both beliefs and actions flow directly from the statistical results."
[2] Using p-values to test a hypothesis in Improving Your Statistical Inferences by Daniël Lakens.
This is my favorite explanation of p-values, their history, theory and misapplications. Has lots of examples from the social sciences.
[3] What is the meaning of p values and t values in statistical tests?
Say you have the hypothesis
"on stackexchange there is not yet an answer to my question"
When you randomly sample 1000 questions then you might find zero answers. Based on this, can you 'accept' the null hypothesis?
You can read about this among many older questions and answers, for instance:
- Why do statisticians say a non-significant result means "you can't reject the null" as opposed to accepting the null hypothesis?
- Why do we need alternative hypothesis?
- Is it possible to accept the alternative hypothesis?
Also check out the questions about two one-sided tests (TOST) which is about formulating the statement behind a null hypothesis in a way such that it can be a statement that you can potentially 'accept'.
More seriously, a problem with the question is that it is unclear. What does 'accept' actually mean?
And also, it is a loaded question. It asks for something that is not true. Like 'why is it that the earth is flat, but the moon is round?'.
There is no 'acceptance' of an alternative theory. Or at least, when we 'accept' some alternative hypothesis then either:
- Hypothesis testing: the alternative theory is extremely broad and reads as 'something else than the null hypothesis is true'. Whatever this 'something else' means, that is left open. There is no 'acceptance' of a particular theory. See also: https://en.m.wikipedia.org/wiki/Falsifiability
- Expression of significance: or 'acceptance' means that we observed an effect, and consider it as a 'significant' effect. There is no literal 'acceptance' of some theory/hypothesis here. There is just the consideration that we found that the data shows there is some effect and it is significantly different from a case when to there would be zero effect. Whether this means that the alternative theory should be accepted, that is not explicitly stated and should also not be assumed implicitly. The alternative hypothesis (related to the effect) works for the present data, but that is different from being accepted, (it just has not been rejected yet).
Videos
Why can a null hypothesis not be accepted?
We can’t accept a null hypothesis because a lack of evidence does not prove something that does not exist. Instead, we fail to reject it.
Failing to reject the null indicates that the sample did not provide sufficient enough evidence to conclude that an effect exists.
If the p-value is greater than the significance level, then you fail to reject the null hypothesis.
What are some problems with the null hypothesis?
What is the difference between a null hypothesis and an alternative hypothesis?
It is the claim that you expect or hope will be true. The null hypothesis and the alternative hypothesis are always mutually exclusive, meaning that only one can be true at a time.