# What's the Value of the P Value?

Many studies published in current scientific journals include a P value as part of the data analysis, often as an indication of statistical significance of the results. What does the P value mean? A P value is related to the null hypothesis (i.e., there is no difference between the no-treatment and treatment groups) and represents the probability of obtaining the observed results if the null hypothesis is true (1). For example, a P value of 0.05 represents a 5% probability of obtaining the observed results if the null hypothesis (no difference) is true.

In a recent Viewpoint article published in JAMA (2), John P.A. Ioannidis discusses a proposal to lower the P value threshold for statistical significance from 0.05 to 0.005, while also highlighting many limitations with the use of P values. The first limitation is that P values are often misinterpreted as providing evidence that a given hypothesis is either true or false. The second limitation is that P values are overtrusted, when the P value can be highly influenced by factors such as sample size or selective reporting of data. The third limitation discussed by Ioannidis is that P values are often misused to draw conclusions about the research. Additionally, Ioannidis highlights that many results with P values <0.05, yet still close to that threshold, may not represent true effects. Lowering the P value threshold to 0.005 will likely increase the number of true effects that are reported in the literature.

Lowering the P value threshold was not the only proposal for improving statistical significance Ioannidis mentions. Several options, such as publishing the actual P value rather than a threshold or addressing the underlying biases in published research, would be difficult to apply retrospectively to previously published studies. Moving forward, several steps can be taken to help improve statistical significance of published research, such as focusing on the magnitude of the effect or the use of alternative statistical approaches for data analysis, as well as training the scientific workforce on statistical methods of analysis.

Ioannidis acknowledges that lowering the P value threshold is not a perfect solution, but it does represent an improvement in the presentation of research results. More robust solutions to statistical significance of data are likely to take time both to implement and to achieve wide adoption by scientists and scientific journals. While long-term solutions continue to be discussed, perhaps it would be beneficial for all members of the scientific community to review the strengths, limitations, and appropriate use of the P value.

Author Contributions: AH authors confirmed they have contributed to the intellectual content of this paper and have met the following 4 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved.

Authors' Disclosures or Potential Conflicts of Interest: No authors declared any potential conflicts of interest.

Received August 6, 2018; accepted August 9, 2018.

DOI: 10.1373/clinchem.2018.293415

References

(1.) Motulsky H. Intuitive biostatistics: a nonmathematical guideto statistical thinking. 2nd Ed. New York (NY): Oxford University Press; 2010.

(2.) Ioannidis JPA. The proposal to lower P value thresholds to .005. JAMA 2018;319: 1429-30.

Sarah A. Hackenmueller * ([dagger])