SECOND-GENERATION P-VALUES: A NEW PRACTICAL APPROACH FOR EDUCATIONAL RESEARCH

Sally A. Lesik, Anna M. Duffin

Abstract


The limitations, misconceptions, and misuse of traditional p-values remain a concern for researchers in education. Second-generation p-values provide a descriptive measure that can be used to decide between the null and alternative hypotheses. They are intuitive and easy to understand and provide a new approach to addressing some of the limitations of traditional p-values, mostly by considering the practical implication of an estimated effect. This article describes the general idea and principles behind second-generation p-values and illustrates their application in educational research and data science. Limitations and directions for further study are discussed.

 

Article visualizations:

Hit counter


Keywords


statistical inference; p-value; second-generation p-value; interval null hypothesis; effect size; practical significance; statistical significance

Full Text:

PDF

References


Armstrong, J.S. and Hubbard, R. (2008). Why we don’t really know what “statistical significance” means: A major educational failure, SSRN Electronic Journal [Preprint]. doi:10.2139/ssrn.1154386.

Blume, J.D., Greevy, R.A., Welty, V.F., Smith, J.R. and Dupont, W.D. (2019). An Introduction to Second-Generation p-Values. The American Statistician, 73(sup1), pp.157–167. doi: https://doi.org/10.1080/00031305.2018.1537893.

Blume, J.D, D’Agostino McGowan, L, Dupont, W.D, Greevy, R.A., (2018) Second- generation p-values: Improved rigor, reproducibility, & transparency in statistical analyses. PLoS ONE 13(3): e0188299. https://doi.org/10.1371/journal.pone.0188299.

Bormann, S.-K. (2022). ‘A Stata implementation of second-generation P-values’, The Stata Journal: Promoting communications on statistics and Stata, 22(3), pp. 496–520. doi:10.1177/1536867x221124466.

Campbell, H. and Gustafson, P. (2019). ‘The World of Research Has Gone Berserk: Modeling the Consequences of Requiring “Greater Statistical Stringency” for Scientific Publication’, The American Statistician, 73(sup1), pp. 358–373. doi: 10.1080/00031305.2018.1555101.

Cohen, M.P. (2021). ‘Why not an interval null hypothesis?’, Journal of Data Science, 17(2), pp. 383–390. doi:10.6339/jds.201904_17(2).0008.

Colquhoun, D. (2017). The reproducibility of research and the misinterpretation of p -values. Royal Society Open Science, 4(12), p.171085. doi: https://doi.org/10.1098/rsos.171085.

Das D, Das T. (2023). The “P”‑value: The primary alphabet of research revisited. International Journal Preventive Medicine, 14(41).

Diaz-Quijano, F.A., Calixto, F.M. and Da Silva, J.M.N. (2020). 'How feasible is it to abandon statistical significance? A reflection based on a short survey,' BMC Medical Research Methodology, 20(1). https://doi.org/10.1186/s12874-020-01030-x.

Di Leo, G. and Sardanelli, F. (2020). Statistical significance: p value, 0.05 threshold, and applications to radiomics—reasons for a conservative approach. European Radiology Experimental, [online] 4(1). doi: https://doi.org/10.1186/s41747-020-0145-y.

Edgeworth, F.Y. (1885). Methods of Statistics, Jubilee volume of the Statist. Soc., E.

Gao, J. (2020). P-values – a chronic conundrum. BMC Medical Research Methodology, 20(1). doi: https://doi.org/10.1186/s12874-020-01051-6.

Goodman, S. (2008). A Dirty Dozen: Twelve P-Value Misconceptions. Seminars in Hematology, 45(3), pp.135–140. doi: https://doi.org/10.1053/j.seminhematol.2008.04.003.

Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical test, p value, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology, 31(4), 337-350. DOI: 10.1007/s10654-016-0149-3.

Hurlbert, S. H., Levine, R. A. and Utts, J. (2019). Coup de Grâce for a Tough Old Bull: “Statistically Significant” Expires’, The American Statistician, 73(sup1), pp. 352–357. doi: 10.1080/00031305.2018.1543616.

Kmetz, J. L. (2019). Correcting Corrupt Research: Recommendations for the Profession to Stop Misuse of p-Values. The American Statistician, 73(sup1), 36–45. https://doi.org/10.1080/00031305.2018.1518271.

Lin, M., Lucas, H.C. and Shmueli, G. (2013). ‘research commentary—too big to fail: Large samples and the p-value problem’, Information Systems Research, 24(4), pp. 906–917. doi:10.1287/isre.2013.0480.

McShane, B.B., Gal, D., Gelman, A., Robert, C. and Tackett, J.L. (2019). Abandon Statistical Significance. The American Statistician, [online] 73(sup1), pp.235–245. doi: https://doi.org/10.1080/00031305.2018.1527253.

Package ‘SGPV’. (2022). Available at: https://cran.r-project.org/web/packages/sgpv/sgpv.pdf [Accessed 2 May 2024].

Pericchi, L., Pereira, C., and Perez, M. (2014). Adaptive revised standards for statistical evidence. Proceedings of the National Academy of Sciences of the United States of America. 111. 10.1073/pnas.1322191111.

Robert,D.(2020, June26).Demystifying the p-value. The Startup. https://www.tandfonline.com/doi/full/10.1080/00031305.2019.1583913.

Stahel, W.A., (2021). New relevance and significance measures to replace p-values. PLoS ONE 16(6): e0252991. https://doi.org/10.1371/journal.pone.0252991.

Stewart, T.G. and Blume, J.D. (2019). Second-Generation P-Values, Shrinkage, and Regularized Models. Frontiers in Ecology and Evolution, 7. doi: https://doi.org/10.3389/fevo.2019.00486.

Vidgen, B. and Yasseri, T. (2016). ‘P-values: Misunderstood and misused’, Frontiers in Physics, 4. doi:10.3389/fphy.2016.00006.

Wasserstein, R.L., Schirm, A. L. and Lazar, N. A. (2019). ‘Moving to a World Beyond “p < 0.05”’, The American Statistician, 73(sup1), pp. 1–19. doi: 10.1080/00031305.2019.1583913.

Wasserstein, R.L. and Lazar, N.A. (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician, [online] 70(2), pp.129–133. doi: https://doi.org/10.1080/00031305.2016.1154108 .

Zuo, Y., Stewart, T. G., & Blume, J. D. (2022). ProSGPV: An R package for variable selection with second-generation P-values. F1000Research, 11, 58. https://doi.org/10.12688/f1000research.74401.1.




DOI: http://dx.doi.org/10.46827/ejes.v11i12.5712

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Sally A. Lesik, Anna M. Duffin

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2015-2023. European Journal of Education Studies (ISSN 2501 - 1111) is a registered trademark of Open Access Publishing Group. All rights reserved.

This journal is a serial publication uniquely identified by an International Standard Serial Number (ISSN) serial number certificate issued by Romanian National Library (Biblioteca Nationala a Romaniei). All the research works are uniquely identified by a CrossRef DOI digital object identifier supplied by indexing and repository platforms. All authors who send their manuscripts to this journal and whose articles are published on this journal retain full copyright of their articles. All the research works published on this journal are meeting the Open Access Publishing requirements and can be freely accessed, shared, modified, distributed and used in educational, commercial and non-commercial purposes under a Creative Commons Attribution 4.0 International License (CC BY 4.0).