ANÁLISE DA FIDEDIGNIDADE COMPOSTA DOS ESCORES DO ENEM POR MEIO DA ANÁLISE FATORIAL DE ITENS / ANALYSIS OF THE COMPOSITE RELIABILITY OF THE SCORES OF ENEM VIA FACTOR ANALYSIS OF ITEMS
Abstract
O Exame Nacional do Ensino Médio (ENEM) é composto por quatro domínios: Ciências da Natureza, Ciências Humanas, Linguagens e Matemática. O presente estudo busca avaliar a fidedignidade dos escores de cada domínio por meio das seguintes estratégias combinadas: emprego da análise fatorial de itens em duas vertentes, sendo elas a análise fatorial confirmatória e a modelagem exploratória por equação estrutural; o uso da modelagem bifatorial; e o cálculo da fidedignidade composta para os quatro escores do ENEM e o escore geral de desempenho escolar, obtidos pelas estratégias anteriores. Os dados analisados são referentes aos escores dos estudantes que fizeram a prova de 2011. Os resultados apontaram uma fidedignidade satisfatória para o escore geral e os escores nos domínios de Matemática, Ciências da Natureza e Linguagens, enquanto o domínio de Ciências Humanas apresentou uma fidedignidade insatisfatória. Implicações são discutidas.
The National Exam of Upper Secondary Education (Exame Nacional do Ensino Médio [ENEM]) consists of four domains: Natural Sciences, Humanities, Languages, and Mathematics. The present study aims to evaluate the reliability of the scores of each domain through the following strategies: use of item factor analysis via confirmatory factor analysis and exploratory structural equation modeling; bifactor model; and the calculation of the composite reliability for the scores from the four domains, plus the general performance factor, all of them obtained by the previous analyzes. The data of the study are composed of the scores of the students who took the 2011 ENEM. The results showed a satisfactory reliability for the scores of the general factor, and for the domains of Mathematics, Nature Sciences, and Languages, while the Humanities domain showed an unsatisfactory reliability. Implications are discussed.
Article visualizations:
Keywords
Full Text:
PDFReferences
Almeida, Marco Antonio Bettine de, & Sanchez, Lívia Pizauro. (2016). ENEM: ferramenta de implementação da Lei 10.639/2003 - Competências para a transformação social? Educação em Revista, 32(1), 79-103. doi: 10.1590/0102- 4698141429
American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing. (2014). Standards for educational and psychological testing.
Andrade, Eduardo, & Soida, Ivan. (2015). A qualidade do ranking das escolas de ensino médio baseado no ENEM é questionável. Estudos Econômicos (São Paulo), 45(2), 253-286. doi: 10.1590/0101-4161201545221eai
Bentler, P.M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238-246. doi: 10.1037/0033-2909.107.2.238
Browne, M.W. & Cudeck, R. (1993). Alternative ways of assessing model fit. In: Bollen, K.A., Long, J.S. (Ed.) Testing structural equation models (pp.136 -162). Newbury Park: Sage.
Cucina, J., & Byle, K. (2017). The bifactor model fits better than the higher-order model in more than 90% of comparisons for mental abilities test batteries. Journal of Intelligence, 5(27), 1-21. doi: 10.3390/jintelligence5030027
Fornell, C., & Larcker, D.F. (1981). Evaluating structural equations models with unobservable variables and measurement error. Journal of Marketing,18(1), 39- 50. doi: 10.2307/3151312
Golino, H.F. (2014). ENEM: an implementation of functions to help automatic downloading, importing, cleaning and scoring of the Brazilian's National High School Exam (ENEM). Software não-publicado.
Gomes, C.M.A., Almeida, L.S., & Núñez, J.C. (2017). Rationale and Applicability of Exploratory Structural Equation Modeling (ESEM) in psychoeducational contexts. Psicothema, 29(3), 396-401. doi: 10.7334/psicothema2016.369.
Graham, J.M., 2006. Congeneric and (essentially) tau-equivalent estimates of score reliability. What they are and how to use them. Educational and Psychological Measurement, 66(6), 930-944. doi: 10.1177/0013164406288165
Haguette, A., Pessoa, M.K.M., & Vidal, E.M. (2016). Dez escolas, dois padrões de qualidade. Uma pesquisa em dez escolas públicas de Ensino Médio do Estado do Ceará. Ensaio: Avaliação e Políticas Públicas em Educação, 24(92), 609-636. doi: 10.1590/S0104-40362016000300005
Hair, J.F., Black, W.C., Babin, B.J., & Anderson, R.E. (2013). Multivariate data analysis. Pearson new international edition (7th ed.). Essex, UK: Pearson Education Limited.
Hallquist, M. & Wiley, J. (2014). MplusAutomation: Automating Mplus Model Estimation and Interpretation. R package. (Version 0.6-3) [Software]. Available from http://CRAN.R-project.org/package=MplusAutomation
Hogan, T.P. (2013). Psychological testing: A practical introduction (3rd ed.). Hoboken, USA: John Wiley & Sons.
Hu, L.T. & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. doi: 10.1080/10705519909540118
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira - Inep. (2016). Overview of the Brazilian education system. Brasília: Inep.
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira – Inep. (2012a). Microdados do ENEM – 2011. Exame Nacional do Ensino Médio: Manual do Usuário. Retrieved from http://portal.inep.gov.br/web/guest/microdados. Access 17 September 2018.
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira - Inep. (2012b). Matriz de Referência do ENEM. Retrieved from http://download.inep.gov.br/educacao_basica/enem/downloads/2012/matriz_refe rencia_enem.pdf. Access 17 September 2018.
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira - Inep. (2018). ENEM. Retrieved from portal.inep.gov.br/web/guest/enem Access 17 September 2018.
Luna, T.S., & M.,B. (2015). Letramentos literários: o que se Avalia No Exame Nacional Do Ensino Médio? Educação em Revista, 31(3), 195-224. doi: 10.1590/0102- 4698135569
Muthén, L.K. & Muthén, B.O. (2014). Mplus users's guide. Los Angeles: Muthén & Muthén.
R. Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R- project.org/.
Reise, S.P., Bonifay, W., & Haviland, M.G., 2018). Bifactor modelling and the evaluation of scale scores. In P. Irwing, T. Booth, & D. J. Hughes (Eds.). The Wiley Handbook of Psychometric Testing: A multidisciplinary reference on survey, scale and test development (Vols. 1-2) (pp. 677-706). Hoboken, NJ, EUA: John Wiley & Sons.
Reise, S.P., Moore, T.M., & Haviland, M.G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92(6), 544–559. doi: 10.1080/00223891.2010.496477
Revelle, W., & Condon, D.M. (2018). Reliability. In P. Irwing, T. Booth, & D. J. Hughes (Eds.). The Wiley Handbook of Psychometric Testing: A multidisciplinary reference on survey, scale and test development (Vols. 1-2) (pp. 709-749). Hoboken, NJ, EUA: John Wiley & Sons.
Revelle, W., & Zinbarg, R.E. (2009). Coefficients Alpha, Beta, Omega, and the GLB: Comments on Sijtsma, doi: 10.1007/S11336-008-9102-Z
Schmitt, N. (1996). Uses and abuses of coefficient Alpha. Psychological Assessment, 8(4), 350-353. doi: 1040-3590/96/$300
Schumacker, R.E., & Lomax, R.G. (2004). A beginner’s guide to structural equation modeling. London: Lawrence Erlbaum Associates.
Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s Alpha. Psychometrika. 74(1):107-120. doi: 10.1007/s11336-008-9101-0
Silveira, F.L., Barbosa, M.C.B., & Silva, R. (2015). Exame Nacional do Ensino Médio (ENEM): Uma análise crítica. Revista Brasileira de Ensino de Física, 37(1), 1101. doi: 10.1590/S1806-11173710001
Taber, K.S. (2017). The use of Cronbach’s Alpha when developing and reporting research instruments in science education. Research in Science Education, 1-24. doi: 10.1007/s11165-016-9602-2.
Travitzki, R. (2017). Avaliação da qualidade do Enem 2009 e 2011 com técnicas psicométricas. Estudos em Avaliação Educacional, 28(67), 256-288. doi: 10.18222/eae.v28i67.3910
Valentini, F., Gomes, C.M.A., Muniz, M., Mecca, T.P., Laros, J.A., & Andrade, J.M. (2015). Confiabilidade dos índices fatoriais da Wais-III adaptada para a população brasileira. Psicologia: Teoria e Prática, 17(2), 123-139. doi: 10.15348/1980-6906/psicologia.v17n2p123-139
DOI: http://dx.doi.org/10.46827/ejes.v0i0.2178
Refbacks
- There are currently no refbacks.
Copyright (c) 2018 Cristiano Mauro Assis Gomes, Hudson Fernandes Golino, Alexandre José de Souza Peres
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright © 2015-2023. European Journal of Education Studies (ISSN 2501 - 1111) is a registered trademark of Open Access Publishing Group. All rights reserved.
This journal is a serial publication uniquely identified by an International Standard Serial Number (ISSN) serial number certificate issued by Romanian National Library (Biblioteca Nationala a Romaniei). All the research works are uniquely identified by a CrossRef DOI digital object identifier supplied by indexing and repository platforms. All authors who send their manuscripts to this journal and whose articles are published on this journal retain full copyright of their articles. All the research works published on this journal are meeting the Open Access Publishing requirements and can be freely accessed, shared, modified, distributed and used in educational, commercial and non-commercial purposes under a Creative Commons Attribution 4.0 International License (CC BY 4.0).
