ANÁLISE DA FIDEDIGNIDADE COMPOSTA DOS ESCORES DO ENEM POR MEIO DA ANÁLISE FATORIAL DE ITENS / ANALYSIS OF THE COMPOSITE RELIABILITY OF THE SCORES OF ENEM VIA FACTOR ANALYSIS OF ITEMS

Cristiano Mauro Assis Gomes, Hudson Fernandes Golino, Alexandre José de Souza Peres

Abstract


O Exame Nacional do Ensino Médio (ENEM) é composto por quatro domínios: Ciências da Natureza, Ciências Humanas, Linguagens e Matemática. O presente estudo busca avaliar a fidedignidade dos escores de cada domínio por meio das seguintes estratégias combinadas: emprego da análise fatorial de itens em duas vertentes, sendo elas a análise fatorial confirmatória e a modelagem exploratória por equação estrutural; o uso da modelagem bifatorial; e o cálculo da fidedignidade composta para os quatro escores do ENEM e o escore geral de desempenho escolar, obtidos pelas estratégias anteriores. Os dados analisados são referentes aos escores dos estudantes que fizeram a prova de 2011. Os resultados apontaram uma fidedignidade satisfatória para o escore geral e os escores nos domínios de Matemática, Ciências da Natureza e Linguagens, enquanto o domínio de Ciências Humanas apresentou uma fidedignidade insatisfatória. Implicações são discutidas.

 

The National Exam of Upper Secondary Education (Exame Nacional do Ensino Médio [ENEM]) consists of four domains: Natural Sciences, Humanities, Languages, and Mathematics. The present study aims to evaluate the reliability of the scores of each domain through the following strategies: use of item factor analysis via confirmatory factor analysis and exploratory structural equation modeling; bifactor model; and the calculation of the composite reliability for the scores from the four domains, plus the general performance factor, all of them obtained by the previous analyzes. The data of the study are composed of the scores of the students who took the 2011 ENEM. The results showed a satisfactory reliability for the scores of the general factor, and for the domains of Mathematics, Nature Sciences, and Languages, while the Humanities domain showed an unsatisfactory reliability. Implications are discussed.

 

Article visualizations:

Hit counter

DOI

Keywords


Exame Nacional do Ensino Médio (ENEM); análise fatorial confirmatória; modelagem exploratória por equações estruturais; modelo bifactor; fidedignidade composta / National Exam of Secondary Education (ENEM); confirmatory factor analysis; exploratory struct

Full Text:

PDF

References


Almeida, Marco Antonio Bettine de, & Sanchez, Lívia Pizauro. (2016). ENEM: ferramenta de implementação da Lei 10.639/2003 - Competências para a transformação social? Educação em Revista, 32(1), 79-103. doi: 10.1590/0102- 4698141429

American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing. (2014). Standards for educational and psychological testing.

Andrade, Eduardo, & Soida, Ivan. (2015). A qualidade do ranking das escolas de ensino médio baseado no ENEM é questionável. Estudos Econômicos (São Paulo), 45(2), 253-286. doi: 10.1590/0101-4161201545221eai

Bentler, P.M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238-246. doi: 10.1037/0033-2909.107.2.238

Browne, M.W. & Cudeck, R. (1993). Alternative ways of assessing model fit. In: Bollen, K.A., Long, J.S. (Ed.) Testing structural equation models (pp.136 -162). Newbury Park: Sage.

Cucina, J., & Byle, K. (2017). The bifactor model fits better than the higher-order model in more than 90% of comparisons for mental abilities test batteries. Journal of Intelligence, 5(27), 1-21. doi: 10.3390/jintelligence5030027

Fornell, C., & Larcker, D.F. (1981). Evaluating structural equations models with unobservable variables and measurement error. Journal of Marketing,18(1), 39- 50. doi: 10.2307/3151312

Golino, H.F. (2014). ENEM: an implementation of functions to help automatic downloading, importing, cleaning and scoring of the Brazilian's National High School Exam (ENEM). Software não-publicado.

Gomes, C.M.A., Almeida, L.S., & Núñez, J.C. (2017). Rationale and Applicability of Exploratory Structural Equation Modeling (ESEM) in psychoeducational contexts. Psicothema, 29(3), 396-401. doi: 10.7334/psicothema2016.369.

Graham, J.M., 2006. Congeneric and (essentially) tau-equivalent estimates of score reliability. What they are and how to use them. Educational and Psychological Measurement, 66(6), 930-944. doi: 10.1177/0013164406288165

Haguette, A., Pessoa, M.K.M., & Vidal, E.M. (2016). Dez escolas, dois padrões de qualidade. Uma pesquisa em dez escolas públicas de Ensino Médio do Estado do Ceará. Ensaio: Avaliação e Políticas Públicas em Educação, 24(92), 609-636. doi: 10.1590/S0104-40362016000300005

Hair, J.F., Black, W.C., Babin, B.J., & Anderson, R.E. (2013). Multivariate data analysis. Pearson new international edition (7th ed.). Essex, UK: Pearson Education Limited.

Hallquist, M. & Wiley, J. (2014). MplusAutomation: Automating Mplus Model Estimation and Interpretation. R package. (Version 0.6-3) [Software]. Available from http://CRAN.R-project.org/package=MplusAutomation

Hogan, T.P. (2013). Psychological testing: A practical introduction (3rd ed.). Hoboken, USA: John Wiley & Sons.

Hu, L.T. & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. doi: 10.1080/10705519909540118

Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira - Inep. (2016). Overview of the Brazilian education system. Brasília: Inep.

Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira – Inep. (2012a). Microdados do ENEM – 2011. Exame Nacional do Ensino Médio: Manual do Usuário. Retrieved from http://portal.inep.gov.br/web/guest/microdados. Access 17 September 2018.

Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira - Inep. (2012b). Matriz de Referência do ENEM. Retrieved from http://download.inep.gov.br/educacao_basica/enem/downloads/2012/matriz_refe rencia_enem.pdf. Access 17 September 2018.

Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira - Inep. (2018). ENEM. Retrieved from portal.inep.gov.br/web/guest/enem Access 17 September 2018.

Luna, T.S., & M.,B. (2015). Letramentos literários: o que se Avalia No Exame Nacional Do Ensino Médio? Educação em Revista, 31(3), 195-224. doi: 10.1590/0102- 4698135569

Muthén, L.K. & Muthén, B.O. (2014). Mplus users's guide. Los Angeles: Muthén & Muthén.

R. Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R- project.org/.

Reise, S.P., Bonifay, W., & Haviland, M.G., 2018). Bifactor modelling and the evaluation of scale scores. In P. Irwing, T. Booth, & D. J. Hughes (Eds.). The Wiley Handbook of Psychometric Testing: A multidisciplinary reference on survey, scale and test development (Vols. 1-2) (pp. 677-706). Hoboken, NJ, EUA: John Wiley & Sons.

Reise, S.P., Moore, T.M., & Haviland, M.G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92(6), 544–559. doi: 10.1080/00223891.2010.496477

Revelle, W., & Condon, D.M. (2018). Reliability. In P. Irwing, T. Booth, & D. J. Hughes (Eds.). The Wiley Handbook of Psychometric Testing: A multidisciplinary reference on survey, scale and test development (Vols. 1-2) (pp. 709-749). Hoboken, NJ, EUA: John Wiley & Sons.

Revelle, W., & Zinbarg, R.E. (2009). Coefficients Alpha, Beta, Omega, and the GLB: Comments on Sijtsma, doi: 10.1007/S11336-008-9102-Z

Schmitt, N. (1996). Uses and abuses of coefficient Alpha. Psychological Assessment, 8(4), 350-353. doi: 1040-3590/96/$300

Schumacker, R.E., & Lomax, R.G. (2004). A beginner’s guide to structural equation modeling. London: Lawrence Erlbaum Associates.

Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s Alpha. Psychometrika. 74(1):107-120. doi: 10.1007/s11336-008-9101-0

Silveira, F.L., Barbosa, M.C.B., & Silva, R. (2015). Exame Nacional do Ensino Médio (ENEM): Uma análise crítica. Revista Brasileira de Ensino de Física, 37(1), 1101. doi: 10.1590/S1806-11173710001

Taber, K.S. (2017). The use of Cronbach’s Alpha when developing and reporting research instruments in science education. Research in Science Education, 1-24. doi: 10.1007/s11165-016-9602-2.

Travitzki, R. (2017). Avaliação da qualidade do Enem 2009 e 2011 com técnicas psicométricas. Estudos em Avaliação Educacional, 28(67), 256-288. doi: 10.18222/eae.v28i67.3910

Valentini, F., Gomes, C.M.A., Muniz, M., Mecca, T.P., Laros, J.A., & Andrade, J.M. (2015). Confiabilidade dos índices fatoriais da Wais-III adaptada para a população brasileira. Psicologia: Teoria e Prática, 17(2), 123-139. doi: 10.15348/1980-6906/psicologia.v17n2p123-139


Refbacks

  • There are currently no refbacks.


Copyright (c) 2018 Cristiano Mauro Assis Gomes, Hudson Fernandes Golino, Alexandre José de Souza Peres

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2015-2018. European Journal of Education Studies (ISSN 2501 - 1111) is a registered trademark of Open Access Publishing Group. All rights reserved.


This journal is a serial publication uniquely identified by an International Standard Serial Number (ISSN) serial number certificate issued by Romanian National Library (Biblioteca Nationala a Romaniei). All the research works are uniquely identified by a CrossRef DOI digital object identifier supplied by indexing and repository platforms. All authors who send their manuscripts to this journal and whose articles are published on this journal retain full copyright of their articles. All the research works published on this journal are meeting the Open Access Publishing requirements and can be freely accessed, shared, modified, distributed and used in educational, commercial and non-commercial purposes under a Creative Commons Attribution 4.0 International License (CC BY 4.0).