WEB 2.0 AUTOMATED ESSAY SCORING APPLICATION AND HUMAN ESL ESSAY ASSESSMENT: A COMPARISON STUDY

Mohammad Radzi Manap, Nor Fazlin Ramli, Aini Akmar Mohd Kassim

Abstract


Educators from all levels of education have comfortably embraced ICT for decades. More and more applications are being developed which has direct and indirect advantages on learning and teaching. What is more interesting is the fact that more free applications are now being made available to be used by the public. In this paper, the focus is on the comparison between a free Automated Essay Scoring (AES) web-based application called ‘PaperRater.com’ and human assessment by English instructors from a public university in Malaysia. Ten selected human assessed essays were assessed by individual lecturer (IL) and also a group of lecturers (GL). Those essays were then fed into ‘PaperRater.com’ and a comparison was made in terms of total scores generated by the application and the ones by the instructors. A descriptive statistical analysis was carried out to compare the scores. As a result, assessment for both types of essay recorded a difference of between 0.3 to 38.7 marks where bigger disparity was recorded between human assessed and computer based (AES) assessment. Overall, however, the strength of the relationship between GL and PR, based on the Pearson’s product moment correlation coefficient, was recorded at 0.678407 which means that there is a moderate positive linear relationship.

 

Article visualizations:

Hit counter

DOI

Keywords


Computer Assisted Language Learning, e-learning, web-based 2.0 application, Automated Essay Scoring

Full Text:

PDF

References


Attali, Y. & Burstein, J. (2006). Automated Essay Scoring With e-rater® V.2. Journal of Technology, Learning and Assessment, 4(3). Available from http://www.jtla.org

Creswell, J. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage.

Dikli, S. (2006). Automated Essay Scoring. Turkish Online of Distance Education. Vol. 7 (1). pp 49-62.

Grimes, D. & Warschauer, M. (2010). Utility in a Fallible Tool: A Multi-Site Case Study of Automated Writing Evaluation. Journal of Technology, Learning, and Assessment,8(6). Retrieved [22.5.2016] from http://www.jtla.org.

Hinkle D.E, Wiersma W, Jurs. (2003). 'Applied Statistics for the Behavioral Sciences'.

Kheradmand, N. and Sayadiyan, S. (2016). Comparative Investigation of the Effects of Immediate and Delayed Error Correction on the Achievement of Male and Female Iranian EFL Learners’ Writing Skill. International Journal of Social Science and Education. Vol. 6 (1). Retrieved from www.ijsse.com/sites/default/files/issues/2016/v6i1/paper-04.pdf [21 July 2015]

Moore, D. S., Notz, W. I., & Flinger, M. A. (2013). The basic practice of statistics (6th ed.). New York, NY: W. H. Freeman and Company.

Mo Zhang. (2013). Contrasting Automated and Human Scoring of Essays. R&D Connections, 21. Retrieved [11.6.16] from 2013 https://www.ets.org/Media/Research/pdf/RD_Connections_21.pdf

Nafizah Hamidun, Shafiq Hizwari Md Hashim and Nur Farhinaa Othman. (2012). Enhancing Students’ Motivation by Providing Feedback on Writing: The Case of International Students from Thailand. International Journal of Social Science and Humanity. Vol. 2 (6). Retrieved [21 July 2016] from www.ijssh.org/papers/179-A10062.pdf

Neda Kheradmand and Sima Sayadiyan. (2016). Comparative Investigation of the Effects and Delayed Error Correction on the Achievement of Male and Female Iranian EFL Learners’ Writing Skill. International Journal Soc. Sci. & Education. Vol. 6 (1). Retrieved [21 July 2016] from www.ijsse.com/sites/default/files/issues/2016/v6i1/paper-04.pdf

Ng Sing Yii, Bong Chih How, Lee Nung Kion and Hong Kian Sam. (2016). Automated Essay Scoring Feedback (AESF): An Innovative Writing Solution to the Malaysian University English Test (MUET). International Journal on E-Learning and Higher Education, 4. pp. 130-143.

Qiu Yubing. (2016). Pigai Smart Essay Scoring System and Its Implications for Teaching English Writing. Journal of Applied Science and Engineering Innovation, Vol. 3 (6). pp. 217-219. Retrieved [2 February 2018] from http://www.jasei.org/PDF/3-6/3-217-219.pdf

Saadiyah Darus, Siti Hanim Stapa, Supyan Hussin & Koo Yew Lie. (2000). A Survey of Computer–based Essay Marking (CBEM) Systems. International Conference “Education & ICT in the New Millenium’. pp 519-528. Retrieved [14.3.2019] from https://www.academia.edu/6977567/A_survey_of_computer_based_essay_marking_CBEM_system

Saadiyah Darus, Siti Hamin Stapa & Supyan Hussin. (2003). Experimenting A Computer-Based Essay Marking System At Universiti Kebangsaan Malaysia. Jurnal Teknologi, 39(E) Dis. 2003: 1–18. Retrieved [23.5.2016] from http://www.academia.edu/263332/Experimenting_a_Computer-Based_Essay_Marking_System_at_Universiti_Kebangsaan_Malaysia

Shermis, M.D., Burstein, J., Higgins, D., and Zechner, K. (2010). Automated essay scoring: Writing assessment and instruction. In E. Baker, B. McGaw, and N.S. Petersen (Eds.), International encyclopedia of education (3rd ed., pp 75-80). Oxford, England: Elsevier.

Somaye Toranj and Dariush Nejad Ansari. (2012). Automated Versus Human Essay Scoring: A Comparative Study. Theory and Practice in Language Studies, Vol. 2, No. 4, pp. 719-725, April 2012

Uzun, K. (2018). Home-Grown Automated Essay Scoring in the Literature Classroom: A Solution for Managing the Crowd? Contemporary Educational Technology, 9(4). pp.423-436. Retrieved [15.3.19] from http://doi.org/10.30935/cet.471024

Valenti, S., Neri, F., and Cucchiarelli, A. (2003). An Overview of Current Research on Automated Essay Grading. Journal of Information Technology Education. 2. pp 319-330.

Wang, J. & Brown, M. S. (2007). Automated Essay Scoring Versus Human Scoring: A Comparative Study. Journal of Technology, Learning, and Assessment, 6(2). Retrieved [25.5.2016] from http://www.jtla.org

Warschauer, M., & Grimes, D. (2008). Automated writing assessment in the classroom. Pedagogies: An International Journal, 3, pp 22–36. Retrieved [3 February 2018] from http://education.uci.edu/uploads/7/2/7/6/72769947/awe-pedagogies.pdf.




DOI: http://dx.doi.org/10.46827/ejel.v0i0.2650

Refbacks

  • There are currently no refbacks.


Copyright © 2015 - 2023. European Journal of English Language Teaching (ISSN 2501-7136) is a registered trademark of Open Access Publishing Group. All rights reserved.

This journal is a serial publication uniquely identified by an International Standard Serial Number (ISSN) serial number certificate issued by Romanian National Library (Biblioteca Nationala a Romaniei). All the research works are uniquely identified by a CrossRef DOI digital object identifier supplied by indexing and repository platforms.

All the research works published on this journal are meeting the Open Access Publishing requirements and can be freely accessed, shared, modified, distributed and used in educational, commercial and non-commercial purposes under a Creative Commons Attribution 4.0 International License (CC BY 4.0).