MEDIATING EFFECT OF ARTIFICIAL INTELLIGENCE TOOLS ON THE LINK BETWEEN PERFORMANCE-BASED ASSESSMENT TASKS DIFFICULTY AND TRAINEES’ INNOVATION SKILL DEVELOPMENT IN SELECTED TECHNICAL INSTITUTIONS, KENYA

James Ngeti Musyimi, John Mugun Boit, Anne Syomwene

Abstract


The fundamental purpose of this study was to assess the mediating effect of Artificial Intelligence (AI) technology tools on the link between Performance-Based Assessment (PBA) tasks’ difficulty and innovation skill development among Information and Communication Technology (ICT) diploma trainees in selected technical institutions in Kenya. Guided by Vygotsky’s Social Constructivism theory, the research employed a convergent parallel mixed-methods design. A sample of 346 trainees was drawn from a target population of 2,247 Module III ICT diploma trainees. Quantitative data were collected using an ICT Diploma Trainees’ Questionnaire (IDTQ), while qualitative data were obtained through Focus Group Discussions (FGDs). The questionnaire’s reliability was established using Cronbach’s alpha for internal consistency, and its face and content validity were confirmed through expert judgment. Trustworthiness of qualitative findings was enhanced through member checking, while triangulation ensured confirmability. Quantitative data were analysed using descriptive statistics and linear regression-based mediation analysis, whereas qualitative data were analysed through thematic analysis. Integration of findings was achieved using a side-by-side comparison approach. The study found that the use of AI technology tools partially mediated the relationship between PBA task difficulty and innovation skill development (β = 0.041, BootLLCI = 0.009, BootULCI = 0.082). Qualitative findings indicated that difficult PBA tasks encouraged repetition, exploration and experimentation. However, the transition of this to innovation skill development was dependent on the availability of adequate support, including but not limited to relevant AI technology tools. The study concluded that the effectiveness of innovation skill development is not merely a function of the usage of supporting AI tools but also the task difficulty itself. The study recommends that technical institutions offering ICT diploma courses should establish structured AI-enabled support applications so that trainees can effectively engage with cognitively demanding PBA tasks.

Keywords


artificial intelligence, innovation skills, tasks difficulty, performance-based assessment, ICT diploma

Full Text:

PDF

References


Albay, E. M., & Eisma, D. V. (2021). Performance task assessment supported by the design thinking process: Results from true experimental research. Social Sciences & Humanities Open, 3(1). https://doi.org/10.1016/j.ssaho.2021.100116

Avvisati F., Jacotin, G. & Vincent-Lancrin, S. (2013). Educating Higher Education Students for Innovative Economies: What International Data Tell Us. Tuning Journal for Higher Education, 1(1), 223-240. https://doi.org/10.18543/tjhe-1(1)-2013pp223-240

Beghetto, R., & Kaufman, J. (2014). Classroom contexts for creativity. High Ability Studies, 25(1), 53–69. https://doi.org/10.1080/13598139.2014.905247

Braun, V., & Clarke, V. (2012). Thematic analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol. 2. Research designs: Quantitative, qualitative, neuropsychological, and biological (pp. 57–71). American Psychological Association. https://shorturl.at/drAFS

Bruner, J. (1996). The Culture of Education. Cambridge, MA: Harvard University Press. https://doi.org/10.2307/j.ctv136c601

Creswell, J. W., & Creswell, J. D. (2018). Research design: qualitative, quantitative, and mixed methods approaches. Fifth edition. Thousand Oaks, California, SAGE Publications, Inc. Retrieved from https://url-shortener.me/GRN9

Darling-Hammond L. & Adamson F. (2010). Beyond basic skills: The role of performance assessment in achieving 21st century standards of learning. Stanford, CA: Stanford University, Stanford Center for Opportunity Policy in Education. Retrieved from https://url-shortener.me/GRNC

Dawson, P. (2017). Assessment rubrics: towards clearer and more replicable design, research and practice. Assessment & Evaluation in Higher Education, 42(3), 347-360. https://doi.org/10.1080/02602938.2015.1111294

Dziallas, M. & Blind, K. (2019). Innovation indicators throughout the innovation process: An extensive literature analysis, Technovation, 81, 3-29. Retrieved from https://doi.org/10.1016/j.technovation.2018.05.005

Espinosa T. (2015). Effective Use of Performance-based Assessments to Identify English Knowledge and Skills of EFL Students in Ecuador. Theory and Practice in Language Studies 5(12), 2441-2447. Retrieved from https://doi.org/10.17507/tpls.0512.02

Gallardo, K. (2020). Competency-based assessment and the use of performance-based evaluation rubrics in higher education: Challenges towards the next decade. Problems of Education in the 21st Century, 78(1), 61–79. https://doi.org/10.33225/pec/20.78.61

Grech, A., & Camilleri, A.F. (2020). The Digitization of TVET and Skills Systems. Geneva, Switzerland: International Labour Organization.

Green, S. L., & du Plessis, E. C. (2023). Student Perceptions of Project-Based Learning in Technical and Vocational Education and Training (TVET) Hospitality Education: A Case of a TVET College in Gauteng. Journal of Educational Studies, 22(4), 56-74. https://doi.org/10.59915/jes.2023.22.4.4

Hamilton, I. (2023). Artificial intelligence in education: Teachers’ opinions on AI in the classroom. Forbes Advisor. Retrieved on 9/5/2024, from https://shorturl.at/hRT56

Hayes, A. F. (2017). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. Guilford publications. Retrieved from https://url-shortener.me/GRNI

Hayes, A. F. (2022). Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach (Vol. 3). The Guilford Press. Retrieved from https://url-shortener.me/GRNP

Herrera, S., Cabral, R., & Murry, K. (2013). Assessment accommodations for classroom teachers of culturally and linguistically diverse students (2nd ed.). Boston, MA: Allyn & Bacon. Retrieved from https://url-shortener.me/GRNN

Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial intelligence in education: promises and implications for teaching and learning. Center for Curriculum Redesign. Retrieved from https://url-shortener.me/F27K

James, M. (2008). Assessment and Learning. Retrieved from https://bit.ly/3qEY8i2

Jenkins, D. G., & Quintana-Ascencio, P. F. (2020). A solution to minimum sample size for regressions. PLoS ONE, 15(2). https://doi.org/10.1371/journal.pone.0229345

Jepkemoi, B., Mulwa, P. K., & Mwanda, S. O. (2024). Influence of ChatGPT Affordances on Adaptive Learning Experiences among Undergraduate Religious Education Teacher Trainees at the University of Nairobi, Kenya. Canadian Journal of Educational and Social Studies, 4(1), 25-35. https://doi.org/10.53103/cjess.v4i1.206

Kithae P., Awour E., Leting N., & Gesimba P. (2014). Impact of TVET Institutions as Drivers of Innovative Skills for Sustainable Development in Kenya. The Kenya Journal of Technical and Vocational Education and Training, 2, 65-74. Retrieved from https://url-shortener.me/GRNT

Klopfer, E., Reich, J., Abelson, H., & Breazeal, C. (2024). Generative AI and K-12 Education: An MIT Perspective. Retrieved from https://mit-genai.pubpub.org/pub/4k9msp17/release/1

Komari, R. N., Aryanti, T., & Sudjani, S. (2019). Skill and performance assessment using problem-based learning in TVET. 5th UPI International Conference on Technical and Vocational Education and Training (ICTVET 2018), 514–518. https://www.atlantis-press.com/proceedings/ictvet-18/55914003

Kothari, C. R., & Garg, G. (2019). Research Methodology: Methods and Techniques (4th ed). New Delhi: New Age International Publishers. Retrieved from https://www.newagepublishers.com/samplechapter/000896.pdf

Luckin, R. (2017). Towards artificial intelligence-based assessment systems. Nature Human Behaviour, 1(3), 0028. Retrieved from https://doi.org/10.1038/s41562-016-0028

Manyukwe, C., (2023). 40% of Students Have Used AI in Their Studies-Survey. Retrieved from https://www.universityworldnews.com/post.php?story=20231128122611248 on 9/5/2024.

Martin, P., Potocnik, K, & Fras, A. (2015). Determinants of students’ innovation in Higher Education. Studies in Higher Education, 42(7). https://doi.org/10.1080/03075079.2015.1087993

Mc Tighe (2014). Designing Cornerstone Performance Tasks to Promote Meaningful Learning and Assess What Matters Most. Accessed from https://bit.ly/3dBBqC5

Mcleod, S. (2023). Lev Vygotsky's sociocultural theory of cognitive development. Simply Psychology. Retrieved 9/5/2024, from https://www.simplypsychology.org/vygotsky.html

Moindi, R., & Nyatuka, B. (2022). Competency-Based Assessment in Entrepreneurship Education in Kenya’s Tertiary Institutions. African Journal of Teacher Education, 11(1), 100-120. https://doi.org/10.21083/ajote.v11i1.7052

Musyimi, J., Orodho, J. A., & Thuo, O. (2021). Frequency of Performance-Based Assessments in Secondary School Computer Studies and Its Influence on Students’ Innovation Capacity in Kandara Sub-County, Kenya. Journal of Education and Practice, 12(30). Retrieved from https://ir-library.ku.ac.ke/items/a4fc8c56-2578-49c2-b076-f88220df2e37

Naqvi, A. (2020). Artificial intelligence for audit, forensic accounting, and valuation: A strategic perspective. John Wiley & Sons. https://doi.org/10.1002/978111960 1906

Olatunde-Aiyedun, T. G. (2024). Artificial Intelligence (AI) in Education: Integration of AI into Science Education Curriculum in Nigerian Universities. International Journal of Artificial Intelligence for Digital, 1(1). https://doi.org/10.13140/RG.2.2.31699.76320

Oranga, J. (2023). Benefits of Artificial Intelligence (ChatGPT) In Education and Learning: Is ChatGPT Helpful?. International Review of Practical Innovation, Technology and Green Energy (IRPITAGE), 3(3), 46-50. Retrieved from https://url-shortener.me/GRMZ

Orodho A.J., Nzabalirwa W., Odundo P., Waweru N.P. & Ndayambaje I., (2016). Quantitative and Qualitative research methods: A step-by-step guide to scholarly excellence. Nairobi. Kanezja publishers

Ovbiagbonhia A., Kollöffel B. & Brok P. (2019). Educating for innovation: students' perceptions of the learning environment and of their own innovation competency. Learning Environments Research, 2019(22), 387–407. Retrieved from https://bit.ly/3jxDi2I

Owan, V. J., Abang, K. B., Idika, D. O., Etta, E. O., & Bassey, B. A. (2023). Exploring the potential of artificial intelligence tools in educational measurement and assessment. EURASIA Journal of Mathematics, Science and Technology Education, 19(8). https://url-shortener.me/GRQI

Patton, M. Q. (2015). Qualitative Evaluation and Research Methods. Thousand Oaks, CA: Sage. Retrieved from https://url-shortener.me/GRQN

Republic of Kenya (2018). Competency-Based Assessment Guidelines. Nairobi. TVET Curriculum Development, Assessment and Certification Council.

Republic of Kenya (2022). TVET Knowledge and Key Highlights Report Mapping Technical and Vocational Education and Training Data in Kenya. Nairobi. Ministry of Education, Science and Technology

Salma, N., & Prastikawati, E. F. (2021). Performance-based assessment in the English learning process: Washback and barriers. Getsempena English Education Journal, 8(1), 164–176. https://doi.org/10.46244/geej.v8i1.1305

Sandu, N., & Gide, E. (2019, September). Adoption of AI-Chatbots to enhance student learning experience in higher education in India. In 2019 18th International Conference on Information Technology Based Higher Education and Training (ITHET) (pp. 1-5). IEEE. https://doi.org/10.1109/ITHET46829.2019.8937382

Stoll, L., & Schultz, S. E. (2019). How to design a performance task. Science Scope, 42(7), 40-45. https://www.jstor.org/stable/26898943

Syahril, S., Nabawi, R. A., & Safitri, D. (2021). Students’ perceptions of the project based on the potential of their region: A Project-based learning implementation. Journal of Technology and Science Education, 11(2), 295-314. Retrieved from https://www.jotse.org/index.php/jotse/article/view/1153

Syomwene, A. (2016). Vygotsky’s social development and interaction theory: Implications to the teaching of the English language curriculum in Kenya. European Journal of Education Studies 1(2). Retrieved from https://oapub.org/edu/index.php/ejes/article/view/52

Turyatemba, E. B. (2023). The Evaluation of the Implementation of Real-Life Project-Based Learning for Competence Development of TVET Trainees in Uganda (Doctoral dissertation, University of Eldoret). Retrieved from http://erepository.uoeld.ac.ke/handle/123456789/2110

Vasconcelos, M. A. R., & dos Santos, R. P. (2023). Enhancing STEM learning with ChatGPT and Bing Chat as objects to think with: A case study. EURASIA Journal of Mathematics, Science and Technology Education, 19(7). https://doi.org/10.29333/ejmste/13313

Verenikina, I. (2010). Vygotsky in Twenty-First-Century Research. Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications, 16-25. Retrieved from https://ro.uow.edu.au/ndownloader/files/50563527

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press. Retrieved from https://home.fau.edu/musgrove/web/vygotsky1978.pdf

Wachira, K., Wachira, L. N., Mwangi, E., Absaloms, H. O., & Jeon, G. (2023, September). Tertiary Education Integrity in a ChatGPT Conscious World: Preliminary Kenyan Observations. In 2023 IEEE AFRICON (pp. 1-6). IEEE. Retrieved from https://ieeexplore.ieee.org/abstract/document/10293448

Wang, T., Díaz, D. V., Brown, C., & Chen, Y. (2023, October). Exploring the Role of AI Assistants in Computer Science Education: Methods, Implications, and Instructor Perspectives. In 2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (pp. 92-102). IEEE. Retrieved from https://arxiv.org/abs/2306.03289

Wang’ang’a, W. A. (2024). Consequences of Artificial Intelligence on Teaching and Learning in Higher Education in Kenya: Literature Review. East African Journal of Education Studies, 7(1), 202-215. Retrieved from https://journals.eanso.org/index.php/eajes/article/view/1718

Yonda Consulting (2023). 2023 Global Student Survey. Retrieved from yondaconsulting.com on 8/3/2024.




DOI: http://dx.doi.org/10.46827/ejes.v13i3.6578

Refbacks

  • There are currently no refbacks.


Copyright (c) 2026 James Ngeti Musyimi, John Mugun Boit, Anne Syomwene

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2015-2026. European Journal of Education Studies (ISSN 2501 - 1111) is a registered trademark of Open Access Publishing Group. All rights reserved.

This journal is a serial publication uniquely identified by an International Standard Serial Number (ISSN) serial number certificate issued by Romanian National Library (Biblioteca Nationala a Romaniei). All the research works are uniquely identified by a CrossRef DOI digital object identifier supplied by indexing and repository platforms. All authors who send their manuscripts to this journal and whose articles are published on this journal retain full copyright of their articles. All the research works published on this journal are meeting the Open Access Publishing requirements and can be freely accessed, shared, modified, distributed and used in educational, commercial and non-commercial purposes under a Creative Commons Attribution 4.0 International License (CC BY 4.0).