THE IMPLEMENTATION OF EPORTFOLIOS IN AN ONLINE GRADUATE PROGRAM

A myriad of research has documented that an ePortfolio provides a comprehensive platform for learners to record their mastery of practical skillsets, create meaning from their learning, and reflect upon years of experiences. An ePortfolio also serves as a powerful digital resume and useful marketing vehicle to attract prospective employers. This case study examined the impacts of implementing ePortfolios in a capstone course of an online master’s program in health care administration. Students’ perceptions of this powerful pedagogical tool and the process of creating a Portfolio were explored. The findings supported previous research on the implementation of ePortfolios in various programs. Students enjoyed the ePortfolio process, critically evaluated their academic work and accomplishments, and valued peer-reviewed feedback offered by their peers. There were challenges observed in this study. While some challenges were unique to the study population, others were also disclosed in previous empirical studies. Recommendations were made for better integration of ePortfolios into the curricula of a program. Limitations and future research were also presented in this paper.


Introduction
We are living in a digital world. The advancement in technologies stimulated the use of electronic Portfolios, or ePortfolios, in academic settings and enabled students to move beyond the binder and paper portfolios to showcase their skills, knowledge, and competencies in digital forms (Farrell, 2020;Farrelly & Kaplin, 2019). Over the past decade, there has been a trend in higher education towards an increase in using ePortfolios globally (Hume & Hamilton, 2019;Madden et al., 2019;Roberts & Kirk, 2019). This trend is expected to continue to expand. In particular, the Association of American Colleges and Universities (AAC&U) in the United States added ePortfolios to its High-Impact Practices (HIPs) list in 2016 (Kuh, 2017;Watson et al., 2016). HIPs were established i Correspondence: email hwangji.lu@uagc.edu through a proliferation of research and data analysis on the National Survey of Student Engagement about the benefits of HIPs to student learning (Kuh et al., 2017). In response to the AAC&U's endorsement, many colleges and universities are dedicated to considerable resources developing new curriculum and integrating ePortfolios to promote skills of integrative lifelong learning (Madden et al., 2019). As a matter of fact, more than 50% of higher education institutions in the United States have utilized some form of this powerful pedagogical practice (Fallowfield, 2019;Freeman, 2020).
In this article, an introduction provides an overview concerning the implementation of ePortfolios from a global perspective. The remainder of this paper proceeds as follows. Section two surveys the literature to define an ePortfolio, describe its benefits, and develop the research questions. Section three articulates the context of the study, research design, participants, data collection, and data analyses. It is followed by the findings and discussions from this study organized in Section four. Section five depicts pedagogical implications and recommendations on implementing ePortfolios into a program to meet multiple demands. Section six highlights the research limitations and future research. The final section ends this paper by concluding this current research study.

Literature Review
Even though the introduction of portfolios into higher education can be traced back to the early 1970s (Farrell, 2020), the literature review will mainly focus on electronic portfolios and provide context for the use of ePortfolios in higher education. It first describes the definition of ePortfolios. It also details four benefits of ePortfolios. Finally, two research questions are raised.

What is an ePortfolio?
An ePortfolio can be defined as an assortment of digitalized arrangements about a student's learning experiences, progress, and academic achievements during their learning journey (Andrade & Zeigner, 2021;Farrelly & Kaplin, 2019). In the ePortfolio platform, students create, compile, and store artifacts that exemplify their learning, accompanied by reflections on what and how they have learned (Andrade & Zeigner, 2021;Bair et al., 2019). Elaborating on this, Andrade and Zeigner (2021) pointed out that ePortfolios offer a personal learning space where students record and reflect on their strengths and weakness, identify areas for improvement, share the journal of continued growth. An ePortfolio can be considered as an exemplary tool for capturing learners' behaviors, knowledge, and skills in digital formats (Bair et al. 2019;Madden et al., 2019). During the process of creating the ePortfolio, students should document all learning that occurs inside and outside of the classroom (Rhodes, 2019). An artifact, or tangible work product, within an ePortfolio is a piece of evidence for a particular competency (Bair et al., 2019). The artifacts can be collaborative assignments, community engagements, professional achievements, reflections on both formal and informal learning experiences, research activities, and research papers (Farrelly & Kaplin, 2019;Lu, 2021). ePortfolios may also include extra-curricular activities, for example internships, community services, and involvements in student organizations or professional organizations that support the achievements of learning outcomes (Andrade & Zeigner, 2021). Such evidence could be electronic files, texts, images, multimedia, blog entries, and hyperlinks.

Benefits of ePortfolios
In recent years, research studies around the world have evidenced a link between the use of ePortfolios and higher retention or completion rates (Collins & Crawley, 2016;Madden et al., 2019). ePortfolios have emerged as a valid and reliable tool for helping the student learn, enabling the student to reflect deeply upon, and assessing student development over a duration of time. Furthermore, ePortfolios can be used as a powerful marketing tool to display evidence of needed knowledge and skills for employment.

Learning
ePortfolios help students articulate their educational experience at the educational institution throughout their academic learning journey visibly and audibly (Madden et al., 2019). An ePortfolio is not a random collection of loosely related artifacts. Instead, an effective ePortfolio pedagogy creates supportive scaffolding for learners to incorporate multiple artifacts that are connected and contribute to their learning (Freeman, 2020). The completion of an ePortfolio helps develop students' adaptability, creativity, and flexibility as the competencies are required for the contemporary workplace in the 21st century (Watson et al., 2016).

Reflection
Moreover, an ePortfolio, as a powerful pedagogical approach, enables students to reflect meaningfully and deepen their learning when making accomplishments visible to their peers and faculty as well as external stakeholders such as employers (Kuh et al., 2017). Creating a comprehensive and meaningful ePortfolio demands students to reflect upon what they have learned (Freeman, 2020). Reflection is needed for the processing learning experience and producing the meaning of learning (Watson et. al., 2016); it can also uncover how we learn and what we already know or do not know (Freeman, 2020). However, this type of reflection is grounded in metacognition, which is the ability to think about thinking (Reynolds & Patton, 2014). Metacognitive learning occurs when students critically analyze, compare, and adjust their actions to improve their learning experiences and the goals they have achieved (de Jager, 2019). As Rhodes (2019) and de Jager (2019) argued, the use of ePortfolio is a vivid example of a meta high-impact practice because students process their thinking through their learning from a specific course or the entire program of study; ePortfolio purposefully makes connections among different contents and helps students to think critically on how learning has prepared them for the future.

Assessment
An ePortfolio can measure learning in more authentic ways. It is a reliable and valid tool for assessing student's learning, progression, and change over their academic learning journey (Andrade & Zeigner, 2021;Madden et al., 2019). It is often used for assessment, offering insights into growth in learning, application of learning, and deep learning. Andrade and Zeigner (2021) asserted that ePortfolios used for assessment could also be accompanied by peer reviews, where students provide their constructive feedback to each other and develop crucial awareness of their own work and those of others. The peer feedback experience enables students to learn from their peers, see the opportunities for professional development, and make connections between learning experiences and professional careers (Robles, 2012). It also helps students develop constructive feedback skills, which are essential communication skills in today's workplace.

Employability
An ePortfolio is deemed as a compelling digital resume and an effective marketing tool to invite prospective employers' and other stakeholders' comments (Moore, 2019). Students can utilize their ePortfolios beyond the learning process when they showcase career-related skills for potential employers (Blair, 2019). Employers highly value these cross-cutting skills such as communication, critical thinking, teamwork, which they believe are critical to professional success, but find that college graduates lack preparation in this area. Building ePortfolios to connect students' learning experiences, critical skills, and career goals through creating an online professional presence will address employers' concerns and set students apart from their competitors (Cordie et al., 2019). Additionally, many employers or professional associations require individuals to provide a record of evidence of up-to-date knowledge in their fields. ePortfolios strengthen employability when students continue to keep track of their professional development after graduation (Andrade & Zeigner, 2021).

Research Questions
Although ePortfolios have been widely used in various disciplines, literature has not documented the utilization of ePortfolio in the online master's program of health care administration. There is a need to evaluate the application of ePortfolios in this field of study. Thus, the following two research questions were raised: Research question 1: Do ePortfolio processes help students critically assess their academic work and accomplishments?
Research question 2: Is the peer-reviewed feedback about ePortfolio helpful to students?

The Context of The Study
This study was conducted in a university located in the western region of the United States. The university offers fully online asynchronous associate's, bachelor's, master's, and doctoral programs to adult learners. The ePortfolio pedagogic approach was incorporated into the curriculum of the capstone course, which is the last course in the Master of Health Care Administration (MAHCA) program, to highlight students' acquisition of core competencies. This pedagogic approach also provides a vehicle for students to use their ePortfolio as an addition to their resumes to assist in job placement or role progression in health care administration positions. Thus, the objectives of this study were to investigate the impacts of the ePortfolio curricular processes from the perspectives of students and instructors in the capstone course and offer recommendations for further improvement of the ePortfolio curricular processes in the MAHCA program.
In this revised capstone course, students completed a competency-based ePortfolio. During the first week of the class, students started building their ePortfolio by articulating who they are, including the information concerning their leadership style, value, personal mission, vision statement, and career goals, in the introduction section of their ePortfolio. In the second week, students were required to upload their previous schoolwork as the evidence of their competencies in health care administration according to the American College of Healthcare Executives (ACHE) Healthcare Executive Competency model using five competency domains: communication and relationship management, leadership, professionalism, knowledge of the health care environment, and business skills and knowledge (The International Hospital Federation, 2015). For displaying their skills, students could either upload artifacts as evidence or a summary of their acquired skills in a Word document. In the following week, students continued to upload artifacts for their education, training, certifications, badges, accomplishments, and community involvement (if applicable). In week four, students revised their resume by using the feedback provided from the Jobcan tool and then uploaded their revised resume to the ePortfolio platform. Students finalized their ePortfolio by making necessary changes and uploading their capstone project to the ePortfolio by the last day of week five. In week six, students posted the links to their ePortfolios in the ePortfolio Peer Review discussion board. Based upon the requirements, each student reviewed at least two peers' ePortfolios and applied the Sandwich Technique by composing positive feedback, formulating constructive criticism to include areas and specific suggestions for improvement, and supporting their peers with a positive ending note. The dual purposes of the peer review process were to ensure sufficient feedback from a variety of perspectives so students can improve upon their ePortfolio and to engage students in effective peer review processes so students will be able to exercise this skill in their professional practice.

Research Design
A mixed-methods research design was employed. The population of this study included MAHCA students who had enrolled in the revised capstone course during the first year of implementation as well as the instructors had taught this course during the same period of timeframe. Multiple data sources were utilized to examine the effectiveness and impact of the ePortfolio in this revised course and the students' learning outcomes. Some data sources included qualitative analyses in the discussion boards and the evaluation of the artifacts in the classrooms.
Additionally, two in-house questionnaires were created. These two quantitative Likert scale surveys, along with opened ended questions, were designed to gather students' and instructors' experiences and opinions about the ePortfolio processes in the capstone course. Towards the end of each class, students who were enrolled in the capstone course of MAHCA program and instructors who had taught this course were invited to participate in the surveys. Both students and instructors were provided an opted-out option if they wished not to participate. To address the privacy concern, two in-house questionnaires were anonymously managed for those who wished to participate in helping the MAHCA program improve the curricula. A total of 73 students and five instructors responded to the surveys.

Results
The first research question examined students' perceptions about how ePortfolio processes helped them critically assess their academic work and accomplishments. Based upon the data collected, over three-fourths (78.08%) of surveyed students reported that e-Portfolio helped them critically assess their academic work and accomplishments, as shown in Table 1. Students' verbal comments were captured and analyzed. The examples of positive comments are as follows. "The e-folio was an amazing tool that helped me organize my work and realize all I have accomplished." "It's an organization way to celebrate individual accomplishments and prepare for future career." "I really enjoyed the e-portfolio because it has made me reflect on my accomplishments." "The E-portfolio gave me an opportunity to organize my skills and accomplishments and really think about my plans post degree." "I really enjoyed showcasing my talents and work experience. I believe this better equipped me with a great format that will be applying to employers." "As stated, the ePortfolio streamlined my resume, it was a real time 'balanced scorecard' to coin a phrase from our courses." "The activities associated with

my e-Portfolio helped me revise my resume and put forth a product that could help me with my career advancement." "I enjoyed this process as it helped me see where I am at in my career growth and what areas I can improve upon." "The e-Portfolio assignment required a deeper thinking of skills and matching them to competencies."
On the other hand, some students had negative experiences in the ePortfolio processes. "I felt that the e-Portfolio was overwhelming and a bit cluttered with information." "e-Portfolio was a little difficult to navigate sometimes. More resume work would be helpful a head of time." "Not a fan of the portfolio and seems like a waste of time. I'll never use it." "I had never done an e-Portfolio before, and this was a real challenge. I felt frustrated at times, but while I was preparing my e-Portfolio, I have used the section where it says chat with a professional--I utilized that quite a bit and it really helped me." "I found the platform a bit difficult to master and is a new idea for myself. I do like the idea of it but need more experience building this type of platform." "It seemed like a lot of busy work. I think there are other ways of summarizing accomplishments." "I will likely delete my ePortfolio account as I primarily use LinkedIn to display my professional experience. I would not likely show a potential employer my schoolwork." The second research question explored the helpfulness of peer-reviewed feedback in ePortfolio. When asked if the e-Portfolio Peer Review helped me validate my competencies in healthcare administration, 71.83% of respondents agreed with this statement as displaced in Table 2. Despite the fact that students did not rate this activity higher than their overall view about e-Portfolio processes, the verbal comments made by students were overwhelmingly positive. Only one student stated that "I did not think their feedback was entirely helpful." The following are some examples of verbal comments. "This peer review allowed me the opportunity to get constructive criticism from my classmates and the professor in order to fully showcase the skills and experiences I have obtained. It also allowed me to offer constructive criticism to my classmates that would help to enhance their ePortfolio." "I found that the e-Portfolio peer review allowed me see the large variation in myself and my peers." "We were taught to give constructive but honest criticism. Once the mindset of taking the criticism as a honest helping tool, one can use it to fine tune." "It's always nice to hear negative and positive comments but yet at the same time I have learned to take criticism constructively." "The feedback was beneficial and I saw how others presented themselves. It gave me confidence in my level of submitted work." "We don't always see where improvements need to be made when we are analyzing our own work. It was nice to have our own peers give feedback on areas that needed to be improved and also compare to their portfolios." "I was able to understand the competency more." "The e-Portfolio Peer Review allowed me to see what other healthcare professionals have done in their careers, and how that information could help me grow in my own healthcare career." "Looking at others e-portfolio was not only interesting but helped to see the bigger picture of what employees look for."

Discussion
The objectives of this study were to determine the impacts of ePortfolios on student learning experience in the capstone course of an online program in health care administration and make recommendations for improvement about the ePortfolio curricular processes in the MAHCA program. The majority of study participants reported that e-Portfolio helped them critically assess their academic work and accomplishments. This finding is in accordance with de Jager's study (2019), in which most of the participants engaged in reflective metacognitive learning experiences while developing an ePortfolio. Existing literature identifies that the peer feedback on ePortfolio allows for peer-to-peer learning, professional development, and connections between learning experiences and professional career (Andrade & Zeigner, 2021;Robles, 2012). The current research study also finds that ePortfolio peer review helped students validate their competencies in health care administration, comprehend their strengths and weaknesses for future professional development, and perceive the link between their learning experiences and the employer's expectations.
In their research, Cordie et al. (2019), de Jager (2019), and Fallowfield et al. (2019) expressed concern about the fact that students did not possess sufficient technical skills and struggled with the ePortfolio technical development. The findings of this study confirm those of the researchers' discussions. Another study conducted by O'Loghlen (2015) in Ireland to examine the effectiveness of ePortfolios as a learning tool revealed that 65% of the participants felt that the process of ePortfolio was too time-consuming; the study suggested that students need training and support in the creation of ePortfolios. The qualitative comments collected from this research study substantiate the finding, although the frequency of such a comment is not as high as O'Loghlen's study (2015). Nevertheless, the results derived from the current research study espouse that enhanced support and guidance are needed to reduce students' frustration and make the ePortfolio process a more enjoyable learning experience.
Some challenges in the implementation phase are discovered from instructors' observations collected by the in-house instructor survey. This is the first MAHCA course asking students to create their ePortfolio. MAHCA students are mid-aged adult learners. It is hard for some students to be out of their comfort zone to learn the new technological platform in the last course of the program. Moreover, most of our students are frontline health care professionals. Since the beginning of the COVID-19 pandemic, they have dealt with more challenges than other professionals, which also affects their ability to learn new technology and follow the requirements to complete their assignments fully. Along the line, some students might lose their family members to COVID-19, which is also a crucial distracting factor. As one instructor observed, some students might have anxiety or other mental challenges. Those students would have a hard time focusing and learning on top of their job demands and family obligations. Some students tend to panic in week one because they are required to learn how to navigate the ePortfolio platform and understand the requirements to complete their ePortfolio. They are also more than likely to complain by saying "terrified," "fret out," or "traumatized." Finally, certain students were not tech-savvy, which is also revealed in previous empirical research (Cordie et al., 2019;de Jager, 2019;Fallowfield et al., 2019, O'Loghlen,2015.
In week two, some students are still confused and struggling with uploading proper evidence to showcase their skills and competencies in health care administration. In week three, while a small portion of students is still perplexed, the majority of students follow the instructions to upload more evidence concerning their education, training, certificates, achievements, and community services. In week four, students enjoy working on their resume and uploading their resume to the ePortfolio. In week five, students are delighted to conclude their ePortfolio.

Recommendations
Based on the data collected from the student survey, instructor survey, instructors' evaluations on artifacts in the classroom, it is recommended that the MAHCA program enhance the support of building ePortfolios in the capstone course and include additional courses dedicated to utilizing ePortfolio in more meaningful ways. The recommendations are further translated into five actionable items. First, to reduce students' frustrations, instructors who teach this capstone course should schedule a Zoom meeting in the first week of the class, explaining the learning activities due each week and addressing students' concerns. Second, additional guidelines and resources to support the use of ePortfolios are provided and embedded under the section of Instructor Guidance in the course shell. Third, teaching strategies and best practices are shared among instructors to facilitate this course smoothly. Fourth, the instructions for ePortfolio assignments are further refined to reduce students' confusion. Fifth, the MAHCA program plans to add ePortfolio learning activities to two other MAHCA courses when the revisions are due. Once the other two courses are modified, students will have exposed and practiced ePortfolio assignments before enrolling in the capstone course. It will help students realize the impacts of ePortfolios on their professional identity over a longer period of time. Therefore, it is also suggested that educators and institutions consider the integration of this pedagogical approach to be thoughtfully woven throughout the curricula in a program, capturing the entire students' learning experience while pursuing the degree.

Limitations and Future Research
Similar to other studies, this research study has two main limitations. First, a sample size from one program was utilized. As such, there is a limitation in the ability to generalize beyond this study. The second limitation is that students' perceptions and experiences accounted for a large portion of data sources. COVID-19 pandemic has impacted our students, who are frontline healthcare professionals significantly. Although the university allows students to take academic leave during hardship, those students who remain in the program might not concentrate on the schoolwork fully. Their experiences and perceptions about the ePortfolio processes would have been affected. This limits our ability to draw the conclusion about the effectiveness of ePortfolio on student learning outcomes. Therefore, it is suggested that future studies pertaining to students' and instructors' perceptions and experiences in ePortfolios allow for longitudinal data collection or data collection from various programs to ensure larger sample sizes.

Conclusion
An ePortfolio is a collection of electronic documentations highlighting students' academic progress, achievements, competencies, experiences, and skills over time. In addition, ePortfolios function as an expanded curriculum vitae to present the best work of a student to the prospective employer or other stakeholders. This paper presents a case study of integrating ePortfolios into an online master's program in health care administration. Students indicate positive comments about their critical assessments on their academic work and professional accomplishments while developing their ePortfolios. Students also appreciate the peer review opportunity to learn from others and practice providing feedback. Even though compiling ePortfolios is time-consuming due to a lack of technical skills or unfamiliarity with the platform, many students point out that they enjoy the process by reflecting on what they learned in the program and sharing their learning experiences with their peers via their ePortfolios.
Effective ePortfolio requires intense revision and reflection by the students. The one-year journey has taught us many important lessons about the successful implementation of a program-wide ePortfolio plan. Unquestionably, we focused too intensely on the ePortfolio as a product and did not allow enough time for students to embrace the process. The best practice of implementing ePortfolios into the curriculum should permit curricular activities to scaffold student learning experience over time to develop their professional identities. An effective ePortfolio is valuable when students build their ePortfolio and construct their professional identity in accordance with the consideration of employers' points of view, which shapes students' reflections to communicate their readiness for a professional career following the completion of the academic program. The findings from this current research study support the debate that ePortfolios have provided an authentic learning experience and are seen as relevant and significant to professional identity development for students. The findings and recommendations from this study also build the best practices for health care The International Hospital Federation, 2015. Leadership competencies for healthcare services managers. https://www.ache.org/-/media/ache/aboutache/leadership_competencies_healthcare_services_managers.pdf. Accessed 29 September 2021 Watson C E, Kuh G D, Rhodes T, Penny Light T, Chen H L, 2016. Editorial: ePortfolios-The eleventh high impact practice. International Journal of ePortfolio, 6(2): 65-69.