The Evaluation of Competency-Based Diagnosis System and Curriculum Improvement of Information Management

Introduction

In order to improve the learning process in higher education, most of the universities in Taiwan conduct teaching evaluation and alumni survey and then hopefully to form a competence-based diagnosis system to facilitate the process. Teaching evaluation is implemented by asking students to fill a questionnaire to give their feedback on the course to the instructor so that a revised pedagogy can be adopted in the next semester. Thus, teachers get the opportunity to know students' opinions to modify their teaching process (Johari et al., 2016; Harshman & Yezierski, 2017; Xiong & Niu 2018).

Besides teaching evaluation, feedback from recent graduates and alumni about their experience at workplaces is also very important (de St Jorre & Oliver, 2018). Repeated alumni surveys facilitate in detecting students’ emerging needs and problems and help management in introducing necessary changes in the curriculum (Lüer & Aebi 2017; Kumm & Laverentz, 2017; Ratri et al, 2019).

Several items in the alumni survey need the attention of management in higher education institutions for the improvement of students’ competencies. A competency is a combination of attitude, behavior, knowledge, and skill that contribute to an individual’s needs and success (Boury et al., 2013; Mah, 2017; McCall, 1998). Many reports have shown that competency-based teaching and learning can improve the quality of curriculum and education (Gunawardena, 2014; Montoya, 2013; Rizalyn & Shuichi, 2003; Steel, 2018). In order to examine the authentic and self‐steering study programs, competency-based diagnosis is increasingly being used for designing a course and teaching method, especially in vocational education (CAHME, 2015; Wesselink et al., 2010). Also, the evaluation of students’ competencies at workplaces is important to ascertain that graduating students have developed necessary professional competencies to carry out the tasks at their workplaces (Bossers et al., 2002, 2007; Miller et al., 2001).

However, the general teaching evaluation and competency diagnosis are independent of each other. Due to this, there is insufficient evidence to improve curriculum planning and instruction activities. Also, concerning the competency evaluation, the number of samples and the written reflections are limited and inefficient (Davies et al., 2013; Elliott, 2017; Irvine et al, 2008). This creates room for more research, and through a cross-domain information system, the survey results can be more efficient and more complete. This is the first objective of the study. Several researchers reported that the planning and application of evaluation systems by students at the campus, and alumni after graduation is important (Elstad et al., 2015; Iezzi, 2005; Liu et al., 2016; Onwuegbuzie et al., 2007; Xu, 2012). Therefore, systematic competency diagnosis and teaching evaluation are necessary.

Keeping this need in mind, since 2014, the Ministry of Education (MOE), Taiwan had set up an alumni tracking mechanism for university graduates on career paths, career situations, their needs, and perceived gaps between their training and current work, competency acquisition, etc. The data is collected from students’ one-year after their graduation. Students provide feedback to universities in order to improve students’ career training and curriculums. With this objective, each year, the case university carries out alumni surveys to collect alumni’ work feedback and then shares it with concerned departments to let them evaluate curriculum maps.

In addition to the alumni survey, the MOE has set up a University Career and Competency Assessment Network (UCAN) system to evaluate the effect of competency training before the students join their jobs. The goal of UCAN is to help students understand the direction of their career development and strengthen their competency. Through career interest exploration and competency diagnosis, students can increase their understanding of a workplace, plan self-cultivation goals, realize competency gaps, and improve their competitiveness. Therefore, the second objective of the study is to link the UCAN and graduate flow surveys to disclose the competency gaps through the analysis reports.