Article Preview
TopIntroduction
In the field of higher education, assessments offer essential information in terms of establishing whether institutions and faculty are effective, and whether students are ready for employment (O’Hagan & Wigglesworth, 2015). The way in which such assessments have been performed has not changed noticeably over past decades, despite the fact that research has shown that current grading systems do not work satisfactorily for either students or examiners (Nilson, 2016).
An undergraduate thesis is the final piece of work undertaken by a student in the last term of a three-year bachelor’s degree. The assessment of undergraduate theses is often based on certain criteria that aim to guide examiners and offer students transparency in the assessment process. Due to the complexity and vagueness of the criteria (Golding et al., 2014; Hand & Clewes, 2000), and examiners’ interpretations of these (Bloxham et al., 2016; O’Donovan et al., 2001), it is more common for examiners to use a holistic approach when assessing theses (Bloxham et al., 2016). There is often no consensus among examiners on the criteria, meaning that assessments may be made on an arbitrary basis. Different interpretations of criteria can create variation in grades, which makes it challenging to connect assessments to explicit criteria (Bloxham et al., 2016; Sadler, 2009; Webster et al., 2000).
Research into criterion-referenced assessment has established the importance of explicit criteria (Ecclestone, 2001; Hornby, 2003; Woolf, 2004), mutual understanding between students and examiners (Bloxham et al., 2011; Ecclestone, 2001; O’Donovan, Price, & Rust, 2004,), the co-creation of criteria (Hughes & Cappa, 2007; Pathirage et al., 2007; Rust et al., 2005), and the development of criteria as a continuous process for examiners and students (Bloxham et al., 2011; Saunders & Davis, 1998).
There is also a lack of research into how digital assessment tools could be designed for higher education to aid examiners in the co-creation of explicit criteria for the assessment of undergraduate theses and the creation of a mutual understanding between students and examiners in the assessment process. In particular, in the area of design education, there is an additional layer of complexity relating to how the level of creativity is assessed in theses. This means that examiners might have to trust to their own experience and carry out holistic thesis assessments, as some aspects are not explicitly articulated in the relevant criteria.
This article aims to provide a theoretical anchoring and underpinning for a design concept for a digital assessment tool, referred to here as grading by automated criteria evaluation (GRACE). The aim of GRACE is to offer a co-creative assessment tool to facilitate the creation of criteria by students and examiners, and to support examiners in reaching a consensus on how these criteria are applied in practice. This research focuses on bachelor’s theses written in the design field, and the research process is guided by the following question: How can GRACE be conceptualized and theoretically anchored to aid examiners in co-creating assessment criteria for undergraduate theses while concurrently supporting students’ learning processes? The concept-driven design research approach put forward by Stolterman and Wiberg (2010) will be used to address this research question. The scope of this article is limited to a focus on examiners’ views and challenges when evaluating and grading undergraduate theses.