Abstract
Systematic evaluation of distance-education programs is necessary for program improvement (formative) and accountability (summative). When evaluating distanceeducation programs, it is advisable to consult the best practices literature for determining the criteria and standards for quality programming in order to judge the merit and worth of the program under consideration. The most frequently cited reference for best practices comes from the Western Cooperative for Educational Telecommunications (1997) (http://www.wiche.edu/ telecom/), otherwise known as WICHE. Pennsylvania State University (PSU) (1998) published a document for designing distance-education programs that incorporate the literature base of best practices. WICHE and PSU offer principles for best practices in distance education in regard to curriculum and instruction, institutional context and commitment, and evaluation and assessment. General concerns with each category should be considered in developing an instrument for measuring quality in distance-education programs (Law, Hawkes, & Murphy, 2002).
TopIntroduction And Background: Using Best Practices As A Standard For Program Evaluation
Systematic evaluation of distance-education programs is necessary for program improvement (formative) and accountability (summative). When evaluating distance-education programs, it is advisable to consult the best practices literature for determining the criteria and standards for quality programming in order to judge the merit and worth of the program under consideration.
The most frequently cited reference for best practices comes from the Western Cooperative for Educational Telecommunications (1997) (http://www.wiche.edu/telecom/), otherwise known as WICHE. Pennsylvania State University (PSU) (1998) published a document for designing distance-education programs that incorporate the literature base of best practices. WICHE and PSU offer principles for best practices in distance education in regard to curriculum and instruction, institutional context and commitment, and evaluation and assessment. General concerns with each category should be considered in developing an instrument for measuring quality in distance-education programs (Law, Hawkes, & Murphy, 2002). These concerns are outlined in Table 1.
Table 1. Considerations for measuring quality in a distance-education program
| Consistency of program with institutional mission | Evaluators should look for evidence that offering distance education is in line with the institutional mission and is well supported with adequate budgets and support staff. |
| Provisions for program oversight and accountability | Academic and technical oversight should be obvious to evaluators. |
| Provision of student support | Evaluators should examine Web sites and other media to ensure that students have access to all the required contexts for learning online. |
| Implementation of evaluation and assessment measures | Evaluation is a critical component of an excellent distance education program and should not be an afterthought but rather incorporated into the planning phase of the program from inception. |
Main Focus: How Do We Evaluate?
Given the variety of distance-education programs available today, from offering one course to an entire degree program, program evaluators should not adopt a standardized instrument to use for evaluating programs. Rather, they should consider the best-practices literature as a guide to developing an evaluation plan that addresses the unique context and setting for their distance-education program (Law et al., 2002).
Best practices for distance education can be used as a gold standard for setting the criteria for evaluating distance-education programs. The underlying question to ask when designing an evaluation should be: Is this particular standard relevant to the program? If so, to what degree of quality should it be measured? Rubrics built upon best practices are helpful tools in developing criteria for measuring program impacts. Not every program will have every best practice, as not all practices are appropriate for all programs. The evaluator’s role is to make a salient argument for inclusion or exclusion of best practices given the uniqueness of each program.
Key Terms in this Chapter
Evaluate: “The process of determining the merit, worth, or value of something, or the product of that process. Terms used to refer to this process or part of it include: appraise, analyze, assess critique, examine, grade, inspect, judge, rate, rank, review, study, test” (Scriven, 1991, p. 139).
Institutional Context and Commitment: Electronically offered programs both support and extend the roles of educational institutions. Increasingly they are integral to academic organization, with growing implications for institutional infrastructure (Western Association of Schools and Colleges, 2001, p. 1).
Evaluation and Assessment: Both the assessment of study achievement and evaluation of the overall program take on added importance as new techniques evolve. For example, in asynchronous programs, the element of seat time is essentially removed from the equation. For these reasons, the institution conducts sustained, evidence-based, and participatory inquiry as to whether distance-learning programs are achieving objectives. The results of such inquiry are used to guide curriculum design and delivery, pedagogy, and educational processes, and may affect future policy and budgets, and perhaps have implications for the institution’s roles and mission (Western Association of Schools and Colleges, 2001, p. 12).
Stakeholders: “The people whose lives are affected by the program under evaluation, and the people whose decisions will affect the future of the program” (Bryk, 1983, p. 1)
Curriculum and Instruction: The important issues for distance-education evaluation are not technical, but curriculum driven and pedagogical. Decisions about such matters are made by qualified professionals and focus on learning outcomes for an increasingly diverse student population (Western Association of Schools and Colleges, 2001, p. 4).