Article Preview
TopIntroduction
Despite an on-going discussion for many years, there is no satisfying solution for the problem of an adequate and comprehensive evaluation or measurement of information system project success (ISPS) in research and practice (Aladwani, 2002; Cuellar, 2010). There is no generally agreed on definition of ISPS, that is, which criteria to use and the way of operationalizing these. There are several possible reasons for this observable heterogeneity: (1) IS project success is a construct, not a concrete object. That is, it is not possible to delineate a definition of ISPS directly from observation. (2) Evaluating success is subjective by nature. Every definition of ISPS inherently reflects subjective values and objectives of for instance a stakeholder group or a research approach. (3) Research of ISPS stems from different domains, the most important ones are Project Management and Information Systems, that have own research traditions and an own body of knowledge.
However, there is agreement concerning the following points. First, measuring ISPS is important and relevant (Peslak, 2012; Thomas & Fernández, 2008). Second, ISPS is a complex and multidimensional, that is, multicriterial construct (e.g., Agarwal & Rathod, 2006; Aladwani, 2002). Finally, ISPS encompasses more than the classic planning related criteria of time, cost, and quality (Agarwal & Rathod, 2006; Cuellar, 2010; Wateridge, 1998) and can therefore not be adequately measured by using only these.
There is a multitude of examples of projects that are developed in conformance with their budget, schedule, and requirements, but nevertheless are subjectively perceived as failure by some or all project stakeholders. Accordingly, projects that exceed one or more of the traditional criteria are nevertheless often regarded successful according to project stakeholders’ subjective perceptions. Obviously, measurement of ISPS is amongst others a matter of perception. It is usually argued that more criteria must exist to explain these perceptions. As there is no consensus with regard to the question what criteria these are, the problem of adequately defining and measuring ISPS is not solved yet.
We found two articles explicitly researching the actual measurement of project success in general (Collins & Baccarini, 2004) or ISPS in particular (Thomas & Fernández, 2008). Consequently, there are on the one hand, extensive findings concerning theoretical approaches on ISPS evaluation and the related expert knowledge. On the other hand, there is mainly anecdotal evidence about the actual evaluation of ISPS in companies. It is often assumed that companies still exclusively or mainly rely on the traditional evaluation of project success, that is, time, cost, and quality. This assumption has not been checked for correctness.
In this paper, we aim to fill this gap and show how German companies actually measure ISPS. Additionally, we provide insights into the reasons for their way of measurement. We conduct semi-structured interviews with decision makers in IS developing companies and their customers in Germany. By doing this, we focus on the following research questions:
- •
Which criteria do companies use to measure the success of their IS projects?
- •
How do decision makers subjectively evaluate IS projects? Do they use additional or different criteria compared to their companies?
- •
In which way do the practical approaches differ from the theoretically developed ones that are proposed in literature and how can this discrepancy be interpreted?
By answering these questions, we gather knowledge concerning the ISPS evaluation in practice. Moreover, we advance the body of knowledge by assessing the relation between actual practice and the current state of research.