Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 The Extent of Software Project Failures

The popular computing literature is awash with stories of software development failures and their adverse impacts on individuals, organisations, and societal infrastructure. Indeed, contemporary software development practice is regularly characterised by runaway projects, late delivery, exceeded budgets, reduced functionality, and questionable quality that often translate into cancellations, reduced scope, and significant rework cycles (Dalcher 1994). The net result is an accumulation of waste typically measured in financial terms. For example, in 1995, failed U.S. projects cost $81 billion, with an additional $59 billion of overspend, totalling $140 billion (Standish 2004). Capers Jones contended that the average U.S. cancelled project was a year late, having consumed 200 % of its expected budget at the point of cancellation (1994). In 1996, failed projects alone totalled an estimated $100 billion (Luqi and Goguen 1997). In 1998, 28 % of projects failed, at a cost of $75 billion, while in 2000, 65,000 U.S. projects were reported to be failing (Standish 2000). McManus and Wood-Harper (2008) reported that the cost of software project failure across the European Union in 2004 was €142 billion. More recently, a McKinsey–Oxford survey of more than 5,400 software projects revealed that half of all projects significantly fail on budgetary assessment, while 17 % of projects actually threaten the very existence of the company, with the average project running 45 % over budget and 7 % behind schedule, while delivering 56 % less functionality than predicted (Bloch et al. 2013). According to the report, achieving $15 million in benefits now requires an average spending in excess of $59 million.

Yet, software project failure is not a new phenomenon. The first indications of the problem and mention of the term ‘software crisis’ were made during the NATO conferences in 1968 and 1969 (Naur and Randell 1968; Buxton et al. 1969). Indeed, conference attendees reported a set of symptoms that would resonate with the issues raised by developers and managers today. Over 30 years ago, a GAO report in the USA (Anon 1979) showed that there were serious problems associated with the development of software. Less than 2 % of the total value of contracts could be used efficiently as delivered and a further 3 % could only be used after changes. The rest of the projects had the software delivered but never successfully used; the software paid for but not delivered; or the software used but extensively reworked or later abandoned. Moreover, the first edition of the best-selling book in software engineering tells the story of a huge IBM software project with major cost and schedule delays which teetered on the brink of disaster for a number of years from the perspective of the project manager trying to stabilize the project (Brooks 1975). Indeed, the OS360 project came close to bankrupting IBM.

Consultancies and polling organisations have attempted to collect market data about the prevalence of failure. The Standish Group, for example, has been compiling an annual failure survey since 1994. In 1995, 31.1 % of U.S. software projects were cancelled, while 52.7 % were completed late, over budget (cost 189 % of their original budget), and lacked essential functionality (Standish 2000). Only 16.2 % of projects were completed on time and within budget; only 9 % were in larger companies, where completed projects had an average of 42 % of desired functionality (ibid.). The 1996 cancellation figure rose to 40 % (ibid.) before improving to around 15 % in 2002 (see Fig. 2.1). However, the most recent figures reveal that the current failure rate is 21 % (Standish 2011) with 63 % of overall projects labelled as not successful. Note that problems associated with cost estimation and the apparent optimism of software project managers are covered in Chap. 3.

Fig. 2.1
figure 1

Standish figures 1994–2010

Full size image

While the research approach used by the Standish Group has been challenged over the methodology adopted and its rigour (Glass 2005, 2006; Jørgensen and Moløkken 2006; Eveleens and Verhoef 2010), the figures provide a well-referenced baseline related to the extent of software project failures. Other studies appear to confirm the high failure rates. For example, Taylor (2000) reported that only 130 projects out of 1,027 were considered successful, while a 2004 PriceWaterhouse-Coopers study surveyed 10,640 projects and revealed that only 2.5 % of companies achieve budget, scope, and schedule targets on all their projects. Sauer and Cuthbertson (2003) reported that 16 % of IT projects (with a major emphasis on software development) were considered successful, however Sauer et al. (2007) noted that 67 % of the projects were nonetheless delivered close to budget, schedule, and scope expectations. More recently, McManus and Wood-Harper (2008) discovered that only one in eight IT projects can be considered truly successful, with almost a quarter (23.8 %) cancelled due to issues related to requirements, change, communication, business process alignment, and overspend. Using similar definitions, IBM (2008) reported that only 40 % of projects experienced by 1,500 change management executives met their schedule, budget, and quality targets, while KPMG (2010) observed that 70 % of surveyed organisations in New Zealand had experienced a failure in the previous 12 months. Following interviews with 600 developers, Geneca (2011) reported that 75 % of project participants lacked confidence in project success, admitting that their projects are ‘doomed right from the start’.

Not surprisingly, larger and longer projects fare worse. Figure 2.2 shows the probability of success mapped against project duration relating to a body of 23,000 projects accumulated by the Standish Group. The diagram confirms that the probability of success is much higher for smaller projects; for longer projects, the likelihood of a successful outcome is significantly decreased. Jones (2007) confirms from his detailed studies that the risk of cancellation or major delays rises rapidly as the overall application size (measured in function points) goes up.

Fig. 2.2
figure 2

Standish figures: success against project duration

Full size image

Jones (2007, 2008) investigated the likelihood that the average U.S. software project will be cancelled, typically due to cost and schedule overruns, failure to meet requirements, poor planning, estimating, quality control, or excessive requirements creep, relative to size. The results indicate that none of the eight domains investigated are fully successful for large systems of above 10,000 function points in size, showing the average probability of cancellation at 36 %. He warned that the ‘development of large applications in excess of 10,000 function points is one of the most hazardous and risky undertakings of the modern world’ (Jones 2007). Applications in the region of 100,000 function points are more likely to fail with an average cancellation likelihood reading of 51 %, with some sectors such as Management Information Systems displaying higher failure rates (70 %). Jones (2008, p. 308) concluded that: ‘Cancellations, major delays in excess of one calendar year, and cost overruns in excess of 100 % remain endemic problems for software applications in the 100,000 function point size range, and larger.’ Jones (2010) further added that: ‘large software projects are almost always over budget, usually delivered late, and are filled with bugs when they’re finally delivered. Even worse, as many as 35 % of large applications in the 10,000 function point or more size range will be cancelled and never delivered at all.’

Flyvbjerg and Budzier (2011) contended that IT projects are now so big and their influence so wide ranging across many aspects of the organisation, that they pose a singular new kind of risk that can sink entire corporations, cities, and even nations. Their global survey of 1,471 IT change projects showed that while the average cost overrun on large initiatives was 27 %, one in six projects showed a cost overrun of 200 %, on average, and a schedule overrun of almost 70 %. As software is integrated into bigger products and systems, the concerns can become magnified. ‘The software industry has the highest failure rate of any so-called engineering field. An occupation that runs late on more than 75 % of projects and cancels as many as 35 % of larger projects is not a true engineering discipline’ (Jones 2010).

2 Beyond Simple Success Measures

The relationship between success and failure is not clear. Some view the relationship as a binary function so that a project is either successful, or not. The research by McManus and Wood-Harper describes failure as ‘those projects that do not meet the original time, cost and requirements criteria’. The Standish Group makes a further distinction between ‘failed projects’ and ‘challenged projects’. Failed projects are cancelled before completion, never implemented, or scrapped following installation. Challenged projects are completed and operational projects which are over-budget, late, and with fewer features and functions than initially specified. Successful projects, in contrast, are completed on time, on budget, with all specified features. Figure 2.1 also shows the relationship between successful, challenged, and failed projects. Observing the Standish figures over the past 19 years would appear to indicate a rough rule of thumb, suggesting a split of 25 % of projects being successful, 50 % being challenged, and 25 % failing.

The Oxford Dictionary defines success as: a favourable outcome; doing what was desired or attempted; the accomplishment of an aim or purpose; or the attainment of wealth or fame or position. Failure is broadly defined as lack of success supporting the idea of a binary relationship. In an attempt to make further sense of the relative positions of success and failure, software surveys have clearly found it useful to introduce the idea of partial failure (or challenged projects) as an intermediate position between success and failure, potentially indicating dissatisfaction with a two-state explanation. Indeed many project outcomes do not fall directly into either category.

The majority of the studies mentioned above define success as meeting all the criteria associated with the budget, schedule, and functionality; with failure viewed as a failure to meet all of the same criteria. This implies that if a project is finished on time, within budget, whilst offering the expected functionality, it can be viewed as successful. Conversely, failing to meet any of the criteria will deem it a failure. The view is predicated on the traditional measures applied in project management and generally known as the triple constraint, the golden triangle, or the iron triangle. This idea presupposes high estimation accuracy with regard to the initial formulation of the variables of the triple constraint (Eveleens and Verhoef 2010) when the degree of uncertainty is at its greatest.

Traditional project management theory holds that optimising the three criteria will result in ideal performance on a project. Typical projects thus require a balancing act between the so-called triple constraints of time, cost, and functionality as expressed in the original triangle conceived by Martin Barnes in 1969 (see Fig. 2.3). Note that the third corner is named ‘performance’. The original release named that corner ‘quality’, but this was soon corrected to performance ‘to reflect whatever the finished product was supposed to achieve’ (Barnes 2013). Performance means satisfactory function of the product, which has to be fully defined. This could be specified in terms of rate of return, profit, beat the enemy, impress visitors, or in the case of software the project scope and expected functionality. The whole point of the triangle is that the spot can be placed at such a point that its closeness to each corner represents its relative importance and helps the project manager to make informed decisions about the project. Trade-offs and adjustments are therefore made by restricting, adding to, or adjusting the cost, time, and performance associated with a project. The triangle enables managers to consider each decision and its implications on the dimensions of time, cost, and performance and integrate the different project management functions. For example, the more that is requested in terms of performance, the more it is likely to cost and the longer the expected duration. If the client needs to have a certain performance delivered very rapidly, this will increase the cost due to the need to work faster and have more resources involved in the development, albeit with increased communication costs. The more features expected from a system, the higher the cost and the longer the expected duration. Conversely, if the costs need to be kept to a minimum, one may need to consider the essential performance, or the overall project scope, and compromise there (Dalcher and Brodie 2007).

Fig. 2.3
figure 3

Budget, time, and performance trade-off

Full size image

Many managers quickly discover that the triangle is not flexible. The three factors are closely entwined and project managers are expected to balance the what (performance) with the when (time) and the how much (cost). Kloppenborg and Mantel (1990) discerned that the importance assigned to the factors varies systematically according to the life cycle stage. Optimisation and trade-offs will thus depend on the phase during which decisions are made. While tools such as project priority matrices (see Fig. 2.4) may be utilised to prioritise the constraints underpinning the decision process, managers would need to consider the overall priorities and recognise that priorities may shift according to the stage of development. The priority matrix offers a simplifying mechanism for making decisions. The horizontal axis depicts the three key constraints of time, performance, and cost, while the vertical axis covers the suggested treatment options (constrain, enhance, or accept). In the example below, the proposed project offers a new type of market-leading functionality which must be available in full; that is, it MUST meet the performance criteria—hence fixing, or constraining, that aspect. Given the intention to release before the competition, every effort should be spent on activities that enhance or help project delivery to ensure the product is first to market. The trade-off, therefore, requires that reasonable additional costs are accepted in order to optimise the time criterion, whilst strictly adhering to the performance brief. The matrix offers a structured way of considering the impacts offering a simplified version of the trade-off triangle.

Fig. 2.4
figure 4

Project priority matrix

Full size image

In practice, performance is often determined prior to the project. Moreover, project managers often inherit the overall budget from the contracting activities that may even have imposed a fixed-price contract structure. A fixed overall budget may also exclude typical remedies like the hiring of specialists and the addition of human resources. The only remaining latitude for leverage is in the schedule. However, this may also be imposed on a project through a fixed date for delivery with little regard to the complexity of the intended system or the risks it embodies. Once both budget and schedule are fixed, there appears to be little capacity for compromises and trade-offs.

The three factors clearly play a key part in determining the degree to which a project is challenged (or even deemed a failure); yet they may be uncontrollable by the project manager. Indeed, Capers Jones observed that the most common constraints encountered are: fixed delivery dates; fixed-price contracts; staffing or team size limitations; and performance or throughput constraints (Jones 2000) i.e., fixed time, price, staffing level, performance, and scope. Many managers are thus looking to control other factors that may alter the outcome of the project, in particular as the constraints often occur in concert. Measuring success on the basis of preestablished parameters that cannot be adjusted is therefore of limited value.

Before addressing additional factors in the next section it is also useful to point out that the artefacts of projects, especially the final delivered systems, interact with organisations, customers, stakeholders, and other systems. Their impacts, regardless of whether or not they are delivered on time, can be crucial and perhaps even fatal in financial or real terms. Dalcher reports on the impact of an ambulance despatch system that was delivered to the users, the citizens of a major metropolis (on the third attempt), yet failed in action subsequently, potentially leading to loss of life (Dalcher 2010). Another illustration is a UK disaster which followed an earlier, yet unrelated failure. The delay in introducing the Nirs2 system into the Inland Revenue beginning in 1995 meant that additional backlogs were building up. The backlogs caused the Inland Revenue to stop sending reminders to up to a third of the UK working force, warning them that they needed to top up their national insurance contributions. As a result, around 10 million people face a state pension shortfall. The impacted party includes the lowest paid workers in the UK. While the backlog resulted from a delayed system that itself cost taxpayers millions of pounds, the additional loss will be borne by individuals and only count as a hidden backlog indirectly stemming from another failure. The true cost to individuals is likely to be £15 billion and the hardship that ensues as a result (BBC 2003). Therein lies the complexity of counting the costs of failure. Numerous other compilations have been published identifying major impacts on individuals, organisations, and society at large (see for example, Dalcher 1994, 2012; Flowers 1996; Charette 2005). The success, and failure, of software projects therefore cannot be delimited by a simplified set of factors and constraints associated with the delivery effort.

3 Rethinking Project Success

Project success is a rather nebulous concept and the focus on the triple constraint can be too limiting. Indeed, Linberg (1999) asserted that a whole new theory of project success is needed. Pinto and Slevin (1988) noted that success combines issues related to the project itself with issues related to the client. Moreover, software developers and systems analysts have recognised long ago that user involvement, satisfaction and buy-in are crucial to the success of software projects. Prototyping and user-driven approaches were developed to maximise the potential for satisfaction for various stakeholders and thus increase the likelihood of user acceptance of the ultimate system.

Baccarini identified the need to distinguish between project management success and the success of the product which entails dealing with the effects of the project’s final delivered product (1999), thereby allaying the need to define a further dimension concerned with client expectations which have already been expressed in the desired performance functionality. Ironically, this chimes with the original (but often misunderstood) intention of the Barnes’ triangle (Fig. 2.3) to capture the agreed upon definition of the purpose of the project or how the complete project would perform. Given that the product will be utilised by the client there is a degree of correspondence between the dichotomies put forward by Pinto and Slevin and by Baccarini. Indeed, de Wit (1988) observed that measuring progress and cost are part of project control, which should not be confused with measuring success. Cooke-Davies (2002) likewise made a distinction between the focus on project performance and the need to look at the success of a project.

Having multiple categories of success would suggest that it is possible to be successful in some areas and not successful in others. It thus makes it possible to understand mismatches between the different criteria and groups. Moreover, it implies that the traditional triple constraints of cost, time, and performance only reveal part of the picture. In other words, it may be possible to maximise the traditional criteria and yet deliver a product that is not valued by the users. Likewise, it is also possible to exceed the traditional criteria but deliver a product that is valued and adopted by the user community, despite exceeding the budget or the schedule, or even both.

The discussion thus far indicates that at least two different levels of success can be identified. Indeed, according to Munns and Bjerimi (1996) it is possible to achieve a successful project even when management has failed, and also possible to deliver a failed project following successful management. However most studies and surveys of software project failures tend to focus on the traditional criteria of efficiency embedded through the triple constraints of time, cost and performance. They thus ignore the deeper aspects associated with the delivered product, its perceived utility and value, the expectations and needs of stakeholders, the intended performance of the product, and the project context.

Further evidence of the need to look beyond the traditional criteria is provided in Table 2.1, which summarises an extended and refined set of common issues that were originally identified across six project failures covered in detail in Dalcher and Genus (2003) and extended through a sequence of workshops with practitioners, the mapping of factors in 150 failed projects and a series of four international surveys resulting in the revised figure presented in this chapter. Note: Escalation of commitment and its impact on projects is explained in detail in Sect. 11.3.

Table 2.1 Groupings of crucial issues observed in failures
Full size table

The obvious message from the set of issues is that the traditional efficiency criteria as embedded in the triple constraint do not appear to have played a part in the build-up to any of the failures. Instead, the issues identified were more concerned with the product (as well as the assumptions and expectations surrounding it) and the overall business success.

It may also be instructive to scrutinise other domains and sectors. When the UK Government recognised that the construction sector was underachieving, it assigned a task force to determine the causes of the shortfall. The study recommended substantial changes in the culture and structure of the sector (Egan 1998). Crucially, it perceived the need to replace competitive tendering with long-term relationships to address the growing dissatisfaction of both private and public sector clients. The main criticism was reserved for the way projects were assessed as the focus on time, cost, and quality was recognised to be wholly inadequate. Overall, the task force acknowledged that the construction sector had thus far failed to meet the needs of modern business by failing to look beyond the traditional success criteria to determine the true performance of projects. A 10-year retrospective review re-affirmed that progress had yet to materialise (Egan 2008).

4 Towards Multiple Levels of Success

Success, it would appear, needs to be understood at multiple levels in order to appreciate the complex dynamics and subtle impacts. A tabular representation of four levels of success, which builds on the earlier discussion, is offered in Table 2.2.

Table 2.2 Levels of success
Full size table

Level 1 represents project management success and is thus concerned with internal efficiency and performance measurement and optimisation at the project level through the tracking of the cost, schedule and performance parameters. Level 1 success is therefore to do with project delivery against the constraints or measures imposed on the project.

Level 2 is focused on the overall effectiveness of the project through the lens of what is actually being delivered. Success is measured through the utility and acceptability of the output that has been delivered. The benefits of the projects and the achievement of the objectives are thus assessed in terms of the satisfaction of the customer and the different stakeholder groups. Level 2 success reflects the acceptability and impact of the resulting artefact, the benefits that it delivers, the degree to which it is used, the quality built into it, the match with the project objectives, needs and requirements, the relationship with the different stakeholder groups, and the overall impact on the customer.

Level 3 is centred on the business efficiency which is assessed through the creation and delivery of internal value. The outcome of the project contributes to business success through the satisfaction of business objectives that have been realised. Success equates to maximisation of financial and business efficiency measures, such as sales, profits or ROI as well as delivered value measures.

Level 4 is forward looking and opportunistic and enhances the business horizon by projecting future gains and opening new avenues, capabilities, skills, and markets. Strategic opportunities require a continuous and long-term approach that seeks to derive not just immediate benefit but also maximise opportunities for cornering the market, creating killer applications, and building the potential for self-enhancing positive feedback loops to secure future growth. Level 4 success is achieved through the realisation of new opportunities and harnessing of new potential. It may include new uses or ideas that were not originally considered as well as the development of new competence or capability.

The focus identified in Table 2.2 provides a clue as to the nature of measurements required at each level. Measurement at Level 1 focus on determining the progress and efficiency of the project management effort, for example, through the use of earned value management. Measures for Level 2 are concerned with benefits realisation and measuring the achievements of objectives, requirements, and expectations. Measures for Level 3 emphasise the business value using traditional economic measures such as sales, revenue, and delivered value. Measures for Level 4 require a more creative measurement of opportunities, capabilities, and market position. The combined levels offer a richer way of conceptualising and making sense of the complex phenomena surrounding success in and around projects.

5 Mapping Success

It is interesting to note the horizon of activity for each of the levels of success. At Level 1, project management success is concerned with the execution of the project itself based on performance against internally established constraints. Success at this level is determined upon the delivery of the project, often achieved through the incremental delivery of partial targets. It is primarily concerned with the task of project control. This is what most failure surveys assess and, therefore, where most failures are observed.

Level 2 success is more deeply entwined with the design activities resulting in the product or deliverables; indeed this is where utility and quality provide the key to the assessment of success. Both levels can be said to be output driven as they look at the complementary aspects of technical action and management within established and imposed constraints. Level 2 success can extend to cover the entire operational life of the project. After all, delivering a bridge that stands for 6 months before collapsing is far from being a mark of utility, quality, or success.

The levels of success show a shift from focusing on internal efficiency and outputs, to outcomes and value as more strategic considerations come into play. The main distinction is that outputs occur as a result of a process as they relate to the specified deliverables and artefacts (that are delivered within time, cost, and performance constraints) viewed in terms of tangible products and services, such as a new web interface. Outcomes are the effects of change and how it translates into value for the entire business even beyond the reach of the original project. Outcomes happen as a result of the work and will often support the realisation of strategy through new capabilities or capacities, such as improved access to services or systems resulting from the new web interface. This relationship is depicted in Table 2.3.

Table 2.3 Focus vs. output/outcome
Full size table

Encouraging long-term thinking is important from a strategic perspective as it enables organisations to realise corporate strategies. It also fits with the need to deal with extended life cycles and consider deployment, extended use, and decommissioning of artefacts alongside benefiting from new opportunities and market possibilities. Moreover, it also chimes with the idea of viewing software development as the development of a continuous service (implying the fostering of long-term relationships and the dedication of attention to strategic concerns at the operational end) rather than the delivery of a single artefact. It is worth noting that while Levels 1 and 2 are primarily concerned with the delivery of a single project, the remaining levels look beyond a single delivery view by utilising a more strategic lens that considers operation and investment cycles.

A further important distinction is the separation between efficiency and effectiveness. Project managers and software developers have shown a tendency to focus on efficiency and its implications, as is reflected by the continuing obsession with failure studies. However meaningful solutions emerge from consideration of effectiveness. Efficiency is essentially viewed as an internally oriented productivity metric and method of evaluation, as it is concerned with following procedures and processes, adhering to constraints or achieving with minimum resources. Effectiveness on the other hand, relates to the overall utility; the ends to effectiveness’ means.

In a nutshell, efficiency focuses on optimisation of the available resources by ‘doing things right’, while effectiveness revolves around the fulfilment of objectives and the contribution to achieving organisational goals by ‘doing the right things’. This is depicted in Table 2.4 which also shows the types of measures required.

Table 2.4 Efficiency/effectiveness vs. timing orientation
Full size table

Failure studies and surveys seem to focus on criteria concerned with the efficiency of projects, while ignoring the effectiveness aspects, and thus sidestepping the major issues associated with the outcomes of change. Indeed, even the bodies of knowledge in project management seem concerned with the track record of projects as a measure of quality. However, this is not the case as achievement in line with the triple constraints measures the ability to predict deadlines when uncertainty is at its highest and stick to them. This is not a measure of quality and is therefore addressed as Level 1 success. To attain project success, one needs to relate to the benefits, impacts, and quality aspects and perspectives related to the effectiveness of a project.

Quality, utility, and success are judged by different stakeholders in different ways, employing different criteria, over different timescales (Morris and Hough 1987; Wateridge 1995; Turner 2009)—see also Chap. 6. Recently, there has been a tendency to let the customer define quality. The Kodak organisation defines quality as ‘those products or services that are perceived to meet or exceed the needs and expectations of the customer at a cost that represents outstanding value’ (Kerzner 2009). The interesting point to note with this definition is how the customer viewpoint impacts on a project: a project must take great care that it accurately defines the customers’ needs and expectations, as the ultimate power about deciding on quality is given to the customers. So with this type of definition, conformance to requirements is not necessarily sufficient—the customer must be satisfied with the resulting product or service. Further, in order to maintain the satisfaction of customers and their loyalty and to ensure higher levels of success, products need to be revised and adjusted to reflect shifting needs and expectations (as well as market trends and competition). Consequently, maintaining quality becomes a continuous process of product (and process) improvement (Dalcher and Brodie 2007).

6 Illustrative Examples

To highlight the distinctive features of the levels of success and the differences between them it might be instructive to focus on thumbnail sketch examples of projects from a range of sectors and reflect on their relative success in comparison with the four levels.

Story 1: The operation was successful but the patient died. Level 1 success—Level 2 failure. This chapter began by describing a number of failure surveys focused on project management failure (i.e., the inability to meet time, cost, and performance criteria). Project management success is no guarantee of project success as many targets are assigned arbitrarily at an early phase. For example, the third attempt to deliver a working ambulance despatch system for London was delivered by the agreed deadline (Level 1 success), but stopped working a few days later resulting in potential loss of life (Dalcher 2010). This experience will chime with other software projects that are delivered on time and within budget, covering the agreed scope, which are ultimately never used by the users, or fail within a few weeks of the handover date.

Story 2: The Millennium Dome: Early partial failure becomes Level 4 success. The project to deliver a dome-shaped building to house a 1-year exhibition to celebrate the millennium had to be delivered in time for the new Millennium. The building itself and the infrastructure enabling Londoners to experience the exhibition were just about finished on time, but only following an unexpected injection of additional funds. On opening night, many of the exhibits were not functioning and dignitaries were left to queue outside for hours. Expected visitor numbers as specified in the business case exceeded one million per month, but in practice only about a third of the expected visitors turned up. The exhibition had to be kept open for a further year (in clear violation of the stated intention) to try and recoup some of the costs, while the entry fee was ultimately cut in half in order to attract additional visitors. Following the end of the exhibition, the site was mothballed at a cost of £190,000 a year adding to the accumulated losses. However, once the sale was finally concluded, the renamed O2 Centre became the biggest and most successful sports and entertainment arena complex in Europe. Level 4 success through innovative use of the structure thus managed to make up for the earlier disappointments (albeit in the hands of a new owner).

Story 3: The Sydney Opera House: Technical failure—architectural marvel. An even more heralded failure which clearly failed in terms of project management, project, and business. The Sydney Opera House came in at 14 times over budget, a clear project management failure. The building was unsuitable for its original purpose as the acoustics made it impossible to have concerts inside the hall. However the building has become an icon and is considered to be an architectural marvel. It is attracting tourists from all over the world and generating revenue, not least for the entire city. The revenue is not generated under the original intention but the new potential has been utilised to the full. Interestingly, the building was not fit for its purpose and hence was not of acceptable quality, yet it managed to generate a new purpose for which it was fit enough.

Story 4: Project Orion: Level 1 & 2 innovation success—long-term failure. A massive effort to develop Kodak’s new Advantix photographic system was considered a big success on completion. The product was selected by Business Week as one of the best new products of 1996 (suggesting project success). It also won the Project Management Institute International Project of the Year award, confirming that it was also a project management success. The only problem was that Kodak failed to anticipate the accelerating switch to digital photography which made the product redundant. A successful output that won multiple awards was thus destined to become a failure as an outcome as it failed to deliver the promised value. In terms of utility the resulting product was an award winner but at the wrong end of the utilisation and relevance curve just as the technology slid out of fashion and was replaced by a new innovation.

The brief vignettes highlight the complexity of success and the interplay between the different levels involved. Success is never simple and the mini cases help in shedding further light on the rich, interconnected, and intricate (and sometimes temporary) nature of success.

7 The Impact of Time

Time is often viewed as a strategic resource. In some cases, it is viewed as being more important than money. This brief section focuses on the temporal nature of success and failure through the lens of time. In essence, we all recognise that things change. Perceptions, values, priorities, and objectives shift over time. Views about success and failure may also be similarly impacted. What is considered to be success, or a failure at a given point, may need to be adjusted to reflect new perceptions or changing views. Some of the individual cases described above hint at the change of perception over time. The following mini-case emphasises that point further.

Story 5: Raising the Vasa: Celebrating failure. The Vasa was the pride of the Swedish Royal Navy on August 1628 when it set sail on its maiden voyage. According to Fairley and Wilshire (2003), it was Sweden’s most expensive project ever. The ship was much needed to bolster the Navy following a number of defeats in the Baltic Sea during the war with Poland. It was also meant to outperform a Danish ship being built at the same time. The completion of the ship was heralded as a great success and thousands turned up for cheer. Yet, definite success turned to failure when the ship capsized and sank after managing to sail less than a nautical mile out of Stockholm on a calm day. The crowd, which included foreign ambassadors invited to witness the great occasion, watched the ship disappear only 120 m from shore, with the loss of 30–50 sailors. In the space of minutes the successful release had turned into a catastrophic failure, accompanied by the loss of human life. The Vasa 333 years later, was raised and ultimately installed in a Museum close to where it sank. The Vasa is now one of Sweden’s most popular tourist attractions as an embarrassing episode was radically transformed into a long-term success by ‘celebrating’ and analysing the failure.

The moral of the story is that the perception of success, and failure is wholly dependent on the point at which it is assessed. An undertaking can shift from being viewed as a success, as in the case of a standing bridge, or a floating ship, to being regarded as a total failure following structural, or even human failure. Nonetheless, over time new opportunities emerge and the perception can once again shift. Understanding and labelling of phenomena is therefore clearly aligned to the period in which it is perceived, observed, and categorised. The benefit of hindsight, the burden of maintenance, or the chimes of progress can revive, question, and re-enliven previous pronouncements.

It is also worth noting that outcomes persist beyond the life of a given project or programme. Increasingly, there is talk about the legacy of projects which delays the assessment of the impacts and results on initiatives. For example the legacy targets of the London 2012 Olympic games include local regeneration, increased participation in sports activities, enhanced enrolment in local clubs, and a reduction in rates of obesity among young people (BBC 2012). Such targets and outcomes can only be evaluated longitudinally. The reasons for the longer time horizon include the following:

  • Effects following significant interventions are only temporary and partial.

  • Evaluation requires a longer time horizon.

  • Benefits were designed to be measured over longer time periods.

  • Perceptions and priorities shift with time.

  • Impact is measured in terms of changes to adaptive patterns over time.

8 Measuring Success

Determining the success of a project is not simple. It is often said that success is in the eye of the beholder, and can mean different things to different people. Consequently, analysing the dimensions of success and failure is a complicated task (Cooke-Davies 2002; Shenhar and Dvir 2007; Dalcher 2009). Measuring success requires an understanding of the different levels of success and what each one can offer:

  • Project management success implies tracking data related to predicted cost, time, and scope. Measuring performance against efficiency considerations is relatively straightforward. Determining progress through monitoring the achievement of milestones (e.g., using Earned Value Methods) enables project managers to track the achievement of predefined targets. It is a very useful focus when there is little residual uncertainty or when the project is clearly understood. However, it is debateable whether the measurement of an arbitrarily predefined target is completely meaningful, especially when project managers play little, if any, part in the initial estimation. Typical measures would focus on the efficiency of the process emphasising milestones, identified defects, and delivery and change management measures (including approved change requests), as well as earned value management measures showing project management progress, cost and schedule variances, cost performance index, and estimate to complete.

  • Project success relates to the effectiveness of a project and is normally considered in terms of realised benefits. In order to provide meaningful values, measures should relate to the requirements identified and be established and acknowledged as part of the needs assessment and requirements management processes. Stakeholder management is central to the identification and assessment of the concerns of different stakeholder groups and the issues impacting the development team. de Wit (1988) defined success as encompassing a high level of satisfaction concerning the project result amongst key stakeholders and users, while Lyytinen and Hirschheim (1987) framed the effort in terms of meeting stakeholders’ expectations in terms of the balance between the objectives, constraints, and benefits. Project success can therefore be viewed as either realising the stream of benefits allocated to the project as they are cascaded down from the strategic objectives, or equalling and exceeding the expectations of clients, users, and stakeholder groups, thereby emphasising elements of stakeholder satisfaction and management.

    Drevin and Dalcher (2011a, b, c) reported on the different concerns and issues expressed by different stakeholder groups. Using both narrative and antenarrative approaches to make sense of the stories of different communities, unveiled significantly differing interpretations of the purpose of a project, the approaches for dealing with alternative communities and the understanding of the perception of success and failure as they percolate and aggregate in different stakeholder communities. Typical project success measures would identify realised benefits from the project, achieved project requirements, satisfaction levels, recorded complaints, usage figures for the delivered artefacts, and met expectations.

  • Business success pertains to the funds raised from the initiative, or more generally to the organisational value derived in terms of finance, environmental and social concerns, and their balancing. The perspective often requires a longer timeframe that considers value creation and delivery over investment cycles and the contributions made towards the achievement of strategic objectives of the organisation. Business success can refer to the payback period, but often extends to consider the accumulated benefits accruing from an investment in a project or initiative. Business success measures typically address the delivered value, the rate of return, break-even calculations, payback calculations, sales achieved, revenue measures, environmental and social targets, and increasingly may focus on reputation, influence marketing, and sustainability ratings. Note that some of the calculations may be done from the project portfolio management, a multi-project or an enterprise-wide perspective. Occasionally, project managers are involved in devising some of these calculations during the initiation phase of projects, however the business success readings are likely to materialise well after the delivery phase of the project.

  • Future potential extends the time horizon of consideration into the longer-term utilisation of the outcomes and results of projects and actions. It allows the accumulation of longer-term benefits that result from adjustments and re-balancing. The intention is to seek to increase the accrued value from projects by exploring and exploiting opportunities beyond the formalised project baseline. Given the longterm focus it cannot be assumed that project assumptions, and originally intended outcomes remain relevant over time. The aim therefore is to maximise organisational value in accordance with the strategic direction. When projects are completed under conditions of uncertainty, they are often subject to positive feedback cycles, systems dynamics, and complex interactions that uncover new opportunities and strategic openings. Potential opportunities can often lead organisations to explore new directions, expand into a particular market, or occupy a certain leading position within a sector, and adjust their strategic intentions to match their new ambitions. Measures will focus on the identification and utilisation of emerging opportunities and adaptation to new market conditions that result from the experience, learning, and re-positioning related to the project and its outcomes.

Table 2.5 identifies some of the specific measures that can be utilised within each of the success levels. The measures were derived through a series of workshops with over 120 practicing project managers in four countries. The purpose of the sessions was to validate the model proposed in this chapter and to identify specific measures suited to each of the levels. Groups worked independently through the levels and the results were compared and developed further during a second series of facilitated sessions.

Table 2.5 Measures for determining success by level
Full size table

The accomplishment of any of the measures can be absolute or relative. The factors can be combined into a compound measure for a particular level, or a judgement statement regarding the achievement of each specific measure can be independently derived and assessed. In any project, one or more of the measures may be critical and multi-attribute aggregation methods can be used to combine, and trade-off specific measures.

Measurement of success requires understanding of the relevant levels. The identification of meaningful dimensions and the agreement regarding relevant factors could lead to a richer mapping of the relative success merits of projects alongside the multiple levels. More complex profiles can be devised by combining additional measures and considerations within each of the dimensions. Projects can thus be ranked and rated along four, or any other number of dimensions, providing an alternative to the simplistic measures which are currently being over used in failure studies. By shifting the focus to success and recognising the multi-factor nature of the levels, it thus becomes possible to chart a more meaningful picture of success using the levels as dimensions (see Fig. 2.5).

Fig. 2.5
figure 5

Project success radar chart

Full size image

The radar chart will be able to rate the success at each level independently and provide more sophisticated snapshots of projects and initiatives capable of counter-balancing the simplistic measures used to gauge success in project work hitherto. Figure 2.5 provides a relative benchmarking between Project Orion and the Millennium Dome, described earlier, showing how each performed along the four project success dimensions.

9 Conclusions

Project failures have been used to highlight the need to improve IT software project practice. Many of the studies and surveys focus on project management success (or failure), which can be described as a subset of internal efficiency measures and imposed constraints ignoring the impact on the project and the business. In order to improve project performance, project managers need to look beyond such measures and focus on project success—an area concerned with the effectiveness and quality of the project output. Project managers are also increasingly asked to consider the value derived from the project, the sustainability implications as well as issues related to environmental, social, and societal impacts.

Success is a complex and multi-layered concept that needs to be understood at different levels and time frames. Indeed, the impact of success often extends beyond a single project. This chapter offers a wider perspective, which takes in a range of project success levels thus enabling practitioners to move beyond the simplistic measures that continue to be offered. The success view determines actions and colours new developments. Increased attention to enterprise objectives and utility, rather than simply endeavouring to optimise correctness according to preimposed constraints, can open a new dialogue about the needs of a profession seeking to fundamentally and essentially improve its track record and enable project management practice to rise beyond the continuous obsession with failure.

Further work is needed to encourage the research and practice communities to consider project management success at a number of levels. Practitioners will need to make links between strategy, business, and project management delivery functions, while researchers are likely to try and make sense of requirements and expectations that emerge from a multi-level model that invites new types of surveys to make sense of the success and failure in software projects. Ultimately, in order to overcome failure we must learn to appreciate success and grow up enough to look beyond the simplest manifestations of an imperfect practice.

Success is not final, failure is not fatal: it is the courage to continue that counts.

Winston Churchill