PMI Report: Identifying and Realizing Project Benefits: Effectiveness of Frameworks in Application.

Identifying and Realizing Project Benefits - A Cross-national Comparison of Benefits Management Practices.

The full PMI report is available here.

Frameworks used in practice

We observed a wide range of BRM guidelines, frameworks and practices across and within the four countries.
There was evidence of a tendency towards the use of tailored approaches by department/sector in some countries; frameworks for transport/infrastructure tended to be well developed compared to other areas (such as transformation) in the majority of countries.
Generally, some form of BRM frameworks were used and their purpose understood; however, the level of uptake was variable.
Frameworks were often only advisory, except sometimes where they were effectively mandated for preparation of business cases prior to project sanction, after which the emphasis was on project delivery rather than outcomes/benefits.
We acknowledged increasing emphasis on public scrutiny.

Common reasons why the prescribed frameworks were not followed:

Limited resources,
Concerns regarding the extensiveness and quality of the frameworks,
Lack of buy-in from senior management,
Lack of awareness of the existence of the frameworks. Common motivations behind the prescribed frameworks:

Benefits identification

All of the countries had some organizations which had well-developed frameworks and practices for
benefits identification and Cost-Benefit Analysis (CBA). Policy and guidance on CBA and business cases were sometimes mandated for project proposals exceeding a certain threshold, although implementation varied in different government departments.
There was a general intention to align project benefits with strategic objectives at the project start, although nuances in the detail of strategic fit in different departments and sectors.
Optimism bias and ‘gaming the system’ occasionally occurred to ‘get things through,’ and there were a few systems in place to minimize this. The UK was the only country offering an explicit approach to quantifying optimism bias.
All countries had various classification systems for types of benefits, which varied by departments and sectors.
Stakeholder engagement and discourse were increasingly used in benefits identification in some organizations, although it was unclear whether this process was formal or the extent to which process outcomes fed into decision-making. We did not find methods to help align conflicts between stakeholders.

Benefits management/ realization

There was a clear emphasis on getting the project through
approval. Emphasis on BRM tended to fade once funding was achieved, apart from occasional enthusiasm for benefits tracking.
Most participants recognized changes in expectations of project benefits through the project from identification through to ex-post evaluation; however, there were no practices in place to capture that evolution.
There was not a clear picture of how to balance delivery and benefits during execution.
BRM and Risk Management methods are not generally integrated (except sometimes in the US and NSW); however, there was a strong interest and slow trend moving towards this way. The body of knowledge, e.g., implied that needed a mind-set change.
We did not detect clear evidence suggesting that BRM is more or less relevant in different sectors.
Many participants did not relate to agile methods, but those who did felt that BRM methods seemed well suited to agile projects.

Ex-post evaluation

Long-term ex-post evaluations of projects were considered important but rarely done due to the nature of long-term public projects – and thus there was little learning regarding BRM best practice.
A definite answer on the appropriate time to assess longer-term benefits could not be found since it depends on the type of project and projects varied significantly.
Public reporting on the longer-term evaluation of benefits occurred but again varied between countries. They are usually only done for some specific projects.
Where ex-post evaluations were undertaken, participants recognized the particular problems and challenges of disentangling benefits between projects or from general economic trends.
All felt that capturing lessons learned was important, but practice varied across countries.

Quantification

All participants confirmed the important role of benefits quantification methods in setting a baseline, attaining funding, and assessing changes to the benefits position during the project lifecycle. However, quantifying benefits appears to lack standardization.
The maturity and compulsion of quantification processes varied widely between different parts of government. Most participants felt that current systems were not sufficient and/or sophisticated enough to measure many different types of benefits.
There was a strong emphasis on quantifying benefits which are easy to measure or useful for showing strategic alignment in the project proposal. Most participants could not give evidence of a clear treatment for benefits which could not be quantified or were difficult to monetize.
We found that while all acknowledged the usefulness of ex-post evaluations to determine the effectiveness of various BRM approaches, few efforts were made to collect data on benefits and costs to use for future projects, although there was some evidence that this is possibly increasing.

Effectiveness of frameworks

We found a mixed picture of the efficacy of the frameworks. Many barriers and enablers to the wider adoption and implementation of the frameworks being used were identified, and questions raised about how barriers could be overcome and enablers facilitated.
Common barriers include, e.g.,
- lack of awareness
- lack of senior management buy-in,
- lack of Culture of doing BRM
- lack of requirement for ex-post analysis
Common enablers are, e.g.,:
- desire to maintain a database of best practice.
- the culture of doing BRM,
- senior level support
- increasing stakeholder engagement  clear accountability