Are project metrics dead? A quick guide to Lean Portfolio Measurement

Having spoken with many organizations who have largely adopted the lean-agile delivery model the possibly surprising answer is NO.

Agile delivery model supporters argue that the methodology opposes anything that remotely resembles traditional project management practices. But even those that call themselves agile by nature, confess to some nuance.

Some of this can be attributed to resistance to change: either out of precaution (just to be sure) or out of convenience i.e. sticking to something everybody understands. Most are aware of the limitations of these output metrics, which could lead to the watermelon effect: green (on target) on the outside, red (failing) on the inside.

In a B2B context there are other reasons to stick to old project metrics: customers simply require them. The fact that these metrics still have merit, does not mean nothing has changed. Portfolio management practices have evolved to favor lean, value-stream oriented approaches, especially in B2B software delivery and even in manufacturing.    

From research and observations collected from clients past and present, we describe five insights that are shaping the new field of lean portfolio performance measurement:   

1. Don’t judge portfolio performance through one lens only 
2. It’s not just about financials! 
3. Leading indicators for actionability 
4. Management by Objective: focus on what really matters. 
5. Invest in getting measurements right! 

Close one eye, stick a finger on your left hand out in front of you and try to land the index finger on your right hand exactly on top of it. We all know this game and the outcome.

Similarly, managing complex portfolios requires at least a dual-lens approach to nail (pun intended) portfolio performance. In this context, the metaphor of inspecting both individual goods (lens one) and the factory that produces them (lens two) aptly illustrates the importance of focusing on both specific project outcomes and the overall value-generating capability of the organization producing them.   

• The Project Lens, Project Success Metrics (Individual Goods Produced): This lens is the one we know from project portfolio management approaches and focuses on the quality and effectiveness of each specific project or initiative. It's akin to inspecting each product that comes off the production line and that feeling of success we celebrate following a major milestone.  

• The Organization Lens, Value Stream Metrics (The Factory Producing Them): In contrast, this lens offers a panoramic view of the organization. It evaluates the efficiency, effectiveness and value generated by the operational system, much like assessing the performance of a factory. It’s that feeling when we look each other in the eye at the end of the month or quarter and are proud of the impact we made as a team. 

As a part of its agile transformation, European multinational technology organization, Amadeus IT Group learned to apply both lenses in performance measurement and portfolio governance. The value stream metrics have surpassed the importance of the combined set of project success metrics.

By employing both lenses in decision-making, organizations like Amadeus avoid optimizing one at the expense of the other. Instead, it leads management to the slightly more complex but far more value-adding work of managing the trade-offs between them. Which brings us to the next insights.  

We all love our financial metrics as they represent the ultimate indicator of business success. In the Project Lens we often use Net Present Value (NPV), Internal Rate of Return (IRR) or payback period. With the Organization Lens we tend to manage performance based on the profit and loss. Ideally, we would even be able to approximate this to a revenue / profit per employee. But now, with the Organization Lens, we also have good reason to measure beyond financials, to those indicators predicting the likelihood of future success and the organization’s ability to sustain it. Look through your Organization Lens and see how well your organization scores on the following categories:   

1. Value metrics: These metrics evaluate the overall value delivered by a project, encompassing financial returns, customer satisfaction, market impact, and other qualitative and quantitative benefits. These also include product metrics your customers use to evaluate the value of your company’s offer to them. For a machine manufacturer, this will include machine throughput, maintainability, and total cost of ownership. In product development, all these metrics do a better job to inform on solution intent than mere scope-agreements do. To portfolio decision makers, the abstraction to a financial number merely hides crucial assumptions. Including customer and product metrics allow them to challenge assumptions before and if combined with an iterative development method throughout the entire product development process.   
2. Flow metrics: Flow metrics assess the efficiency and effectiveness of processes and workflows, measuring how smoothly and quickly projects progress from inception to completion. Prevalent in lean manufacturing, these metrics have been successfully translated to the world of software. Shell’s Enterprise Cloud Platform uses them systematically to evaluate productivity over time, feature time-to-market and team happiness as a leading indicator for employee engagement. It gives leaders the insight needed to intervene, mitigate risks, and manage identified bottlenecks that hamper flow.   
3. Compliance metrics: compliance metrics in the way we define them focus on the compliance aspects related to running a business, managing a solution lifecycle, or conducting a project. Semi-government organizations are often required to measure and track CAPEX/OPEX-spend to a high level of detail for their development portfolio, but so do many of their commercial counterparts. an emerging compliance metric is found in in relation to ESG goals. The terms agreed to in customer contracts also fit into this category.  

The first two insights help to broaden our view of metrics. Many of us stick with the measurement systems we know and are comfortable with, and for good reason; to help them guide your business you need to understand what you are looking at, how to use them to effect and understand the possible pitfalls.

We hope these first to insights will lead to a more holistic understanding of ‘portfolio performance’. With such a wide set of metrics to choose from comes the next challenge. How can we make these actionable? What should we really pay attention to?  

To know more read our next blog “Stop looking in the rear-view mirror”.