Agile Metrics in Practice #6: Principles for using metrics

A series for self-managed agile delivery teams wanting to continuously improve and stay ahead.

Excellence is built from the habit of continuously learning and improving over time. With relevant indicators that measure team performance in the delivery cycle, Engineering Leaders can better gauge company-wide progress, bridge the business/delivery divide and enable continuous improvement over time. Forward looking leaders and teams are beginning to embrace data-driven ways of working and learning not just to stay ahead, but to thrive. Here are the 12 principles we’ve learn’t to apply when building a culture of data-driven software delivery:

1. Get everyone to the table

Ensure representation from your cross-functional team is at the table to agree on what matters most.  However you configure your team, there needs to be the voice of the customer, product, engineering, and ideally someone from your practices such as Scrum Master or Agile Coach.

2.Embrace a ‘growth mindset’ to continuously learn and improve 

Growth is the key to success. Using data allows teams to improve in the direction that best delivers positive outcomes. Self-managed teams become truly empowered when they are given the tools and information for making decisions at the coalface. Metrics are a lagging indicator; use them to find improve how you to do things in the future rather than punish the past. Adopting a Growth Mindset should be front and centre.

3. A measure is not a target

Welcome to Goodhart’s law: when a measure becomes a target, it ceases being a measure.  Setting performance goals based on metrics can dampen morale and set teammate against teammate.  Instead of promoting collaboration, it leads to cross-team competition and gaming of metrics to look better.

4. Measure Process, Product and Team success separately

Just because customers are delighted with their experience of the product doesn’t necessarily mean that the team isn’t at risk of collapsing after the next release or that your agile practices are embedded, sustainable and maturing. Measuring your agile workflow practices present a picture of cadence, quality metrics showcase product success and Team metrics to give insight into cultural health.

5. Be consistent

When your team crosses over multiple teams or organisational structures, it’s important to have a consistent view of metrics for trends (not measuring team against teams). These can act as guardrails, provide freedom for rhythm and cadence and empower teams to self-direct their learning.

6. Define relevant delivery metrics from the beginning

Thinking about delivery metrics at the start of your journey helps you focus on what success looks like. They allow you to track progress and course correct along the way.

7. Frame data collection around questions you want to answer

Get alignment on the questions you want to answer, their relative priority and perceived degree of impact on your success goals and practice performance.

8. Avoid using delivery metrics to punish or reward, this will foster ‘gaming the system’

People respond strongly to the incentives on offer. It is very easy to use data to drive the wrong outcomes; bonuses based on lines of code written has entered folklore as one of the worst engineering incentives. Use supporting metrics so that if one metric is gamed, another will reflect it.

9. Observe trends and patterns over time

The more detail a team has, the better equipped they will be to understand and react to the trends. When you find negative trends, determine solutions that have measurable outcomes.

10. Metrics don’t replace thinking or time spent with people

Data needs interpretation. Engaging teams with healthy discussion to uncover the story behind the data builds trust and understanding.

11. Build space for teams to reflect on their practices and set their own goals on where and how to improve

Teams that have real ownership over their goals deliver well. It is important that teams understand their own performance and set meaningful and achievable goals. Data is key to setting effective goals.

12. Visualise your data to communicate with influence

When it comes to aligning stakeholder expectations, people outside the team (like leadership teams) typically need rollup data or creative visualisations so they don’t require a history lesson to understand what they’re seeing.

13. Data is used by teams for teams, not on teams

Agile delivery teams are best empowered with access to realtime insights so they can own their outcomes, their practices and their performance.  Decisions to focus on areas of improvement should first come from within the team, rather than from ‘on-high’.

14. Context, composition and character make teams unique, and often incomparable

When the spirit of continuous improvement is truly embraced, agile delivery teams will compare themselves with one another only to learn how they can level up their practices.

15. Expand your visibility beyond bias and what’s obvious

Survivor bias is the concentration on people or things that made it past some selection process whilst overlooking those that did not typically because of their lack of visibility. This can lead to false conclusions in several different ways.  Avoid the tunnel vision that comes from only looking at one metric. View a situation from multiple angles, a picture enriched with insights from several metrics.

Use supporting metrics so that if one metrics is gamed, another will support it

One-eyed, narrow in outlook

___________

These are the principles that we have defined for ourselves as we work toward building our culture of continuous improvement.  We’ve published them here to support you in getting started to build the right foundations and to ensure that metrics are used responsibly in your team.

Umano is on a mission to help self-directed teams succeed by providing real-time feedback with data-driven insights that help agile teams to continually improve and stay ahead.

Sign up here to access your complimentary Umano account and see how your team’s agile sprint practices are tracking.

Join our slack community to learn from others and be a part of the data-driven movement to improving team performance.