Engineering Metrics That Drive Improvement, Not Just Reporting
How mature engineering teams use meaningful measurement to improve delivery, remove bottlenecks and make better decisions.
“What gets measured gets managed” is a familiar management principle. “What you don’t measure, you can’t manage” captures a related truth: without visibility, it is difficult to understand performance, identify problems or determine whether an improvement has worked.
But measurement is not the same as management. And more measurement does not automatically lead to better outcomes.
Engineering teams can generate extensive data from source control, delivery pipelines, issue trackers and production systems. They can track dozens of KPIs and produce detailed dashboards, yet still struggle to deliver predictably, maintain quality or understand why work gets delayed.
The challenge is not simply to collect data. It is to determine what is worth measuring, what the evidence tells us, and what we should do differently as a result.
For engineering leaders, tech leads and senior engineers, this is an important part of building an effective engineering organisation.
1. Start with established frameworks, not more KPIs
There is no need to invent a new measurement framework for every engineering team. Established approaches provide useful starting points, each with a different emphasis.
DORA: Software delivery performance
DORA’s current framework uses five software delivery performance metrics to assess throughput and instability:
- Change lead time: How long it takes for a change to move from version control to production.
- Deployment frequency: How frequently changes are deployed to production.
- Failed deployment recovery time: How long it takes to recover when a deployment requires immediate intervention.
- Change fail rate: How often deployments require immediate intervention.
- Deployment rework rate: How much deployment activity is unplanned work resulting from production incidents.
Together, these measures help teams understand how effectively they deliver software and how reliably those changes perform.
SPACE: A broader view of developer productivity
The SPACE framework recognises that developer productivity cannot be understood through a single measure. It considers five dimensions:
- Satisfaction and well-being
- Performance
- Activity
- Communication and collaboration
- Efficiency and flow
The frameworks complement each other. DORA provides a structured view of software delivery performance, while SPACE encourages a broader understanding of developer productivity, including the working environment and collaboration that influence outcomes.
Neither framework is a universal scorecard, and adopting them does not mean measuring every dimension at once.
For further reading, see DORA’s software delivery performance metrics and the SPACE framework research from Microsoft Research.
2. Start with the problem, not the dashboard
Before introducing a metric, ask two questions:
What problem are we trying to solve? What decision will this information help us make?
Consider a team that consistently misses delivery commitments. Adding more KPIs will not necessarily explain why. The team first needs to understand where work is delayed and what is contributing to the problem.
Depending on the situation, useful areas to investigate might include:
- Flow of work: Where does work spend most of its time waiting? How often does it return for clarification or rework?
- Quality and reliability: Are defects recurring? Are releases creating avoidable operational issues?
- Engineering friction: Are code reviews, test execution, build pipelines or development environments slowing the team down?
- Predictability: How often does actual delivery differ from expectations, and what explains the difference?
- Sustainability: Is unplanned work repeatedly displacing planned improvements? Is operational support consuming too much engineering capacity?
These are investigation areas, not a mandatory KPI list. The relevant measures depend on the product, the team’s responsibilities and the decisions that need to be made.
A practical approach is to:
- Define the problem. Be clear about the outcome that needs to improve.
- Identify the evidence. Determine what information would help explain the problem.
- Use existing data first. Avoid building a new reporting system when the information is already available.
- Investigate the findings. Look for contributing factors instead of assuming the first explanation is correct.
- Take action and review. Make a targeted change and check whether it improves the intended outcome.
If the evidence already points to a likely cause, investigate it rather than adding another dashboard.
3. Avoid vanity metrics and analysis paralysis
Every metric has a cost. Someone must collect or maintain the data, interpret it, discuss the findings and decide what to do next.
When teams accumulate too many KPIs, they risk analysis paralysis: spending so much time reviewing information and debating interpretations that meaningful action is delayed. The volume of data can also obscure the few signals that deserve attention.
Another risk is the use of vanity metrics: measures that appear to demonstrate productivity or progress but do not reliably establish meaningful outcomes.
Consider some common engineering activity measures:
- Lines of code written: More code does not necessarily mean more value. A simpler solution may deliver a better outcome with less code.
- Number of commits: Commit counts describe certain types of activity, not the quality or impact of the work.
- Pull requests created: Useful for understanding workflow patterns, but higher volume does not necessarily indicate greater productivity.
- Tickets closed: Can help track work, but counts alone do not account for differences in complexity, effort or outcomes.
- Story points completed by an individual: Story points support relative estimation and team planning. They are not standardised units of individual productivity.
These measures are not inherently useless. Their value depends on the question being asked and how the information is interpreted. They become misleading when used in isolation to rank engineers or draw conclusions the data cannot support.
For example, comparing individual story points can encourage inflated estimates, discourage engineers from helping one another and undermine the planning process. An engineer handling a difficult architectural problem or mentoring colleagues may contribute substantially without closing many tickets.
A more mature approach is to:
- Maintain a small set of core measures aligned with delivery, quality and reliability.
- Introduce additional diagnostic measures when a specific problem needs investigation.
- Avoid collecting data simply because a tool makes it available.
- Review whether each measure continues to inform decisions.
- Retire measures that no longer provide useful insight.
The goal is not to measure everything. It is to measure enough to understand performance and make better decisions.
4. Automate measurement, not judgement
Manual metric collection through spreadsheets, recurring status reports and presentations can create unnecessary work. It can also introduce inconsistencies, delays and reporting errors.
Where practical, automate the collection, calculation and presentation of engineering metrics using data already available in the tools teams use.
For example:
- Source control platforms can provide data about code changes, pull requests and review activity.
- CI/CD pipelines can capture build results, test failures, deployment frequency and release outcomes.
- Issue trackers can help reveal work in progress, ageing tasks, cycle times and workflow delays.
- Observability and incident management tools can provide evidence about service reliability, incidents and recovery.
Automation can make data more consistent and timely while reducing the administrative burden on engineers and managers.
However, automation requires judgement of its own.
- Automated does not mean meaningful. A misleading metric remains misleading even when generated perfectly.
- Validate the data. Ensure events are captured consistently and metric definitions are understood.
- Provide context. A dashboard should help explain what is happening, not simply display numbers.
- Keep the solution proportionate. Use existing integrations and capabilities before introducing another platform.
- Include qualitative evidence. Tool-generated data cannot fully explain unclear priorities, collaboration challenges, developer experience or why a problem occurred.
DORA itself cautions against focusing on measurement at the expense of improvement. It can be more valuable to begin with existing data and team discussions than to invest heavily in integrations before understanding the problem.
Automation should reduce reporting effort and improve decisions. It should not turn every aspect of engineering work into a surveillance mechanism.
5. Investigate bottlenecks, not just symptoms
Consider a team that repeatedly misses delivery dates.
A superficial assessment might conclude that engineers need to work faster. A more useful investigation follows the work through the delivery process.
The team might discover that:
- Changes spend several days waiting for code review.
- Reviews are delayed because only a few engineers understand critical components.
- Automated tests are unreliable, so engineers spend time investigating false failures.
- Requirements frequently change after implementation has started.
Each finding suggests a different intervention:
- Improve review practices and clarify responsibilities.
- Share knowledge and reduce dependence on a small number of specialists.
- Invest in test reliability.
- Involve product stakeholders earlier to clarify requirements.
A targeted measure can establish a baseline and help determine whether the intervention is working. If code review delays are the suspected bottleneck, for example, the team could examine review waiting time and the age of outstanding reviews before and after changing its practices.
The process is straightforward:
- Identify a problem.
- Form a hypothesis about its cause.
- Find evidence that supports or challenges that hypothesis.
- Make a targeted improvement.
- Review the results and adjust.
A metric is most useful when it helps a team move from observing a problem to understanding and addressing its cause.
6. Improve team performance before blaming individuals
When delivery performance deteriorates, some leaders turn to individual activity measures to determine who is underperforming. This can lead to the assumption that poor team results must be caused by individuals who are not working hard enough.
Engineering performance is influenced by many factors, including:
- Technical complexity and architectural constraints.
- Dependencies between teams.
- Unclear priorities or changing requirements.
- Tooling, development environments and test reliability.
- Knowledge gaps and dependence on particular specialists.
- Work allocation, review practices and operational interruptions.
Looking at individual activity alone can cause leaders to overlook these constraints.
The first objective should be to understand and improve the team’s delivery system, not to find someone to blame or remove.
Blameless postmortems offer a useful principle. They focus on understanding what happened, the contributing conditions and the changes that could prevent recurrence, rather than beginning with blame. See Google’s SRE guidance on postmortem culture.
The same approach can inform delivery reviews and engineering retrospectives. Ask:
- What happened, and what evidence do we have?
- Where did the process or system make success difficult?
- Were expectations, priorities and responsibilities clear?
- Did people have the skills, information and tools they needed?
- What changes would reduce the likelihood of the problem recurring?
This does not mean avoiding individual accountability. There may be cases where an engineer lacks the required knowledge or skills, needs additional support, or consistently fails to meet reasonable expectations.
The response should follow an objective assessment of the circumstances. It might involve training, mentoring, clearer expectations, improved tools, changes in responsibilities or, where justified, formal performance management, including a performance improvement plan.
The distinction is between addressing an identified problem and assuming the individual is the problem because the team’s numbers are poor.
Blameless does not mean accountability-free. It means understanding the problem before deciding what action is appropriate.
This approach supports better engineering outcomes and fairer individual performance decisions.
7. Build a continuous improvement habit
Measurement creates value when it becomes part of how a team learns and improves, rather than a reporting exercise performed for management.
A lightweight improvement cycle can help:
- Identify the problem. Define the outcome that needs to improve.
- Establish a baseline. Use existing data and team discussions to understand the current situation.
- Form a hypothesis. Identify the most plausible constraint and the evidence needed to test it.
- Make a targeted change. Agree on an improvement and assign ownership where appropriate.
- Review the result. Check whether the intended outcome improved and whether there were unwanted effects.
- Learn and adjust. Keep what works, revise what does not and investigate further only where necessary.
This approach works at every engineering level:
- Engineering leaders can address systemic constraints across teams and improve delivery outcomes.
- Tech leads can improve code review practices, testing, technical debt management and knowledge sharing.
- Senior engineers can investigate recurring defects, remove development friction and evaluate technical improvements.
The process does not require a complex dashboard or a large measurement programme. It requires a clearly understood problem, relevant evidence and a commitment to act on what is learned.
Final Thoughts
“What you don’t measure, you can’t manage” is a useful reminder that effective management needs visibility. But it should not become an argument for measuring every activity or turning every available data point into a KPI.
Engineering maturity is not demonstrated by the number of metrics an organisation tracks. It is demonstrated by how effectively it uses evidence to improve the way it builds, delivers and operates software.
That means:
- Measure outcomes and constraints, not just activity.
- Use established frameworks thoughtfully. DORA and SPACE provide useful perspectives, but neither replaces context or judgement.
- Automate data collection where practical to reduce reporting effort and improve consistency.
- Investigate bottlenecks before prescribing solutions.
- Avoid vanity metrics that reward activity without demonstrating value.
- Improve the system before blaming individuals, while retaining fair and evidence-based accountability.
- Turn findings into action and review whether those actions improve results.
Ultimately, a good engineering measurement system should make it easier for teams to understand problems, make informed decisions and deliver better outcomes. If the organisation spends more time reporting performance than improving it, the measurement system itself needs to be reconsidered.
Measure to understand. Analyse to decide. Act to improve.