Engineering Metrics That Drive Improvement, Not Just Reporting
How mature engineering teams use meaningful measurement to improve delivery, remove bottlenecks and make better decisions. “What gets measured gets managed” is a familiar management principle. “What you don’t measure, you can’t manage” captures a related truth: without visibility, it is difficult to understand performance, identify problems or determine whether an improvement has worked. But measurement is not the same as management. And more measurement does not automatically lead to better outcomes. Engineering teams can generate extensive data from source control, delivery pipelines, issue trackers and production systems. They can track dozens of KPIs and produce detailed dashboards, yet still struggle to deliver predictably, maintain quality or understand why work gets delayed. The challenge is not simply to collect data. It is to determine what is worth measuring, what the evidence tells us, and what we should do differently as a result. For engineering leaders, tech leads and senior engineers, this is an important part of building an effective engineering organisation. 1. Start with established frameworks, not more KPIs There is no need to invent a new measurement framework for every engineering team. Established approaches provide useful starting points, each with a different emphasis. DORA: Software delivery performance DORA’s current framework uses five software delivery performance metrics to assess throughput and instability: Together, these measures help teams understand how effectively they deliver software and how reliably those changes perform. SPACE: A broader view of developer productivity The SPACE framework recognises that developer productivity cannot be understood through a single measure. It considers five dimensions: The frameworks complement each other. DORA provides a structured view of software delivery performance, while SPACE encourages a broader understanding of developer productivity, including the working environment and collaboration that influence outcomes. Neither framework is a universal scorecard, and adopting them does not mean measuring every dimension at once. For further reading, see DORA’s software delivery performance metrics and the SPACE framework research from Microsoft Research. 2. Start with the problem, not the dashboard Before introducing a metric, ask two questions: What problem are we trying to solve? What decision will this information help us make? Consider a team that consistently misses delivery commitments. Adding more KPIs will not necessarily explain why. The team first needs to understand where work is delayed and what is contributing to the problem. Depending on the situation, useful areas to investigate might include: These are investigation areas, not a mandatory KPI list. The relevant measures depend on the product, the team’s responsibilities and the decisions that need to be made. A practical approach is to: If the evidence already points to a likely cause, investigate it rather than adding another dashboard. 3. Avoid vanity metrics and analysis paralysis Every metric has a cost. Someone must collect or maintain the data, interpret it, discuss the findings and decide what to do next. When teams accumulate too many KPIs, they risk analysis paralysis: spending so much time reviewing information and debating interpretations that meaningful action is delayed. The volume of data can also obscure the few signals that deserve attention. Another risk is the use of vanity metrics: measures that appear to demonstrate productivity or progress but do not reliably establish meaningful outcomes. Consider some common engineering activity measures: These measures are not inherently useless. Their value depends on the question being asked and how the information is interpreted. They become misleading when used in isolation to rank engineers or draw conclusions the data cannot support. For example, comparing individual story points can encourage inflated estimates, discourage engineers from helping one another and undermine the planning process. An engineer handling a difficult architectural problem or mentoring colleagues may contribute substantially without closing many tickets. A more mature approach is to: The goal is not to measure everything. It is to measure enough to understand performance and make better decisions. 4. Automate measurement, not judgement Manual metric collection through spreadsheets, recurring status reports and presentations can create unnecessary work. It can also introduce inconsistencies, delays and reporting errors. Where practical, automate the collection, calculation and presentation of engineering metrics using data already available in the tools teams use. For example: Automation can make data more consistent and timely while reducing the administrative burden on engineers and managers. However, automation requires judgement of its own. DORA itself cautions against focusing on measurement at the expense of improvement. It can be more valuable to begin with existing data and team discussions than to invest heavily in integrations before understanding the problem. Automation should reduce reporting effort and improve decisions. It should not turn every aspect of engineering work into a surveillance mechanism. 5. Investigate bottlenecks, not just symptoms Consider a team that repeatedly misses delivery dates. A superficial assessment might conclude that engineers need to work faster. A more useful investigation follows the work through the delivery process. The team might discover that: Each finding suggests a different intervention: A targeted measure can establish a baseline and help determine whether the intervention is working. If code review delays are the suspected bottleneck, for example, the team could examine review waiting time and the age of outstanding reviews before and after changing its practices. The process is straightforward: A metric is most useful when it helps a team move from observing a problem to understanding and addressing its cause. 6. Improve team performance before blaming individuals When delivery performance deteriorates, some leaders turn to individual activity measures to determine who is underperforming. This can lead to the assumption that poor team results must be caused by individuals who are not working hard enough. Engineering performance is influenced by many factors, including: Looking at individual activity alone can cause leaders to overlook these constraints. The first objective should be to understand and improve the team’s delivery system, not to find someone to blame or remove. Blameless postmortems offer a useful principle. They focus on understanding what happened, the contributing conditions and the changes that could prevent recurrence, rather than beginning with blame. See Google’s SRE guidance