Measuring Engineering Velocity
Measuring Engineering Velocity
How to measure and improve engineering performance in your organisation.
Every business and team wants to improve and perform better, and software engineering is a big expense for any business. So, it’s understandable that every business leader aims to improve efficiency and ship value to customers faster and more predictably. In addition to businesses wanting to “go fast”, engineers also want to work in productive and effective teams. Engineering teams are happiest when they’re consistently delivering value and winning.
Low velocity means work takes a long time to complete, which can frustrate stakeholders, customers, and even your engineering teams. If your team consistently misses deadlines, it's a clear signal that you need to invest in understanding and debugging engineering velocity.
When discussing engineering performance, the primary concern is usually: “How can we expedite the delivery of value to our customers whilst maintaining quality?”. When people talk about engineering velocity, they usually mean, “How fast is work being done?” We will use this definition here. People always ask that question to solve the next problem: removing obstacles or bottlenecks that prevent fast, reliable delivery.
In summary, the goal of talking about and measuring engineering velocity is:
Understand performance over time so that you can make informed decisions.
Take action to remove obstacles and bottlenecks so that teams can deliver faster and more predictably.
Be Cautious About Metrics
Every team, service, and business is unique, so there are no standard benchmarks for velocity. When using metrics, it's crucial to understand their purpose and what you hope to achieve from measuring them.
Good engineering metrics are useful for 3 reasons:
They enable you to measure over time, allowing you to see progress and regressions.
They act as a “guardrail” to take action if things go off track.
They can inform decision-making around investments that lead to improvements.
Measuring is not useful if you measure but take no action based on the insights.
Metrics that aim to measure “velocity” are typically incomplete proxy indicators, so they should be treated with caution. If a project takes a long time to complete, it might be because it’s a big, complex project that is supposed to take a long time. That’s perfectly OK. You must be careful when interpreting the data.
A final warning is not to use any velocity measure to compare teams since it will be inaccurate and lead to incorrect assumptions. Instead, use velocity as a diagnostic metric to improve processes and remove bottlenecks, which will make stakeholders, customers, and engineers happier.
Lead Time, Cycle Time, And Lead Time For Change
Three similar-sounding metrics measure how long work takes in a software team: lead time, cycle time, and lead time for changes. Each serves a different purpose and can be useful depending on your goal. Each focuses on debugging a particular part of your software delivery lifecycle.
The importance of lead time and cycle time is obvious: to increase productivity and efficiency, reduce lead time and cycle time as much as possible. The faster you can deliver value, the higher your chances of success and the happier your customers and stakeholders will be.
Lead Time: Lead time measures the time from work being requested until it is completed. Tracking lead time can help you understand how quickly the team can respond to new work and deliver value.
Cycle Time: Cycle time measures the time it takes for a task or user story to move from the start of the development process to completion. It indicates how much time the team devotes to an item of work. When any team member starts working on the task, cycle time continues until completion. You can assess your team’s development velocity by tracking the average cycle time.
Lead Time For Changes: Lead Time For Changes measures the time between code being committed and its availability in production. It is one of the five DORA metrics (Deployment Frequency, Lead Time For Changes, Change Failure Rate, Mean Time To Recovery, and Reliability).
7 Metrics For Measuring Engineering Velocity
There are 7 common metrics for measuring development team velocity. These metrics can be useful for understanding and debugging problems with the software delivery process, but they can also be misused and should be treated with care. None of these metrics accurately measure velocity, but they can be useful proxy indicators.
Story points completed per sprint: Velocity in Agile software development has a specific meaning: How many work units were done in a given period? Usually, this equates to the question, “How many story points did we complete in the sprint?” This potentially dangerous metric should only be used inside a team for planning. Story points are an abstract and imprecise measure, so you cannot use this metric to compare or communicate performance outside of teams.
Number of X completed: A simple metric can be the number of tasks, bugs, and projects completed over time. This approach works best for measuring specific areas of work where the tasks are of similar effort and complexity, such as customer support and other routine work.
Cycle Time (CT): Cycle time measures the time it takes for a task or user story to move from the start of the development process to completion. By tracking the average cycle time, you can assess the team’s development velocity and identify bottlenecks in the process.
Lead Time (LT): Lead time measures the time from work requested (e.g by a customer or stakeholder) until completion. Tracking lead time can help you understand how quickly the team can respond to new work and deliver value.
Deployment Frequency (DORA): The frequency at which code is deployed can indicate development velocity. Frequent releases may indicate a high development velocity, while infrequent releases could signal bottlenecks or other issues.
Lead Time For Changes (DORA): The time it takes for a commit to go into production. This metric focuses more on the release process.
Mean Time To Resolution (MTTR): MTTR measures the average time it takes for a team to resolve an issue, such as a bug or defect. A low MTTR can indicate a high development velocity and the ability to address issues quickly.
How To Improve Velocity
Improving velocity starts by asking questions and understanding current bottlenecks. When you understand the problems, you still need to make tradeoffs and take action. For example, if you have a small team, you can’t expect to ship projects quickly if the team spends 50% of its time dealing with customer support requests and solving operational emergencies. There is always a tradeoff to be made.
Metrics can drive understanding, which enables you to take action to fix problems and improve processes. They can also monitor your progress and the impact of changes you make.
Velocity is not just about increasing activity (coding time, code review time, deployment frequency). Often, problems with velocity can be caused by people and processes. For example, waiting a long time to get approvals on a project, slow decision-making, rework due to misalignment or changing scope, or delays caused by engineers having to context switch between tasks (e.g on-call, customer support).
Here are some ways to handle velocity problems.
Ask The Team
A good starting point for debugging velocity issues is asking your team some basic questions directly or via a survey - “Where are we spending most of our time?” or “What’s slowing down delivery?”. These answers will be powerful because the team likely already knows the big problems. Things like long wait times on getting approvals from managers, waiting for code reviews, context switching, slow deployment process, and unclear requirements lead to long planning cycles.
Once you identify what’s slowing down engineering work, take action on the most pressing issues and repeat.
Use SLAs
Setting SLAs is a powerful and simple way to drive improvements in routine work, such as customer support, bug fixes, and operational incidents. SLAs clarify expectations for how quickly tickets and tasks should be resolved.
Example: Time To First Response < 24 hours - “We will provide an initial response to all customer requests within 24 hours”.
SLAs can be useful because they clarify expectations for teams and customers while being generally useful metrics that set the standard for routine work.
Some examples are time to mitigate high-severity security issues, time to first respond to a customer request, time to resolve a customer support issue, and time to fix a high-severity operational issue.
Find The Bottlenecks
Once you have metrics to identify how fast you are moving as a team, you can use that data to generate insights into potential problem areas.
Your delivery process will consist of multiple stages: planning, approval, designing, coding, testing, and deployment. Each stage can have different problems and constraints that you can improve.
Look at what the data is telling you. Are you spending weeks in a planning state? Is your deployment taking a long time? Find the problem areas based on data and talk to engineers.
Take action to overcome the bottleneck and repeat the process periodically.
Summary
Measuring engineering velocity can help teams find bottlenecks and improvements in their process.
Teams, stakeholders, and customers always want you to “go faster.” Metrics can help you determine how.
Cycle time is the most useful metric for calculating velocity since it includes all the human and technical elements in a delivery system that can lead to low velocity.
You can set SLAs to clarify the team's and customers' expectations regarding velocity (e.g., “We will respond to all customer requests within 24 hours”).
Get In Touch
I would love to hear from you! If you enjoy my writing and want to connect: