I had most of this post written (I actually started it in June but several things happened and I was not able to finish it until today!) when McKinsey released their take on Developer Productivity with their "Yes you can measure software developer productivity" post. Responses came from different people and it soon became a hot topic! My favorite answers are from Kent Beck and Gergely Orosz (here are the links: part 1, part 2 - Kent and part 2 - Gergely in case you want to read their as always great point of view). I won't lie, there were thoughts in my head like "dang, if only I had finished my post in June". Since I can't travel back in time (yet) and because I think this was one of the hottest topics this year the only thing left for me was to finish the post!
Every now and then someone brings the need to understand better (or measure) developer productivity. The reasons sometimes sound legit and with good intent like "increase our overall company productivity" or "promote the best/most performing engineers" other times it just sound like micromanagement "Mrs Stone have written 300 lines of code this week and Mr Smith has written 500". As always, the devil is in the details!
The first little detail we need to consider is actually what will be measured and oh boy this is not little and also not easy. Just as example, in 2020 DORA metrics were released, SPACE followed in 2021 and now in 2023 we have DevEx (and don't forget McKinsey that we already mentioned).
Understanding and tracking the suggested metrics with any of those approaches will certainly bring benefits to your organization and somehow increase the productivity (if you are lucky) but my answer on this topic always go around having a deep understanding on what actually happens from the moment an idea is generated (is it in the discovery sessions with Design & Product teams? Are Software Engineering participating on it?) to the final moment when the code hits production. Once you have this understanding the next step is to pick the major pinpoints for your case and create goals with metrics around your specific "pain". Example: if newcomers are lost with different teams documentation spread in Confluence, Notion, GitHub, SharePoint and the ramp-up necessary for them to onboard a new codebase is steep consider "time to first commit" (the amount of time your new team member takes to make his first commit to production in a code base) as a metric to improve.
This will for sure take some time and a dedicated team to work on what you will find but the key message here is: I really mean your case, those findings are different from company to company and sometimes from team A to team B in the same company! If you don't have the steps/processes mapped out it will be very hard to address your major problems and on defining what should be measured to increase the productivity for your case.
I've used in the past Customer Journey Maps (putting the developers as customers) and Empathy Map Canvas (treating developers as... well...how can I say... people! treating them as people and listening to them!) to tackle this task and it is fascinating what you get as a result. Also, in the process you will hopefully find that lines of codes written or story points done per sprint won't give you any indication on productivity.
If you are still stuck with the "where should I start" question I highly recommend taking a look at some of the articles Google has shared with the rest of the world on the subject, after all they created a team of researchers dedicated to understanding engineering productivity before it was cool to talk about it. This part of the Measuring Engineering Productivity article is a punch in the face for many companies in so many different ways:
Before measuring productivity, ask whether the result is actionable, regardless of whether the result is positive or negative. If you can’t do anything with the result, it is likely not worth measuring.
The hardest detail to account for on measuring something and rewarding people on any measure is avoiding the human nature to game the system. This phenomena was captured by Charles Goodhart and became the Goodhart's law:
When a measure becomes a target, it ceases to be a good measure
Tell your team that individuals will be measured by ticked closed during a quarter and soon if your customer wants to ask a question a ticket needs to be raised, tasks that require two steps suddenly start appearing in two separate tickets and what happens at the end of the quarter? The number of tickets that your entire team received in the previous quarter suddenly doubled (or tripled) in the current quarter with the new policy in place.
This is also hard because, well, if you look purely at the number of lines of codes written, closed tickets or pull requests (PR) you are not getting the full story. It is the equivalent of measuring Sales Representatives by the number of e-mails they send or Managers by the number of meetings they attend!
Still on the fence? Let's look at the following fictitious piece of Mr Smith's-500-lines-of-code-per-week details in a fictitious programming language:
myVariable = 1_
+ 1_
- 2;Now Mrs Stone:
myVariable = 0;It is a simple example but the kind of behavior you will get when you implement this type of measurement and start rewarding on it: people won't optimize for what is the right thing to do instead they will try to survive.
"This is silly because Mr Jones' code would never pass in our code review process" you might say. Thanks for that! You probably haven't noticed but you just reinforced some of the arguments in this post! Code reviews, Pair programming, mentoring, design documents, pipeline adjustments, automation, participation in discovery sessions are just a few examples of things that some individuals do without writing a single line of code or completing a PR that have an incredible positive impact on the overall team productivity (and your codebase!) that should be taken in consideration. A Good Engineering/Tech Manager probably knows that but not all Engineering Managers are good so it is always worth saying out loud!
I know that we have not touched the "I want to measure the productivity so that I have evidence to promote the best/most performing engineers" problem yet. My advice on this is very simple but it requires discipline and some effort: Take notes as you go. You and your manager (doesn't matter how good they are) won't remember everything you did in the past quarter, almost impossible to remember all the details and impact on a yearly-basis-cycle-performance-review. I tend to go on a resume-linkedin-like notes "Worked extensively collaborating with multiple teams (Security, Customer Success and Identity & Access Management) doing training, making the transition plan and promoting the adoption of the new platform tool that saved 200K USD for the company in the last quarter". Here are a few tips:
- If you are a Manager, focus on what matters for your team (getting tired of saying this but will say it again! Lines of codes written doesn't matter!): business impact, customer impact, amount of money saved, system/platform performance, mentoring, providing innovative solutions, etc, etc, etc. ChatGPT is probably better than me on writing this stuff. Yep, it takes time and effort! Don't forget your #1 job is taking care of your people!
- If you are a person (yes, developers and managers are also considered people), focus on what were your contributions towards the Year/Quarterly goals: business impact, amount of money saved, system/platform performance, mentoring, providing innovative solutions, etc, etc, etc. ChatGPT is probably better than me on writing this stuff. Yep, it takes time and effort!
The final detail to consider in measuring developer productivity is: at the end of the day software engineers are humans (for now... we see you Artificial Intelligence (AI) 👀) and if you ask 10 engineers with the same tools about their perceived productivity you likely will get 12 different answers. This again is hard to accomplish and will require due diligence on your part so be extra careful not to jump in the hype train of new tools that most likely someone is trying to sell you or to change all developers machines to a newer model based on a YouTube video someone saw comparing build times for xyz machine.
To summarize this post, there is this great quote that I found online but was not able to find the source (and also haven't done the math to see if the numbers line up):
What is the secret to becoming a 10x more productive organization?
Make 10 engineers 2x more productive. Focus on unlocking other engineers and help them solve their toughest problems
If you are serious about improving developer(s) productivity it will probably take a team to tackle the major problems for your case but the rewards will surely payoff!